ad9384c850
fix(graphql): backup quotas field typing
2023-08-30 12:03:19 +03:00
9fdc536f9f
BROKEN(backups): hooking up quotas to API fails. AutobackupQuotas needs to be an input type, but if input type, it fails because it needs to be an Output type, which is not documented
2023-08-28 18:24:29 +00:00
0c04975ea4
flx(backups): commit forgotten time.py
2023-08-28 17:24:20 +00:00
1fc47b049d
refactor(backups): clean up caps code
2023-08-28 17:23:21 +00:00
9207f5385c
feature(backups): actual finegrained quotas
2023-08-28 17:02:45 +00:00
9db717c774
style: linting
2023-08-25 19:28:37 +03:00
ada89a2494
Merge branch 'master' into fix-restores-wrt-service-status
2023-08-25 19:01:16 +03:00
1333aad57d
feature(backups): temporarily revert restore job status to created for inplace restore to run backup
2023-08-23 14:35:03 +00:00
c68239044f
feature(backups): report status text for restore jobs
2023-08-23 14:18:33 +00:00
02b03cf401
feature(backups): report the error text in a job
2023-08-23 14:02:07 +00:00
de52dffdda
refactor(backups): a better backup-related service timeout error
2023-08-23 13:55:23 +00:00
72535f8655
fix(backups): default timeout to 5 min for service starting and stopping in backup operations
2023-08-23 13:40:04 +00:00
0a852d8b50
fix(backups): consider failing services MORE and not try to stop them
2023-08-23 13:39:12 +00:00
f2c972ed5f
chore: bump version
2023-08-23 14:51:15 +03:00
d6cf2abdc2
style: remove unused imports
2023-08-23 14:51:01 +03:00
b2c7e8b73a
feature(backups): caps for autobackups
2023-08-21 12:45:31 +00:00
30b62c351a
feature(redis): compatibility with str enums
2023-08-21 11:31:29 +00:00
027a37bb47
feature(backup): remember the reason for making a snapshot
2023-08-21 11:11:56 +00:00
d621ca6449
refactor(backups): clean up unused mounting tools
2023-08-14 12:50:45 +00:00
c89f9cf89d
feature(backups): do not rely on mounting
2023-08-14 12:43:44 +00:00
69f6e62877
test(backups): more checks regarding tmpdirs and mounting
2023-08-14 11:59:44 +00:00
2c9011cc87
fix(backups): robustness against stale locks: everything else
2023-08-09 15:18:20 +00:00
0eb70e1551
fix(backups): robustness against stale locks: snapshot sizing
2023-08-09 14:46:27 +00:00
26ab7b4d7b
fix(backups): robustness against stale locks: is_initted
2023-08-09 14:27:14 +00:00
eca4b26a31
fix(backups): robustness against stale locks: backing up
2023-08-09 13:47:18 +00:00
752a0b807e
feature(backups): lock and unlock at will
2023-08-07 13:33:10 +00:00
52336b885d
fix: check if repo is initted by checking retcode
2023-08-04 14:08:23 +03:00
ff70a3588e
chore: bump version
2023-08-04 12:57:31 +03:00
e6189e4e73
Merge branch 'master' into backups-forget
2023-08-04 12:46:08 +03:00
f4263b0288
style: remove unused imports
2023-08-02 08:54:18 +03:00
f08eafc3d8
refactor(services): extract get_drive to the base Service class
2023-08-02 08:51:42 +03:00
a87889b252
refactor: Change Mail Server id from "mailserver" to "email"
...
Nix config uses "email" so we should too.
2023-08-02 07:41:55 +03:00
bba837530a
feature(backups): expose forget to API
2023-07-28 10:40:40 +00:00
88af27a8ba
chore: bump version
2023-07-28 03:20:33 +03:00
641ab26069
fix(storage): fix root device detection and ignore iso9660
2023-07-28 03:14:50 +03:00
6660e1d9d5
fix(services): ocserv and jitsi were returning wrong volumes too
2023-07-28 02:31:28 +03:00
3bff43a6bf
fix(mailserver): Location of the "location" attribute was inconsistent
2023-07-28 02:18:05 +03:00
00317cc7e4
test(backups): erase repos between tests
2023-07-26 16:52:58 +00:00
cfa7f4ae59
feature(backups): add full repo erasure capability
2023-07-26 16:45:08 +00:00
ffec344ba8
test(backups): make the test repo overridable by envs
2023-07-26 14:26:04 +00:00
aa7cc71557
feature(backups):add a function to set provider from env
2023-07-26 11:54:17 +00:00
d664fcbdc4
refactor(jobs):track 100-on-finished as a progress update
2023-07-21 13:06:18 +00:00
26488aa12f
fix(backup):force backup function to return same length ids as when getting snapshots
2023-07-21 11:11:24 +00:00
b01247bc55
refactor: remove legacy backups implementations
2023-07-20 20:11:42 +03:00
413911849d
refactor(backups): remove calls of legacy backups API
2023-07-20 20:06:39 +03:00
eafc7a412c
fix: timezone awareness of automatic backup task
2023-07-20 19:42:44 +03:00
c9d20e8efd
fix(backups): make datetimes from huey timezone aware
2023-07-20 19:39:10 +03:00
e0a5ecdd9d
style: reformat
2023-07-20 19:37:01 +03:00
95dbd4fac2
chore: bump version
2023-07-20 19:35:35 +03:00
4b575b6138
fix(graphql): duplicate imports
2023-07-20 18:25:32 +03:00
0245d629fd
refactor(backups): linting
2023-07-20 18:24:26 +03:00
2df448a4a9
fix(backups): Do not shut down the service during backup
...
We do not want the user to expirience outages during automatic backups.
Generally, they are not even needed.
We should use hooks to service-specific tasks, such as
creating the database dump, so we don't have to shut down Postgres.
2023-07-20 17:11:02 +03:00
f4ac3d29a9
feature(backup):remove code for finegrained autobackup control
2023-07-19 15:36:25 +00:00
466160dbf9
feature(backup):disable finegrain control over services to back up
2023-07-19 15:36:25 +00:00
5253780cc8
style: linting of backups module
2023-07-19 15:59:51 +03:00
3067d353d8
fix(backups): Do not remove failsafe snapshot after recovery
2023-07-19 15:59:29 +03:00
fa53264136
refactor(backup):remove unused verify arg from do_restore
2023-07-19 10:51:36 +00:00
a303e5ce37
Merge branch 'restic-rewrite' of https://git.selfprivacy.org/SelfPrivacy/selfprivacy-rest-api into restic-rewrite
2023-07-19 10:35:30 +00:00
8b504993d0
fix(backups): backup strategies were unused
2023-07-18 20:15:22 +03:00
45011450c5
feature(backup):calculate needed space for inplace restoration
2023-07-17 16:05:00 +03:00
097cf50b37
fix(servers): hopefully fix moving
2023-07-17 16:05:00 +03:00
c53f35c947
feature(servers): set default timeout of server operations to 10 min
2023-07-17 16:05:00 +03:00
b001e198bf
feature(backups): stop services before restores
2023-07-17 16:05:00 +03:00
40ad1b5ce4
feature(backups): stop services before backups
2023-07-17 16:05:00 +03:00
a7427f3cb5
test(backups): do not store the status file in backupped folders
2023-07-17 16:05:00 +03:00
86c2ae2c1f
refactor(backups): make a StoppedService context manager
2023-07-17 16:05:00 +03:00
ea4e53f826
test(backups): make delay settable per dummyservice
2023-07-17 16:05:00 +03:00
e2b906b219
test(backups): test async service start n stop simulation
2023-07-17 16:05:00 +03:00
d33e9d6335
test(backups): simulating async service start n stop
2023-07-17 16:05:00 +03:00
8e29634d02
feature(utils): a hopefully reusable waitloop
2023-07-17 16:05:00 +03:00
be95b84d52
feature(backups): expose restore strategies to the API
2023-07-17 16:05:00 +03:00
cacbf8335d
fix(backups): actually mount if asked for an inplace restore
2023-07-17 16:05:00 +03:00
65ce86f0f9
test(backups): test out that pre-restore backup plays nice with jobs
2023-07-17 16:05:00 +03:00
95e4296d0b
feature(backups): implement inplace restore strategy
2023-07-17 16:05:00 +03:00
59fe386463
feature(backups): restore strategies enum
2023-07-17 16:05:00 +03:00
02e3c9bd5e
feature(backups): forgetting snapshots
2023-07-17 16:05:00 +03:00
f361f44ded
feature(backups): check restore exit code
2023-07-17 16:05:00 +03:00
4423db7458
refactor(backups): download a copy before replacing original
2023-07-17 16:05:00 +03:00
9137536294
feature(backups): mounting a repo
2023-07-17 16:05:00 +03:00
9a28c0ebcb
refactor(backups): move syncing (non-restic) into backup utils
2023-07-17 16:05:00 +03:00
7ad5f91be1
refactor(backups): move output yielding into backup utils
2023-07-17 16:05:00 +03:00
1c28984475
feature(backups): a wrapper for rclone sync
2023-07-17 16:05:00 +03:00
2df930b9ba
feat(backups): Add backup descriptions for UI
2023-07-17 16:05:00 +03:00
2c21bd2a14
feat(backups): expose if the service can be backed up
2023-07-17 16:05:00 +03:00
21c5f6814c
style: fix styling
2023-07-17 16:05:00 +03:00
559de63221
fix(jobs): make finishing the job set progress to 100
2023-07-17 16:05:00 +03:00
ba9270755a
feat(jobs): return type_id of the job in graphql api
2023-07-17 16:05:00 +03:00
0e13e61b73
fix(services): proper backup progress reporting
2023-07-17 16:05:00 +03:00
1fb5e3af97
fix(services): cleanup a stray get_location
2023-07-17 16:05:00 +03:00
2dd9da9a96
fix(backups): register the correct tasks
2023-07-17 16:05:00 +03:00
a7d0f6226f
fix(backups): missing space in rclone args
2023-07-17 16:05:00 +03:00
e8f1f39b18
refactor(backups): rename service_snapshot_size to snapshot_restored_size
2023-07-17 16:05:00 +03:00
f804c88fa6
refactor(backups): remove the by-service getting of cached snapshots
2023-07-17 16:05:00 +03:00
6004977845
refactor(backups): rename force_snapshot_reload to force_snapshot_cache_reload
2023-07-17 16:05:00 +03:00
3551813b34
refactor(backups): merge sync_all_snapshots with force_snapshot_reload
2023-07-17 16:05:00 +03:00
ce55416b26
refactor(backups): straighten get_all_snapshots
2023-07-17 16:05:00 +03:00
16a96fe0fa
refactor(backups): delete sync_service_snapshots
2023-07-17 16:05:00 +03:00
f2161f0532
refactor(backups): privatize assert_restorable and restore_snapshot_from_id
2023-07-17 16:05:00 +03:00
cb2273323f
refactor(backups): group operations together
2023-07-17 16:05:00 +03:00
6369042420
refactor(backups): move reset() to top because toplevel interface
2023-07-17 16:05:00 +03:00
3edb38262f
refactor(backups): make redis and json provider related lowlevels private
2023-07-17 16:05:00 +03:00
3684345c2d
refactor(backups): make construct_provider not public
2023-07-17 16:05:00 +03:00
6b0c55a786
refactor(backups): make lookup_provider not public
2023-07-17 16:05:00 +03:00
dbac010303
refactor(backups): reorder imports
2023-07-17 16:05:00 +03:00
c09f2f393b
refactor(backups): api readability reorg
2023-07-17 16:05:00 +03:00
4b1594ca22
refactoring(backups): backuper -> backupper
2023-07-17 16:05:00 +03:00
c94b4d07bf
fix(tokens-repo): persistent hashing
2023-07-17 16:05:00 +03:00
4225772573
fix(backups): Providers were not initialized corretly
2023-07-17 16:05:00 +03:00
f27a3df807
refactor(backups): fix typing errors
2023-07-17 16:05:00 +03:00
b78ee5fcca
refactor(api): Group mutations
...
I've learned that there is no problem in grouping mutations like we do with queries.
This is a big mistake from my side, now we have legacy not so conveniently placed endpoints.
I've grouped all mutations, left the copies of old ones flattened in the root for backwards compatibility.
We will migrate to mutation groups on client side, and backups now only use grouped mutations.
Tests are updated.
2023-07-17 16:05:00 +03:00
53dfb38284
test(backups): ensure asking to reload snaps does not explode the server
2023-07-17 16:05:00 +03:00
ecf72948b1
test(backups): setting autobackup period
2023-07-17 16:05:00 +03:00
f829a34dc7
refactor(backups): delete legacy provider setting
2023-07-17 16:05:00 +03:00
9f096ed2c0
feature(backups): actually dealing with situation when the provider is not configured
2023-07-17 16:05:00 +03:00
cd32aa83b7
refactor(backups): NoneBackupper class for those cases when we do not know
2023-07-17 16:05:00 +03:00
a56461fb96
refactor(backups): make a dir for backuppers
2023-07-17 16:05:00 +03:00
806fb3c84b
feature(backups): resetting json config too
2023-07-17 16:05:00 +03:00
1fd5db9ff3
fix(backups): fix output API return types for configuration
2023-07-17 16:05:00 +03:00
5d95c1b44e
test(backups): preliminary test of repo reset
2023-07-17 16:05:00 +03:00
1c96743c5d
test(backups): test reinitting repository
2023-07-17 16:05:00 +03:00
38de01da8b
refactor(backups): cleanup localfile-specific logic
2023-07-17 16:05:00 +03:00
8475ae3375
refactor(backups): make localfile repos normal
2023-07-17 16:05:00 +03:00
53638b7e06
test(backups): make dummy service more compliant
2023-07-17 16:05:00 +03:00
93b98cd4fd
fix(backups): Handle orphaned snapshots
2023-07-17 16:05:00 +03:00
421c92d12e
fix(backups): return type of encryption key
2023-07-17 16:05:00 +03:00
c603394449
fix(backups): try to actually get backup configuration
2023-07-17 16:05:00 +03:00
b04dfc6c4e
fix(backups): register queries
2023-07-17 16:05:00 +03:00
42a5b6f70a
test(backups): test backup API - backing up
2023-07-17 16:05:00 +03:00
32a242b560
feat(backups): register backups in GraphQL schema
2023-07-17 16:05:00 +03:00
a4b0e6f208
fix: BackupConfiguration argument order
2023-07-17 16:05:00 +03:00
ad130e392c
feature(backups): check available space before restoring
2023-07-17 16:05:00 +03:00
780c12df6c
refactor(backups): expect one more error of restic json output parsing
2023-07-17 16:05:00 +03:00
6da0791b47
feature(backups): integration between restore and jobs
2023-07-17 16:05:00 +03:00
792dcd459d
fix(backups): return one job, not an array of one
2023-07-17 16:05:00 +03:00
5100f1a497
fix(backups): return 400, not 300
2023-07-17 16:05:00 +03:00
44e45a5124
BREAKING CHANGE(backups): support only individual service backup requests(combinable)
2023-07-17 16:05:00 +03:00
0b8f77e6f7
feature(backups): set autobackup period from gql
2023-07-17 16:05:00 +03:00
e3545d4541
feature(backups): get all snapshots if requested by api
2023-07-17 16:05:00 +03:00
550f7fa620
refactor(backups): introduce get_all_snapshots()
2023-07-17 16:05:00 +03:00
cc073155db
feature(backups): feature(backups): return a snapshot from start_backup
2023-07-17 16:05:00 +03:00
891993e4cd
feature(backups): a graphql call to invalidate cache
2023-07-17 16:05:00 +03:00
7e022e0cfe
feature(backups): graphql mutation for restore
2023-07-17 16:05:00 +03:00
44ddd27e84
fix(backups): return correct snapshots per service
2023-07-17 16:05:00 +03:00
761b6be4e5
refactor(backups): global snapshots
2023-07-17 16:05:00 +03:00
a76b4ac134
feature(backups): start backup graphql API
2023-07-17 16:05:00 +03:00
ac9fbbff3e
feature(backups): drop repository call
2023-07-17 16:05:00 +03:00
bdae6cfb75
feature(backups): global init instead of per-service
2023-07-17 16:05:00 +03:00
e7683352cd
feature(backups): a graphql query to get provider info
2023-07-17 16:05:00 +03:00
d0b27da641
feature(backups): init repo mutation
2023-07-17 16:05:00 +03:00
d10bf99927
fix(backups): make sure location and credentials get properly passed around
2023-07-17 16:05:00 +03:00
c5c41b3ced
refactor(backups): remove extraneous asserts from jobs
2023-07-17 16:05:00 +03:00
c8512eacdc
refactor(backups): refactor realtime updating
2023-07-17 16:05:00 +03:00
d38b8180cb
feature(backups): realtime progress updates of backups
2023-07-17 16:05:00 +03:00
135fb0c42d
feature(backups): job progress logs
2023-07-17 16:05:00 +03:00
ca036b294a
refactor(backups): break out job logs status prefix
2023-07-17 16:05:00 +03:00
afdbf01cfc
refactor(backups): use single repo and multiplex by tags
2023-07-17 16:05:00 +03:00
ecf44e5169
feature(backups): deny adding a backup job if another one is already queued
2023-07-17 16:05:00 +03:00
ebff2b308a
test(backups): test that the job has run
2023-07-17 16:05:00 +03:00
2a87eb80f9
refactor(backups): quick-expiration logs of jobs status updates
2023-07-17 16:05:00 +03:00
f116ce1bdb
feature(backups): set job status to error if backup fails
2023-07-17 16:05:00 +03:00
05f2cc3f14
refactor(backups): cleanup unused imports in tasks
2023-07-17 16:05:00 +03:00
f622d617cf
test(backups): test jobs starting and finishing when from Backups
2023-07-17 16:05:00 +03:00
ac6d25c4c1
refactor(backups): make a backup job running when the backup code itself is executed
2023-07-17 16:05:00 +03:00
026d72b551
refactor(backups): delete unused redis import from backups ckass
2023-07-17 16:05:00 +03:00
029cb47db6
feature(backups): also create a job if not called from a task
2023-07-17 16:05:00 +03:00
fa86c45bd0
feature(backups): simplest jobs intergration in tasks: created and finished
2023-07-17 16:05:00 +03:00
4572c00640
feature(backups): restore task
2023-07-17 16:05:00 +03:00
d3f9ce7bf5
test(backups): test local secrets
2023-07-17 16:05:00 +03:00
ebeb76149b
refactor(services): make local secret setting public
2023-07-17 16:05:00 +03:00
592eb1a1f8
refactor(services): use fully generic foldermoves
2023-07-17 16:05:00 +03:00
f09d21a031
test(services): test derivation of foldermoves
2023-07-17 16:05:00 +03:00
7a5af6af99
test(services): test that we indeed return correct folders and owned folders from real services
2023-07-17 16:05:00 +03:00
aca05f26ea
fix(services): folder methods typing
2023-07-17 16:05:00 +03:00
92be699031
refactor(services): make a foldermove from owned path
2023-07-17 16:05:00 +03:00
71b987da57
refactor(services): add folder owner derivation
2023-07-17 16:05:00 +03:00
9f2dbaa98d
refactor(services): add overridable get owner and get group
2023-07-17 16:05:00 +03:00
6057e350ef
refactor(services): add OwnedPath struct
2023-07-17 16:05:00 +03:00
df5b318fff
refactor(services): remove special storage counting from pleroma
2023-07-17 16:05:00 +03:00
f0d6ac624d
refactor(services): remove special storage counting from ocserv
2023-07-17 16:05:00 +03:00
ae7f53d1ec
refactor(services): remove special storage counting from nextcloud
2023-07-17 16:05:00 +03:00
34854b5118
documentation(services): move the storage count docstring to parent service class
2023-07-17 16:05:00 +03:00
f5de4974e7
refactor(services): remove special storage counting from mail
2023-07-17 16:05:00 +03:00
208e256c0f
refactor(services): remove special storage counting from jitsi
2023-07-17 16:05:00 +03:00
44041662c2
refactor(services): remove special storage counting from gitea
2023-07-17 16:05:00 +03:00
3b8168c25d
refactor(services): remove special storage counting from bitwarden
2023-07-17 16:05:00 +03:00
c2cd972805
refactor(services): add a generic storage counter
2023-07-17 16:05:00 +03:00
0a9848be47
refactor(services): make get_folders() a mandatory part of Service interface
2023-07-17 16:05:00 +03:00
ac04425221
refactor(services): add get_folders() to the rest of the services
2023-07-17 16:05:00 +03:00
1019031b5b
fix(services): use get_foldername() for moving around
2023-07-17 16:05:00 +03:00
95b88ea2e4
test(backups): implement get_folders() for gitea
2023-07-17 16:05:00 +03:00
498208f083
test(backups): implement get_folders() for bitwarden
2023-07-17 16:05:00 +03:00
b21d63be63
refactor(backups): set a list of folders for our dummy service
2023-07-17 16:05:00 +03:00
3aefbaaf0b
refactor(backups): actually accept a list of folders
2023-07-17 16:05:00 +03:00
f0aabec947
refactor(backups): make api accept a list of folders
2023-07-17 16:05:00 +03:00
d1e1039519
refactor(backups): make a dedicated get_folders() function
2023-07-17 16:05:00 +03:00
507cdb3bbd
refactor(services): rename get_location() to get_drive()
2023-07-17 16:05:00 +03:00
6132f1bb4c
test(backups): register dummy service
2023-07-17 16:05:00 +03:00
1940b29161
feature(backups): automatic backup
2023-07-17 16:05:00 +03:00
5e9c651c65
test(backups): test autobackup timing
2023-07-17 16:05:00 +03:00
b305c19559
refactor(backups): split out storage
2023-07-17 16:05:00 +03:00
Inex Code
Houkime