SelfPrivacy/selfprivacy-rest-api
1
0
Fork 0
mirror of https://git.selfprivacy.org/SelfPrivacy/selfprivacy-rest-api.git synced 2024-11-22 20:11:30 +00:00
Code Issues Projects Releases Wiki Activity
910 commits 32 branches 4 tags 4.2 MiB
Commit graph

641 commits

Author SHA1 Message Date
Houkime 44041662c2 refactor(services): remove special storage counting from gitea 2023-07-17 16:05:00 +03:00
Houkime 3b8168c25d refactor(services): remove special storage counting from bitwarden 2023-07-17 16:05:00 +03:00
Houkime c2cd972805 refactor(services): add a generic storage counter 2023-07-17 16:05:00 +03:00
Houkime 0a9848be47 refactor(services): make get_folders() a mandatory part of Service interface 2023-07-17 16:05:00 +03:00
Houkime ac04425221 refactor(services): add get_folders() to the rest of the services 2023-07-17 16:05:00 +03:00
Houkime 1019031b5b fix(services): use get_foldername() for moving around 2023-07-17 16:05:00 +03:00
Houkime 95b88ea2e4 test(backups): implement get_folders() for gitea 2023-07-17 16:05:00 +03:00
Houkime 498208f083 test(backups): implement get_folders() for bitwarden 2023-07-17 16:05:00 +03:00
Houkime b21d63be63 refactor(backups): set a list of folders for our dummy service 2023-07-17 16:05:00 +03:00
Houkime 3aefbaaf0b refactor(backups): actually accept a list of folders 2023-07-17 16:05:00 +03:00
Houkime f0aabec947 refactor(backups): make api accept a list of folders 2023-07-17 16:05:00 +03:00
Houkime d1e1039519 refactor(backups): make a dedicated get_folders() function 2023-07-17 16:05:00 +03:00
Houkime 507cdb3bbd refactor(services): rename get_location() to get_drive() 2023-07-17 16:05:00 +03:00
Houkime 6132f1bb4c test(backups): register dummy service 2023-07-17 16:05:00 +03:00
Houkime 1940b29161 feature(backups): automatic backup 2023-07-17 16:05:00 +03:00
Houkime 5e9c651c65 test(backups): test autobackup timing 2023-07-17 16:05:00 +03:00
Houkime b305c19559 refactor(backups): split out storage 2023-07-17 16:05:00 +03:00
Houkime f9eaaab929 feature(backups): enable snapshot cache usage 2023-07-17 16:05:00 +03:00
Houkime 2c510ae884 feature(backups): add snapshot cache sync functions 2023-07-17 16:05:00 +03:00
Houkime ed0861aacc test(backups): test last backup date retrieval 2023-07-17 16:05:00 +03:00
Houkime 054b07baa3 feature(backups): add a datetime validator function for huey autobackups 2023-07-17 16:05:00 +03:00
Houkime 343fda0630 test(backups): test setting autobackup period 2023-07-17 16:05:00 +03:00
Houkime 0a4338596b test(backups): test setting services as enabled for autobackups 2023-07-17 16:05:00 +03:00
Houkime 79b9bb352a feature(backups): methods for autobackup period setting and getting 2023-07-17 16:05:00 +03:00
Houkime 951bb8d5ec fix(backups): remove self from static method 2023-07-17 16:05:00 +03:00
Houkime d354f4ac0b feature(backups): check, set and unset service autobackup status 2023-07-17 16:05:00 +03:00
Houkime 43b6ebd04d feature(backups): cache snapshots and last backup timestamps 2023-07-17 16:05:00 +03:00
Houkime d57dc3f7d2 test(backups): test that we do return snapshot on backup 2023-07-17 16:05:00 +03:00
Houkime 35a4fec9d4 feature(backups): return snapshot info from backup function 2023-07-17 16:05:00 +03:00
Houkime a134009165 feature(backups): huey task to back up 2023-07-17 16:05:00 +03:00
Houkime d972fdc3cc refactor(backups): make backups stateless 2023-07-17 16:05:00 +03:00
Houkime 6f8f5cbb9e feature(backups): repo init tracking 2023-07-17 16:05:00 +03:00
Houkime 02deae217d feature(backups): provider storage and retrieval 2023-07-17 16:05:00 +03:00
Houkime 48dc63a590 refactor(backups): add a provider model for redis storage 2023-07-17 16:05:00 +03:00
Houkime 873bc8282e refactor(backups): redis model storage utils 2023-07-17 16:05:00 +03:00
Houkime c928263fce feature(backups): load from json 2023-07-17 16:05:00 +03:00
Houkime 0847e16089 feat(backups): local secret generation and storage 2023-07-17 16:05:00 +03:00
Houkime 60dcde458c feat(backups): sizing up snapshots 2023-07-17 16:05:00 +03:00
Houkime 1d403b0e94 test(backups): test restoring a file 2023-07-17 16:05:00 +03:00
Houkime c8a8d45110 feat(backups): add restore_snapshot and restore_service_from_snapshot 2023-07-17 16:05:00 +03:00
Houkime ff6bc2a142 feat(backups): a better error on failed snapshot retrieval 2023-07-17 16:05:00 +03:00
Houkime e56907f2cd feat(backups): return proper snapshot structs when listing 2023-07-17 16:05:00 +03:00
Houkime 228eab44bb feat(backups): throw an error on a failed backup 2023-07-17 16:05:00 +03:00
Houkime 348ece8b9c fix(backups): singleton metaclass was screwing with tests 2023-07-17 16:05:00 +03:00
Houkime a280e5c999 test(backups): localfile repo by default in tests 2023-07-17 16:05:00 +03:00
Houkime add4e21f39 feature(backups): throw an error if repo init fails 2023-07-17 16:05:00 +03:00
Houkime 5efb351159 feature(backups): register localfile backend 2023-07-17 16:05:00 +03:00
Houkime 529608d52e feature(backups): localfile repo 2023-07-17 16:05:00 +03:00
Houkime 3f30469532 refactor(backups): repo init service method 2023-07-17 16:05:00 +03:00
Houkime a405eddbcf refactor(backups): add repo init 2023-07-17 16:05:00 +03:00
Houkime 5371c7feef refactor(backups): snapshotlist and local secret groundwork 2023-07-17 16:05:00 +03:00
Houkime e156e9cd58 test(backup): no snapshots 2023-07-17 16:05:00 +03:00
Houkime 83b24f5fcd refactor(backup): snapshot model 2023-07-17 16:05:00 +03:00
Houkime 4ca2e62b5c feature(backup): loading snapshots 2023-07-17 16:05:00 +03:00
Houkime a42294b706 feature(backup): add a restore function to restic backuper 2023-07-17 16:05:00 +03:00
Houkime a0a0e1fb3b feat(backup): hooks 2023-07-17 16:05:00 +03:00
Houkime 178c456593 refactor(backup): add a backup function to Backups singleton class 2023-07-17 16:05:00 +03:00
Houkime ff72d4124e refactor(backup): add a placeholder Backups singleton class 2023-07-17 16:05:00 +03:00
Houkime 54103973bc test(backup): try to back up! 2023-07-17 16:05:00 +03:00
Houkime a9cd8dda37 fix(backup): add memory backup class,forgot to add to git 2023-07-17 16:05:00 +03:00
Houkime 86c99c0be8 feat(backup): add backuping to restic backuper 2023-07-17 16:05:00 +03:00
Houkime 37c18ead99 feat(backup): add in-memory backup 2023-07-17 16:05:00 +03:00
Houkime e5a965ea29 feat(backup): allow no auth 2023-07-17 16:05:00 +03:00
Houkime 45ab9423b9 test(backup): dummy service 2023-07-17 16:05:00 +03:00
Houkime 9097ba02d7 test(backup): provider class selection 2023-07-17 16:05:00 +03:00
Houkime 7d76b74dbc feature(backups): copy cli logic to new restic backuper 2023-07-17 16:05:00 +03:00
Houkime 1e5fb67374 feature(backups): placeholders for the backupers and backup providers 2023-07-17 16:05:00 +03:00
Houkime a3d58be0d5 feature(backups): placeholders for the modules of the new backup system 2023-07-17 16:05:00 +03:00
Houkime a1071fd2c9 feature(backups): add backup structures and queries 2023-07-17 16:05:00 +03:00
Houkime 7b7f782185 refactor(backup): do not use config file 2023-07-17 16:05:00 +03:00
Houkime f65c0522b0 refactor(backup): pass key and account to exec 2023-07-17 16:05:00 +03:00
Houkime 6bf5ee4b64 refactor(backup): extract restic repo 2023-07-17 16:05:00 +03:00
Houkime 8eab26d552 refactor(backup): extract rclone args 2023-07-17 16:05:00 +03:00
Houkime 70cf0306a9 refactor(backup): delete unused import 2023-07-17 16:05:00 +03:00
Houkime 9f19c677d0 feature(backup):calculate needed space for inplace restoration 2023-07-14 12:34:45 +00:00
Houkime 4aa87edf47 fix(servers): hopefully fix moving 2023-07-14 10:58:31 +00:00
Houkime 92cfd00f93 feature(servers): set default timeout of server operations to 10 min 2023-07-12 17:03:24 +00:00
Houkime e43478d437 feature(backups): stop services before restores 2023-07-12 16:53:49 +00:00
Houkime 7af7600599 feature(backups): stop services before backups 2023-07-12 16:43:26 +00:00
Houkime 326e3d3b0c test(backups): do not store the status file in backupped folders 2023-07-12 16:00:36 +00:00
Houkime de8ef744eb refactor(backups): make a StoppedService context manager 2023-07-12 15:14:54 +00:00
Houkime 713296c520 test(backups): make delay settable per dummyservice 2023-07-12 12:27:55 +00:00
Houkime e7a6700522 test(backups): test async service start n stop simulation 2023-07-12 12:10:40 +00:00
Houkime 169e9ad57d test(backups): simulating async service start n stop 2023-07-10 17:03:10 +00:00
Houkime 6523105d89 feature(utils): a hopefully reusable waitloop 2023-07-10 10:33:01 +00:00
Houkime 1cefaefa3b feature(backups): expose restore strategies to the API 2023-07-07 13:24:05 +00:00
Houkime 2743441e1e fix(backups): actually mount if asked for an inplace restore 2023-07-07 13:14:30 +00:00
Houkime 9075afd38a test(backups): test out that pre-restore backup plays nice with jobs 2023-07-07 12:50:20 +00:00
Houkime af5edb695f feature(backups): implement inplace restore strategy 2023-07-07 11:54:48 +00:00
Houkime c74b3df32c feature(backups): restore strategies enum 2023-07-07 10:50:59 +00:00
Houkime 53bb5cc4e2 feature(backups): forgetting snapshots 2023-07-05 13:13:30 +00:00
Houkime 03313b739a feature(backups): check restore exit code 2023-07-03 20:53:21 +00:00
Houkime 3dc6fb91f2 refactor(backups): download a copy before replacing original 2023-07-03 20:41:52 +00:00
Houkime b6eb27dc5e feature(backups): mounting a repo 2023-07-03 15:32:00 +00:00
Houkime 953860a02c refactor(backups): move syncing (non-restic) into backup utils 2023-07-03 15:32:00 +00:00
Houkime 6a00d3cff9 refactor(backups): move output yielding into backup utils 2023-07-03 15:32:00 +00:00
Houkime 9793201ca1 feature(backups): a wrapper for rclone sync 2023-07-03 15:32:00 +00:00
Inex Code 64e7afe53e feat(backups): Add backup descriptions for UI 2023-06-29 14:27:08 +03:00
Inex Code 15ce344bc8 feat(backups): expose if the service can be backed up 2023-06-29 13:45:00 +03:00
Inex Code b480c84041 style: fix styling 2023-06-29 13:44:29 +03:00
Houkime cd2e9d3ba3 fix(jobs): make finishing the job set progress to 100 2023-06-28 13:22:53 +00:00
Inex Code 11184a55e8 feat(jobs): return type_id of the job in graphql api 2023-06-28 15:56:30 +03:00
Houkime e4865aa094 fix(services): proper backup progress reporting 2023-06-28 11:45:07 +00:00
Houkime 35258bad38 fix(services): cleanup a stray get_location 2023-06-28 09:48:32 +00:00
Houkime 59fef1d016 fix(backups): register the correct tasks 2023-06-28 09:41:03 +00:00
Inex Code d9b26e12e2 fix(backups): missing space in rclone args 2023-06-28 12:10:12 +03:00
Houkime ae16a527a2 refactor(backups): rename service_snapshot_size to snapshot_restored_size 2023-06-26 19:41:26 +00:00
dettlaff 19005158a8 fix: add timewait 2023-06-26 23:33:41 +04:00
Houkime 7147f97077 refactor(backups): remove the by-service getting of cached snapshots 2023-06-26 19:29:29 +00:00
Houkime bae81b2f69 refactor(backups): rename force_snapshot_reload to force_snapshot_cache_reload 2023-06-26 19:20:49 +00:00
Houkime a76834d1ba refactor(backups): merge sync_all_snapshots with force_snapshot_reload 2023-06-26 19:16:18 +00:00
Houkime f8029d133a refactor(backups): straighten get_all_snapshots 2023-06-26 19:01:26 +00:00
Houkime 4ad4c3cc67 refactor(backups): delete sync_service_snapshots 2023-06-26 18:50:37 +00:00
Houkime b9be0be6a2 refactor(backups): privatize assert_restorable and restore_snapshot_from_id 2023-06-26 18:42:26 +00:00
Houkime 2f2c4f14af refactor(backups): group operations together 2023-06-26 18:30:31 +00:00
Houkime e6efd1b42d refactor(backups): move reset() to top because toplevel interface 2023-06-26 18:21:50 +00:00
Houkime 1593474dc1 refactor(backups): make redis and json provider related lowlevels private 2023-06-26 18:20:22 +00:00
Houkime 71eeed926d refactor(backups): make construct_provider not public 2023-06-26 18:14:15 +00:00
Houkime 81d1762518 refactor(backups): make lookup_provider not public 2023-06-26 18:11:11 +00:00
Houkime e159d2f1a7 refactor(backups): reorder imports 2023-06-26 18:07:47 +00:00
Houkime 8604caa331 refactor(backups): api readability reorg 2023-06-26 18:00:42 +00:00
Houkime 2f71469f39 refactoring(backups): backuper -> backupper 2023-06-23 12:04:33 +00:00
Houkime 6ca68fae17 fix(tokens-repo): persistent hashing 2023-06-23 11:54:40 +00:00
Inex Code 1664f857ea fix(backups): Providers were not initialized corretly 2023-06-23 14:36:16 +03:00
Inex Code e7e0fdc4a1 refactor(backups): fix typing errors 2023-06-23 12:40:10 +03:00
dettlaff 993ea93ee4 fix: delete old gen 2023-06-21 10:52:31 +04:00
Inex Code c6919293b6 refactor(api): Group mutations 
I've learned that there is no problem in grouping mutations like we do with queries.
This is a big mistake from my side, now we have legacy not so conveniently placed endpoints.
I've grouped all mutations, left the copies of old ones flattened in the root for backwards compatibility.
We will migrate to mutation groups on client side, and backups now only use grouped mutations.
Tests are updated.
 2023-06-21 06:46:56 +03:00
dettlaff 3d4caaf4ce fix: encoding 2023-06-20 23:25:54 +04:00
dettlaff 79ef4c9310 fix: task registry 2023-06-20 01:25:04 +04:00
Houkime e3a87f1d98 test(backups): ensure asking to reload snaps does not explode the server 2023-06-19 14:12:50 +00:00
Houkime 25f3115c05 test(backups): setting autobackup period 2023-06-19 13:28:02 +00:00
Houkime b5e2499a30 refactor(backups): delete legacy provider setting 2023-06-19 11:17:09 +00:00
Houkime 612a46ee42 feature(backups): actually dealing with situation when the provider is not configured 2023-06-19 11:09:10 +00:00
dettlaff 83cf7cbd6f fix: jobs 2023-06-18 08:37:27 +04:00
Houkime 4f2f24daac refactor(backups): NoneBackupper class for those cases when we do not know 2023-06-16 16:03:32 +00:00
Houkime 3449837de9 refactor(backups): make a dir for backuppers 2023-06-16 15:50:31 +00:00
Houkime 79c3b9598c feature(backups): resetting json config too 2023-06-16 15:09:39 +00:00
Houkime 47aee3c1f1 fix(backups): fix output API return types for configuration 2023-06-16 14:39:00 +00:00
Houkime 8e1e37c766 test(backups): preliminary test of repo reset 2023-06-16 14:33:48 +00:00
Houkime 33c60f971d test(backups): test reinitting repository 2023-06-16 13:43:41 +00:00
dettlaff d6c4d458f7 tests: fix some tests 2023-06-16 08:08:36 +04:00
dettlaff 28701e1608 fix: do a redis reset() 2023-06-16 08:08:36 +04:00
dettlaff 96c013c4e7 refactor: delete purest of nix_collect_garbage() 2023-06-16 08:08:36 +04:00
ВашÐdettlaff 03b5a6b5e4 fix: percentage 2023-06-16 08:08:36 +04:00
def af0129bdfe fix: types, add tests 2023-06-16 04:06:49 +00:00
def 7553202a69 test: fix nix collect garbage, add tests 2023-06-16 04:06:49 +00:00
def 7e3adffb68 refactor: nix-collect-garbage is now pure 2023-06-16 04:06:49 +00:00
def f42594f1fb refactor: delete unnecessary import 2023-06-16 04:06:49 +00:00
def 6ef9222e32 feat: loading percentage 2023-06-16 04:06:49 +00:00
def ba168fc92b fix: subprocess.check_output 2023-06-16 04:06:49 +00:00
def 57ec87bf43 feat: add nix-collect-garbage job 2023-06-16 04:06:49 +00:00
Inex Code b3a37e8b1f fix: Migrate to SP channel from 22.11 installations 2023-06-14 19:27:11 +03:00
Houkime b3724e240e refactor(backups): cleanup localfile-specific logic 2023-06-14 14:17:04 +00:00
Houkime 9e0b0d8caa refactor(backups): make localfile repos normal 2023-06-14 14:07:51 +00:00
Houkime 21791f20a2 test(backups): make dummy service more compliant 2023-06-14 11:55:46 +00:00
Houkime 8554879dc2 Merge remote-tracking branch 'upstream/restic-rewrite-api' into restic-rewrite 2023-06-14 10:27:50 +00:00
Houkime d9bde6930b fix(backups): register queries 2023-06-14 10:16:53 +00:00
Houkime 865e304f42 test(backups): test backup API - backing up 2023-06-14 10:16:31 +00:00
Inex Code d356fad534 fix(backups): Handle orphaned snapshots 2023-06-14 02:52:10 +03:00
Inex Code f4df1f6a62 fix(backups): return type of encryption key 2023-06-14 01:40:53 +03:00
Inex Code 0ef6569d97 fix(backups): try to actually get backup configuration 2023-06-14 00:43:01 +03:00
Inex Code f950dd1e93 feat(backups): register backups in GraphQL schema 2023-06-14 00:00:29 +03:00
Inex Code d02302c7b2 fix: BackupConfiguration argument order 2023-06-13 23:54:02 +03:00
Houkime 549b149aaf feature(backups): check available space before restoring 2023-06-07 16:33:13 +00:00
Houkime 981445d594 refactor(backups): expect one more error of restic json output parsing 2023-06-07 15:55:38 +00:00
Houkime 7f984b678f feature(backups): integration between restore and jobs 2023-06-07 15:05:58 +00:00
Houkime 98de85e569 fix(backups): return one job, not an array of one 2023-06-07 14:27:37 +00:00
Houkime 4b8abb6a66 fix(backups): return 400, not 300 2023-06-07 14:25:17 +00:00
Houkime 0c95c5913b BREAKING CHANGE(backups): support only individual service backup requests(combinable) 2023-06-07 14:11:22 +00:00
Houkime ff58bdccbb feature(backups): set autobackup period from gql 2023-06-05 11:49:37 +00:00
Houkime d6609b29e8 feature(backups): get all snapshots if requested by api 2023-06-05 11:36:58 +00:00
Houkime 2168037a10 refactor(backups): introduce get_all_snapshots() 2023-06-05 11:28:53 +00:00
Houkime 9d5335f62c feature(backups): feature(backups): return a snapshot from start_backup 2023-06-05 11:26:58 +00:00
Houkime 9a94f7624e feature(backups): a graphql call to invalidate cache 2023-06-01 16:12:32 +00:00
Houkime 50952d688a feature(backups): graphql mutation for restore 2023-06-01 14:03:40 +00:00
Houkime f7c0821675 fix(backups): return correct snapshots per service 2023-06-01 12:44:14 +00:00
Houkime 4b07d4de41 refactor(backups): global snapshots 2023-05-31 13:16:08 +00:00
Houkime 603ed2ddf9 feature(backups): start backup graphql API 2023-05-31 12:07:47 +00:00
Houkime 9c4d2a0ba5 feature(backups): drop repository call 2023-05-29 18:24:38 +00:00
Houkime c3696d3a4b feature(backups): global init instead of per-service 2023-05-29 16:50:14 +00:00
Houkime 345e2c10c4 feature(backups): a graphql query to get provider info 2023-05-29 16:12:31 +00:00
Houkime 97e4c529f6 feature(backups): init repo mutation 2023-05-29 15:34:26 +00:00
Houkime 0dc6f74754 fix(backups): make sure location and credentials get properly passed around 2023-05-22 16:01:57 +00:00
Houkime 03d4632465 refactor(backups): remove extraneous asserts from jobs 2023-05-17 20:27:42 +00:00
Houkime 9fab596f91 refactor(backups): refactor realtime updating 2023-05-17 20:27:42 +00:00
Houkime b1d0a80963 feature(backups): realtime progress updates of backups 2023-05-17 20:27:42 +00:00
Houkime 60806cd536 feature(backups): job progress logs 2023-05-17 20:27:42 +00:00
Houkime ddd2176a5d refactor(backups): break out job logs status prefix 2023-05-17 20:27:42 +00:00
Houkime deb857bca9 refactor(backups): use single repo and multiplex by tags 2023-05-17 20:27:42 +00:00
Houkime a6b3a5e590 feature(backups): deny adding a backup job if another one is already queued 2023-05-17 20:27:42 +00:00
Houkime e25aa2cb33 test(backups): test that the job has run 2023-05-17 20:27:42 +00:00
Houkime 7ddfad10d4 refactor(backups): quick-expiration logs of jobs status updates 2023-05-17 20:27:42 +00:00
Houkime b2e231ebae feature(backups): set job status to error if backup fails 2023-05-17 20:27:42 +00:00
Houkime 0bf18dcdc7 refactor(backups): cleanup unused imports in tasks 2023-05-17 20:27:42 +00:00
Houkime 396b42b69c test(backups): test jobs starting and finishing when from Backups 2023-05-17 20:27:42 +00:00
Houkime ad6cc5e1bc refactor(backups): make a backup job running when the backup code itself is executed 2023-05-17 20:27:42 +00:00
Houkime c0dfbb3ca2 refactor(backups): delete unused redis import from backups ckass 2023-05-17 20:27:42 +00:00
Houkime 3eb8382d9b feature(backups): also create a job if not called from a task 2023-05-17 20:27:42 +00:00
Houkime 108fca0eb3 feature(backups): simplest jobs intergration in tasks: created and finished 2023-05-17 20:27:42 +00:00
Houkime d1425561d9 feature(backups): restore task 2023-05-17 20:27:42 +00:00