first commit

.gitignore

@@ -0,0 +1,2 @@
+book
+.obsidian

book.toml

@@ -0,0 +1,6 @@
+[book]
+authors = ["Inex Code"]
+language = "en"
+multilingual = false
+src = "src"
+title = "SelfPrivacy developer docs"

src/SUMMARY.md

@@ -0,0 +1,8 @@
+# SelfPrivacy REST API
+
+- [API Reference](rest-api/api.md)
+- [Authorization](rest-api/auth.md)
+- [Changelog](rest-api/changelog.md)
+- [Migrations](rest-api/migrations.md)
+- [Testing, linting](rest-api/testing.md)
+- [Vulnerabilities](rest-api/vulnerabilities.md)

src/rest-api/auth.md

@@ -0,0 +1 @@
+# Authorization

src/rest-api/changelog.md

@@ -0,0 +1,70 @@
+# Changelog
+## authorization_tokens branch
+- Added authorization tokens module to allow
+	- Having more than one device controlling the server
+	- Refreshing the current token
+	- Obtaining a recovery token
+### API changes
+- 
+## 1.1.1
+Released on 26 Jan 2022
+
+- Added [[migrations|migration module]] which handles one-shot actions which cannot be done by nixos-configuration.
+- Added [[migrations#Fix NixOS configuration branch|migration]] which changes rolling-testing branch to master, bug introduced in 0.4.0 version of the app.
+## 1.1.0 
+Released on 9 Dec 2021
+
+- Added system configuration endpoints.
+- SSH management.
+- Backups controller.
+- Systemd is now used for system rebuilds.
+- `WriteUserData` and `ReadUserData` utils added to reduce boilerplate of file locking.
+- Unit tests are now a thing.
+### API changes
+- `/system/upgrade` is now `/system/configuration/upgrade`.
+- `/api/version` added.
+- `/services/ssh` added.
+- `/services/ssh/<username>` added.
+- `/configuration/timezone` added.
+- `/configuration/autoUpgrade` added.
+- `/restic/backblaze/config` added.
+- `/system/configuration/pull` added.
+- `/services/restic/backup/reload` added.
+---
+# Pre-1.1.0 changes
+This is an API refactoring stage. There was no endpoint to retrieve API version, but these all were referenced as 1.1.0 version in code.
+
+## [Input sanitization, added swagger](https://git.selfprivacy.org/SelfPrivacy/selfprivacy-rest-api/pulls/5)
+Released on 17 Nov 2021
+
+- **Addressed [[vulnerabilities#SPCVE-0001|SPCVE-0001]].**
+- Added Swagger documentation.
+
+## [Add basic API auth](https://git.selfprivacy.org/SelfPrivacy/selfprivacy-rest-api/pulls/4)
+Released on 16 Nov 2021
+
+- Added basic token auth.
+## [Move to JSON controlled server settings](https://git.selfprivacy.org/SelfPrivacy/selfprivacy-rest-api/pulls/3)
+Released on 16 Nov 2021
+
+- **Assumes server already moved to JSON-controlled Nix config**.
+- System is now configured by usage of userdata.json.
+- File locking is used now.
+## [Decomposition](https://git.selfprivacy.org/SelfPrivacy/selfprivacy-rest-api/pulls/2)
+Released on 12 Nov 2021
+### API changes
+* `/systemVersion` is now `/system/version`.
+* `/getDKIM` is now `/services/mailserver/dkim`.
+* `/pythonVersion` is now `/system/version/pythonVersion`.
+* `/users/create` is now `POST` on `/users`.
+* `/users/deleteUser` is now `DELETE` on `/users`.
+* `/services/SERVICE_NAME/enable` and `/services/SERVICE_NAME/disable` now also return message in their responses.
+* `/services/ssh/key/send`: response field `result` renamed to `status`.
+### Other changes
+* Moved to Python 3.9.
+* Moved to `flask_restful`'s `Resource` objects.
+* `distutils` is [deprecated](https://docs.python.org/3/whatsnew/3.10.html#distutils-deprecated), moved to `setuptools`.
+### Code formatting
+[Black](https://github.com/psf/black) is used for code *formatting*.
+
+No *linting* rules applied yet.

src/rest-api/migrations.md

@@ -0,0 +1,66 @@
+# Migrations
+Migrations module is introduced in [[changelog#1 1 1|v1.1.1]] of API and provides one-shot migrations which cannot be performed from the NixOS configuration file changes.
+These migrations are checked and ran before every start of the API.
+
+You can disable certain migrations if needed by creating an array at `api.skippedMigrations` in `userdata.json` and populating it with IDs of the migrations to skip. Adding `DISABLE_ALL` to that array disables the migrations module entirely. 
+
+For example:
+```json
+...
+    "api": {
+        "token": "secret_token",
+        "enableSwagger": false,
+		"skippedMigrations": [ "fix_nixos_config_branch", "create_tokens_json" ]
+    },
+...
+```
+
+## Fix NixOS configuration branch
+**Migration ID**: `fix_nixos_config_branch`
+
+**Introduced in**: [[changelog#1 1 1|v1.1.1]]
+
+### Description
+Mobile SelfPrivacy app introduced a bug in version 0.4.0.
+New servers were initialized with a rolling-testing nixos config branch.
+This was fixed in app version 0.4.2, but existing servers were not updated.
+This migration fixes this by changing the nixos config branch to master.
+
+### Run conditions
+Git repository at `/etc/nixos` is using `rolling-testing` branch.
+
+### Migration process
+1. Moving context to `/etc/nixos`.
+2. Running `git config remote.origin.fetch +refs/heads/*:refs/remotes/origin/*` to fix the consequences of `--single-branch` flag.
+3. `git fetch --all` to fetch all existing branches.
+4. `git pull` to pull them.
+5. `git checkout master` moves to master.
+6. Leaving the context.
+
+## Create tokens JSON file
+**Migration ID**: `create_tokens_json`
+
+**Introduced in**: [[changelog#authorization_tokens branch]]
+
+### Description
+Selfprivacy API used a single token in userdata.json for authentication.
+This migration creates a new tokens.json file with the old token in it.
+
+### Run conditions
+`/etc/nixos/userdata/tokens.json` does not exist.
+
+### Migration process
+1. Current token is retrieved from `userdata.json`.
+2. `tokens.json` is created with 0600 permission.
+3. File is populated with the following data:
+```json
+{
+	"tokens": [
+		{
+			"token": "token_string",
+			"name": "primary_token",
+			"date": "current date from str(datetime.now())",
+		}
+	]
+}
+```

src/rest-api/testing.md

@@ -0,0 +1,9 @@
+# Testing
+We use pytest.
+`coverage run -m pytest && coverage xml && coverage report`
+# Code formatting
+[Black](https://github.com/psf/black).
+# Security
+[Bandit](https://github.com/PyCQA/bandit).
+# Linting
+Pylint is recommended, but not enforced right now.

src/rest-api/vulnerabilities.md

@@ -0,0 +1,22 @@
+# SPCVE-0001
+**API versions affected**: [[changelog#Input sanitization added swagger 17 Nov 2021 https git selfprivacy org SelfPrivacy selfprivacy-rest-api pulls 5|All pre-1.1.0 releases]]
+
+**SelfPrivacy app versions affected:** ≤0.2.4; fixed in 0.3.0
+
+**Discovered on**: 16 Nov 2021
+
+**Addressed on**: 17 Nov 2021
+## Description
+Remote code execution vulnerability allowed root access to anyone, without any authorization. Was caused by the following factors:
+- API had no authentication.
+- No input sanitation used.
+- Python's ``subprocess.Popen`` was called with `shell=True`.
+
+At that time, there was no mechanism to upgrade API, so the server had to be recreated.
+
+## Taken measures
+- Basic API auth added.
+- All `subprocess` calls now don't use `shell=true`.
+- CI pipeline now includes [bandit](https://github.com/PyCQA/bandit) to prevent same mistakes in the future.
+- More input sanitation added.
+- Created a [nix overlay](https://git.selfprivacy.org/SelfPrivacy/selfprivacy-nix-repo) to provide API upgrades automatically.