first commit

This commit is contained in:
Inex Code 2022-01-27 16:39:19 +03:00
commit cbe95faf31
9 changed files with 1438 additions and 0 deletions

2
.gitignore vendored Normal file
View file

@ -0,0 +1,2 @@
book
.obsidian

6
book.toml Normal file
View file

@ -0,0 +1,6 @@
[book]
authors = ["Inex Code"]
language = "en"
multilingual = false
src = "src"
title = "SelfPrivacy developer docs"

8
src/SUMMARY.md Normal file
View file

@ -0,0 +1,8 @@
# SelfPrivacy REST API
- [API Reference](rest-api/api.md)
- [Authorization](rest-api/auth.md)
- [Changelog](rest-api/changelog.md)
- [Migrations](rest-api/migrations.md)
- [Testing, linting](rest-api/testing.md)
- [Vulnerabilities](rest-api/vulnerabilities.md)

1254
src/rest-api/api.md Normal file

File diff suppressed because it is too large Load diff

1
src/rest-api/auth.md Normal file
View file

@ -0,0 +1 @@
# Authorization

70
src/rest-api/changelog.md Normal file
View file

@ -0,0 +1,70 @@
# Changelog
## authorization_tokens branch
- Added authorization tokens module to allow
- Having more than one device controlling the server
- Refreshing the current token
- Obtaining a recovery token
### API changes
-
## 1.1.1
Released on 26 Jan 2022
- Added [[migrations|migration module]] which handles one-shot actions which cannot be done by nixos-configuration.
- Added [[migrations#Fix NixOS configuration branch|migration]] which changes rolling-testing branch to master, bug introduced in 0.4.0 version of the app.
## 1.1.0
Released on 9 Dec 2021
- Added system configuration endpoints.
- SSH management.
- Backups controller.
- Systemd is now used for system rebuilds.
- `WriteUserData` and `ReadUserData` utils added to reduce boilerplate of file locking.
- Unit tests are now a thing.
### API changes
- `/system/upgrade` is now `/system/configuration/upgrade`.
- `/api/version` added.
- `/services/ssh` added.
- `/services/ssh/<username>` added.
- `/configuration/timezone` added.
- `/configuration/autoUpgrade` added.
- `/restic/backblaze/config` added.
- `/system/configuration/pull` added.
- `/services/restic/backup/reload` added.
---
# Pre-1.1.0 changes
This is an API refactoring stage. There was no endpoint to retrieve API version, but these all were referenced as 1.1.0 version in code.
## [Input sanitization, added swagger](https://git.selfprivacy.org/SelfPrivacy/selfprivacy-rest-api/pulls/5)
Released on 17 Nov 2021
- **Addressed [[vulnerabilities#SPCVE-0001|SPCVE-0001]].**
- Added Swagger documentation.
## [Add basic API auth](https://git.selfprivacy.org/SelfPrivacy/selfprivacy-rest-api/pulls/4)
Released on 16 Nov 2021
- Added basic token auth.
## [Move to JSON controlled server settings](https://git.selfprivacy.org/SelfPrivacy/selfprivacy-rest-api/pulls/3)
Released on 16 Nov 2021
- **Assumes server already moved to JSON-controlled Nix config**.
- System is now configured by usage of userdata.json.
- File locking is used now.
## [Decomposition](https://git.selfprivacy.org/SelfPrivacy/selfprivacy-rest-api/pulls/2)
Released on 12 Nov 2021
### API changes
* `/systemVersion` is now `/system/version`.
* `/getDKIM` is now `/services/mailserver/dkim`.
* `/pythonVersion` is now `/system/version/pythonVersion`.
* `/users/create` is now `POST` on `/users`.
* `/users/deleteUser` is now `DELETE` on `/users`.
* `/services/SERVICE_NAME/enable` and `/services/SERVICE_NAME/disable` now also return message in their responses.
* `/services/ssh/key/send`: response field `result` renamed to `status`.
### Other changes
* Moved to Python 3.9.
* Moved to `flask_restful`'s `Resource` objects.
* `distutils` is [deprecated](https://docs.python.org/3/whatsnew/3.10.html#distutils-deprecated), moved to `setuptools`.
### Code formatting
[Black](https://github.com/psf/black) is used for code *formatting*.
No *linting* rules applied yet.

View file

@ -0,0 +1,66 @@
# Migrations
Migrations module is introduced in [[changelog#1 1 1|v1.1.1]] of API and provides one-shot migrations which cannot be performed from the NixOS configuration file changes.
These migrations are checked and ran before every start of the API.
You can disable certain migrations if needed by creating an array at `api.skippedMigrations` in `userdata.json` and populating it with IDs of the migrations to skip. Adding `DISABLE_ALL` to that array disables the migrations module entirely.
For example:
```json
...
"api": {
"token": "secret_token",
"enableSwagger": false,
"skippedMigrations": [ "fix_nixos_config_branch", "create_tokens_json" ]
},
...
```
## Fix NixOS configuration branch
**Migration ID**: `fix_nixos_config_branch`
**Introduced in**: [[changelog#1 1 1|v1.1.1]]
### Description
Mobile SelfPrivacy app introduced a bug in version 0.4.0.
New servers were initialized with a rolling-testing nixos config branch.
This was fixed in app version 0.4.2, but existing servers were not updated.
This migration fixes this by changing the nixos config branch to master.
### Run conditions
Git repository at `/etc/nixos` is using `rolling-testing` branch.
### Migration process
1. Moving context to `/etc/nixos`.
2. Running `git config remote.origin.fetch +refs/heads/*:refs/remotes/origin/*` to fix the consequences of `--single-branch` flag.
3. `git fetch --all` to fetch all existing branches.
4. `git pull` to pull them.
5. `git checkout master` moves to master.
6. Leaving the context.
## Create tokens JSON file
**Migration ID**: `create_tokens_json`
**Introduced in**: [[changelog#authorization_tokens branch]]
### Description
Selfprivacy API used a single token in userdata.json for authentication.
This migration creates a new tokens.json file with the old token in it.
### Run conditions
`/etc/nixos/userdata/tokens.json` does not exist.
### Migration process
1. Current token is retrieved from `userdata.json`.
2. `tokens.json` is created with 0600 permission.
3. File is populated with the following data:
```json
{
"tokens": [
{
"token": "token_string",
"name": "primary_token",
"date": "current date from str(datetime.now())",
}
]
}
```

9
src/rest-api/testing.md Normal file
View file

@ -0,0 +1,9 @@
# Testing
We use pytest.
`coverage run -m pytest && coverage xml && coverage report`
# Code formatting
[Black](https://github.com/psf/black).
# Security
[Bandit](https://github.com/PyCQA/bandit).
# Linting
Pylint is recommended, but not enforced right now.

View file

@ -0,0 +1,22 @@
# SPCVE-0001
**API versions affected**: [[changelog#Input sanitization added swagger 17 Nov 2021 https git selfprivacy org SelfPrivacy selfprivacy-rest-api pulls 5|All pre-1.1.0 releases]]
**SelfPrivacy app versions affected:** ≤0.2.4; fixed in 0.3.0
**Discovered on**: 16 Nov 2021
**Addressed on**: 17 Nov 2021
## Description
Remote code execution vulnerability allowed root access to anyone, without any authorization. Was caused by the following factors:
- API had no authentication.
- No input sanitation used.
- Python's ``subprocess.Popen`` was called with `shell=True`.
At that time, there was no mechanism to upgrade API, so the server had to be recreated.
## Taken measures
- Basic API auth added.
- All `subprocess` calls now don't use `shell=true`.
- CI pipeline now includes [bandit](https://github.com/PyCQA/bandit) to prevent same mistakes in the future.
- More input sanitation added.
- Created a [nix overlay](https://git.selfprivacy.org/SelfPrivacy/selfprivacy-nix-repo) to provide API upgrades automatically.