The server backs itself up: every collection byte for byte, the media linked

A backup is a directory <stamp>-<kind> under Backups:Root (/var/lib/privapub/backups, 2770, files 0640), written as
.partial and renamed once whole: a manifest (host, build, newest migration, each collection's count, size, sha256 and
indexes, what was left out and why, the media list), each collection as gzipped canonical Extended JSON read raw, and
hard links to the files of untrashed media rows (copies where a link can't be made). On a replica set every collection
is read in one snapshot session. Never in a backup: the statistics salt, jobs, recovery codes, sessions, the
maintenance lock and the configuration's copy with its SMTP password.

One backup or restore at a time (MaintenanceLock, a heartbeat document), and the janitor purges nothing meanwhile.
BackupScheduler backs up nightly at 03:30 UTC (or at once after missing a night); rotation keeps 7 daily, 4 weekly,
3 pre-deploy and 3 pre-restore backups. CLI: admin backup [--kind] [--db-only], admin backups, admin backup verify;
these run before migrations, so the deploy's own pre-deploy backup, which replaces mongodump, is of the database as the
live build left it. EntityMaps.Warm runs once under a lock, since test hosts now boot side by side.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
This commit is contained in:
thepraandClaude Opus 5.5 committed 2026-10-07 11:40:06 +02:00
1 parent 37b12c5fee
commit 12bb75809f
20 files changed
+1132 -22

No files matched your search

+23 -2
View File
@@ -484,6 +484,26 @@ group www-data and reaches the private mongod; `sudo -u www-data` works too.
(`--replSet rs0`, a 990 MB oplog; owner decision 2026-10-07), so a backup reads every collection at one instant;
`setup.sh` converts it once (PrivaPub stopped, mongod restarted, `rs.initiate`), and every connection string says
`directConnection=true`, which also works against a standalone. The pasture's mongo is one too.
- **Backups** (`Infrastructure/Backup/`; owner decisions 2026-10-07: the whole server, plain files protected by their
permissions, run from the CLI, nightly and from the administrator's page). Each backup is a directory
`<yyyyMMdd-HHmmss>-<kind>` under `Backups:Root` (`/var/lib/privapub/backups`, www-data 2770, files 0640), written as
`.partial` and renamed once whole:
- `manifest.json` (`ArchiveManifest`): host, build, newest migration, whether it was read at one instant, each
collection's count, size, sha256 and indexes, what was left out and why, and the media list;
- `db/<collection>.jsonl.gz`: each document as one line of canonical Extended JSON, read raw, so every BSON type comes
back byte for byte; on a replica set every collection is read in one snapshot session
(`minSnapshotHistoryWindowInSeconds` is raised to an hour only while it reads);
- `media/`: hard links to the files of untrashed `MediaAttachment` rows (no disk spent; a deleted file stays until the
backup rotates out), copies where a link can't be made. `--db-only` lists them instead.
**Never in a backup** (`ServerBackup.Excluded`): `InteractionSalt` (owner rule), `Job` and `Delivery` (work in flight),
`EmailRecovery`, `openiddict.tokens` and `.authorizations` (sessions), `MaintenanceLock`, and `AppConfiguration`
(its SMTP password; it is made again from appsettings at boot). One backup or restore runs at a time
(`MaintenanceLock`, a heartbeat document; a holder silent for 2 minutes is taken over), and the media janitor purges
nothing meanwhile. `BackupScheduler` backs up at `Backups:NightlyAt` (03:30 UTC), or at once when it missed the night;
rotation keeps 7 daily and 4 weekly nightly backups, 3 pre-deploy, 3 pre-restore, and manual and uploaded ones until
deleted. CLI: `PrivaPub admin backup [--kind manual|pre-deploy] [--db-only]`, `admin backups`, `admin backup verify
<id>`; these run before migrations, so the deploy's backup is of the database as the live build left it.
## Code style
@@ -840,8 +860,9 @@ the owner's GoToSocial account.**
## Deploy
- **CI/CD:** push to `master` runs `build.yml` (build + tests) on the instance-wide `build` runner. A `v*` tag runs
`deploy.yml`: tests, self-contained linux-x64 publish, snapshot and `mongodump` to `/var/backups/privapub.thepra.dev`
(never the statistics salt: `InteractionSalt` is excluded and the dump is checked for it),
`deploy.yml`: tests, self-contained linux-x64 publish, a snapshot of the site to `/var/backups/privapub.thepra.dev`,
the server's own pre-deploy backup (`PrivaPub admin backup --kind pre-deploy --db-only`, run from the new build before
its migrations, verified, and checked to hold no statistics salt; no deploy while `restore.json` waits),
stop → rsync → start, a `127.0.0.1:6970/build.json` health loop with rollback, then public checks (actor, NodeInfo,
Swagger 404, inbox junk 400, unsigned 401) and `tools/smoke/mastodon-api.sh` (app registration, client credentials,
discovery, instance, public timeline). Then it checks that what production should be running is running: