Mongo is a one-member replica set
Owner decision 2026-10-07: production's mongod becomes a one-member replica set (rs0), so that a backup can read every collection at one instant (a snapshot read session), which a standalone mongod can't. The unit runs mongod with --replSet rs0 and a 990 MB oplog; setup.sh converts it once, idempotently (PrivaPub stopped, mongod restarted, rs.initiate, the primary awaited, PrivaPub started); every connection string says directConnection=true, which works against the standalone too, so this code can deploy before the conversion. The CI's throwaway mongod and the pasture's are replica sets as well, so the snapshot path is what the tests exercise; MongoTopology tells which a mongod is, and TopologyTests holds the test mongod to it. The suite passes on it (906). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
This commit is contained in:
1 parent
830385ea2b
commit
37b12c5fee
10 files changed
+90
-11
No files matched your search
@@ -12,7 +12,7 @@ env:
|
|||||||
BACKUPS: /var/backups/privapub.thepra.dev
|
BACKUPS: /var/backups/privapub.thepra.dev
|
||||||
LOCAL_URL: http://127.0.0.1:6970
|
LOCAL_URL: http://127.0.0.1:6970
|
||||||
PUBLIC_URL: https://privapub.thepra.dev
|
PUBLIC_URL: https://privapub.thepra.dev
|
||||||
MONGO_URI: mongodb://127.0.0.1:27022
|
MONGO_URI: mongodb://127.0.0.1:27022/?directConnection=true
|
||||||
MONGO_DB: PrivaPub
|
MONGO_DB: PrivaPub
|
||||||
|
|
||||||
jobs:
|
jobs:
|
||||||
|
|||||||
@@ -480,7 +480,10 @@ group www-data and reaches the private mongod; `sudo -u www-data` works too.
|
|||||||
`_NNN_` order, each runs once), then `Indexes.Create()`. A new entity needs nothing; a new unique index needs a
|
`_NNN_` order, each runs once), then `Indexes.Create()`. A new entity needs nothing; a new unique index needs a
|
||||||
dedupe migration before it.
|
dedupe migration before it.
|
||||||
- Production runs its **own mongod** on 127.0.0.1:27022 (unit `privapub-mongod`, no auth, data in
|
- Production runs its **own mongod** on 127.0.0.1:27022 (unit `privapub-mongod`, no auth, data in
|
||||||
`/var/lib/privapub/mongo`). The box's shared mongod needs credentials nobody here has.
|
`/var/lib/privapub/mongo`). The box's shared mongod needs credentials nobody here has. It is a one-member replica set
|
||||||
|
(`--replSet rs0`, a 990 MB oplog; owner decision 2026-10-07), so a backup reads every collection at one instant;
|
||||||
|
`setup.sh` converts it once (PrivaPub stopped, mongod restarted, `rs.initiate`), and every connection string says
|
||||||
|
`directConnection=true`, which also works against a standalone. The pasture's mongo is one too.
|
||||||
|
|
||||||
## Code style
|
## Code style
|
||||||
|
|
||||||
@@ -505,6 +508,8 @@ group www-data and reaches the private mongod; `sudo -u www-data` works too.
|
|||||||
`PrivaPub.Tests` (xUnit v3). Unit tests need nothing; tests marked `Category=Integration` need a mongod and skip
|
`PrivaPub.Tests` (xUnit v3). Unit tests need nothing; tests marked `Category=Integration` need a mongod and skip
|
||||||
without `PRIVAPUB_TEST_MONGOD=1`. CI (`build.yml` and `deploy.yml`) runs all of them through
|
without `PRIVAPUB_TEST_MONGOD=1`. CI (`build.yml` and `deploy.yml`) runs all of them through
|
||||||
`tools/ci/with-test-mongod.sh`, which starts a throwaway mongod on a random localhost port and deletes it afterwards.
|
`tools/ci/with-test-mongod.sh`, which starts a throwaway mongod on a random localhost port and deletes it afterwards.
|
||||||
|
Like production's, it is a one-member replica set (`rs0`, reached with `directConnection=true`), so backups' snapshot
|
||||||
|
reads are what the tests exercise (`TopologyTests`).
|
||||||
The box's own mongods are production, so `MongoFixture` refuses port 27022, a data directory under `/var/lib/privapub`,
|
The box's own mongods are production, so `MongoFixture` refuses port 27022, a data directory under `/var/lib/privapub`,
|
||||||
and in CI anything but the wrapper's mongod. With `PRIVAPUB_TEST_REQUIRE_MONGOD=1`, which the wrapper sets, a missing
|
and in CI anything but the wrapper's mongod. With `PRIVAPUB_TEST_REQUIRE_MONGOD=1`, which the wrapper sets, a missing
|
||||||
mongod fails the run instead of silently skipping half the tests.
|
mongod fails the run instead of silently skipping half the tests.
|
||||||
|
|||||||
@@ -0,0 +1,21 @@
|
|||||||
|
using MongoDB.Entities;
|
||||||
|
|
||||||
|
using PrivaPub.Infrastructure.Data;
|
||||||
|
using PrivaPub.Tests.Support;
|
||||||
|
|
||||||
|
namespace PrivaPub.Tests.Infrastructure
|
||||||
|
{
|
||||||
|
[Trait("Category", "Integration")]
|
||||||
|
public sealed class TopologyTests
|
||||||
|
{
|
||||||
|
// tools/ci/with-test-mongod.sh starts a one-member replica set, as production's mongod is: the backup's snapshot reads
|
||||||
|
// are what the tests exercise
|
||||||
|
[Fact]
|
||||||
|
public async Task The_test_mongod_is_a_replica_set_like_productions()
|
||||||
|
{
|
||||||
|
Assert.SkipUnless(MongoFixture.Enabled, MongoFixture.Skip);
|
||||||
|
Assert.SkipUnless(MongoFixture.Connection.Contains("directConnection=true"), "a mongod not started by tools/ci/with-test-mongod.sh");
|
||||||
|
Assert.True(await MongoTopology.IsReplicaSet(DB.Default.Database(), TestContext.Current.CancellationToken));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,23 @@
|
|||||||
|
using MongoDB.Bson;
|
||||||
|
using MongoDB.Driver;
|
||||||
|
|
||||||
|
namespace PrivaPub.Infrastructure.Data
|
||||||
|
{
|
||||||
|
// Whether the mongod is a replica set member (production's is a one-member set, rs0): only then can a read session
|
||||||
|
// hold one point in time across collections (a snapshot), which a backup needs to be consistent.
|
||||||
|
public static class MongoTopology
|
||||||
|
{
|
||||||
|
public static async Task<bool> IsReplicaSet(IMongoDatabase database, CancellationToken token)
|
||||||
|
{
|
||||||
|
try
|
||||||
|
{
|
||||||
|
var hello = await database.Client.GetDatabase("admin").RunCommandAsync<BsonDocument>(new BsonDocument("hello", 1), cancellationToken: token);
|
||||||
|
return hello.Contains("setName");
|
||||||
|
}
|
||||||
|
catch (MongoCommandException)
|
||||||
|
{
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -28,7 +28,7 @@
|
|||||||
"MongoSettings": {
|
"MongoSettings": {
|
||||||
"Database": "PrivaPub",
|
"Database": "PrivaPub",
|
||||||
"LogsDatabase": "logs",
|
"LogsDatabase": "logs",
|
||||||
"ConnectionString": "mongodb://127.0.0.1:27022"
|
"ConnectionString": "mongodb://127.0.0.1:27022/?directConnection=true"
|
||||||
},
|
},
|
||||||
"AppConfiguration": {
|
"AppConfiguration": {
|
||||||
"Version": "0.0.0",
|
"Version": "0.0.0",
|
||||||
@@ -74,7 +74,7 @@
|
|||||||
{
|
{
|
||||||
"Name": "MongoDBCapped",
|
"Name": "MongoDBCapped",
|
||||||
"Args": {
|
"Args": {
|
||||||
"databaseUrl": "mongodb://127.0.0.1:27022/logs",
|
"databaseUrl": "mongodb://127.0.0.1:27022/logs?directConnection=true",
|
||||||
"collectionName": "PrivaPub",
|
"collectionName": "PrivaPub",
|
||||||
"cappedMaxSizeMb": "1024",
|
"cappedMaxSizeMb": "1024",
|
||||||
"cappedMaxDocuments": "10000"
|
"cappedMaxDocuments": "10000"
|
||||||
|
|||||||
@@ -13,6 +13,8 @@ echo "== directories"
|
|||||||
install -d -o "$RUNNER" -g www-data -m 755 /var/www/$HOST
|
install -d -o "$RUNNER" -g www-data -m 755 /var/www/$HOST
|
||||||
install -d -o "$RUNNER" -g "$RUNNER" -m 750 /var/backups/$HOST
|
install -d -o "$RUNNER" -g "$RUNNER" -m 750 /var/backups/$HOST
|
||||||
install -d -o www-data -g www-data -m 750 /var/lib/privapub /var/lib/privapub/mongo
|
install -d -o www-data -g www-data -m 750 /var/lib/privapub /var/lib/privapub/mongo
|
||||||
|
# backups: the service writes them, and so does the deploy (as $RUNNER, a member of www-data) before each deploy
|
||||||
|
install -d -o www-data -g www-data -m 2770 /var/lib/privapub/backups
|
||||||
|
|
||||||
echo "== sudoers"
|
echo "== sudoers"
|
||||||
SUDOERS=/etc/sudoers.d/$RUNNER
|
SUDOERS=/etc/sudoers.d/$RUNNER
|
||||||
@@ -28,6 +30,18 @@ systemctl enable --now privapub-mongod >/dev/null
|
|||||||
systemctl enable $UNIT >/dev/null
|
systemctl enable $UNIT >/dev/null
|
||||||
systemctl is-active privapub-mongod
|
systemctl is-active privapub-mongod
|
||||||
|
|
||||||
|
echo "== replica set"
|
||||||
|
# once: mongod restarted with --replSet (PrivaPub stopped meanwhile), the one-member set rs0 made, its primary awaited
|
||||||
|
if [ "$(mongosh --quiet --port 27022 --eval 'db.hello().setName' 2>/dev/null)" != "rs0" ]; then
|
||||||
|
systemctl stop $UNIT
|
||||||
|
systemctl restart privapub-mongod
|
||||||
|
for _ in $(seq 1 60); do mongosh --quiet --port 27022 --eval 'db.hello()' >/dev/null 2>&1 && break; sleep 1; done
|
||||||
|
mongosh --quiet --port 27022 --eval "rs.initiate({_id: 'rs0', members: [{_id: 0, host: '127.0.0.1:27022'}]})" >/dev/null
|
||||||
|
for _ in $(seq 1 60); do [ "$(mongosh --quiet --port 27022 --eval 'db.hello().isWritablePrimary')" = "true" ] && break; sleep 1; done
|
||||||
|
systemctl start $UNIT
|
||||||
|
fi
|
||||||
|
echo "replica set: $(mongosh --quiet --port 27022 --eval 'db.hello().setName')"
|
||||||
|
|
||||||
echo "== nginx snippet and bootstrap vhost"
|
echo "== nginx snippet and bootstrap vhost"
|
||||||
install -m 644 "$SRC/nginx/privapub-headers.conf" /etc/nginx/snippets/privapub-headers.conf
|
install -m 644 "$SRC/nginx/privapub-headers.conf" /etc/nginx/snippets/privapub-headers.conf
|
||||||
if [ -f /root/.acme.sh/${HOST}_ecc/fullchain.cer ]; then
|
if [ -f /root/.acme.sh/${HOST}_ecc/fullchain.cer ]; then
|
||||||
|
|||||||
@@ -6,7 +6,8 @@ After=network.target
|
|||||||
Type=exec
|
Type=exec
|
||||||
User=www-data
|
User=www-data
|
||||||
Group=www-data
|
Group=www-data
|
||||||
ExecStart=/usr/bin/mongod --dbpath /var/lib/privapub/mongo --port 27022 --bind_ip 127.0.0.1 --noauth --wiredTigerCacheSizeGB 0.25 --quiet
|
# a one-member replica set (rs0), so a backup reads every collection at one instant; its oplog kept small
|
||||||
|
ExecStart=/usr/bin/mongod --dbpath /var/lib/privapub/mongo --port 27022 --bind_ip 127.0.0.1 --noauth --wiredTigerCacheSizeGB 0.25 --replSet rs0 --oplogSizeMB 990 --quiet
|
||||||
Restart=always
|
Restart=always
|
||||||
RestartSec=5
|
RestartSec=5
|
||||||
LimitNOFILE=64000
|
LimitNOFILE=64000
|
||||||
|
|||||||
@@ -1,6 +1,7 @@
|
|||||||
#!/usr/bin/env bash
|
#!/usr/bin/env bash
|
||||||
# Runs a command (normally `dotnet test`) with a throwaway mongod: a fresh data directory on a random localhost port,
|
# Runs a command (normally `dotnet test`) with a throwaway mongod: a fresh data directory on a random localhost port,
|
||||||
# stopped and deleted when the command ends. The box's own mongods (27017 shared, 27022 PrivaPub's) are never touched.
|
# stopped and deleted when the command ends. The box's own mongods (27017 shared, 27022 PrivaPub's) are never touched.
|
||||||
|
# Like production's, it is a one-member replica set (rs0), so the backup's snapshot reads are what the tests exercise.
|
||||||
# usage: tools/ci/with-test-mongod.sh dotnet test PrivaPub.sln -c Release
|
# usage: tools/ci/with-test-mongod.sh dotnet test PrivaPub.sln -c Release
|
||||||
set -euo pipefail
|
set -euo pipefail
|
||||||
|
|
||||||
@@ -39,11 +40,11 @@ trap cleanup EXIT INT TERM
|
|||||||
if command -v mongod >/dev/null 2>&1; then
|
if command -v mongod >/dev/null 2>&1; then
|
||||||
mkdir -p "$dir/db"
|
mkdir -p "$dir/db"
|
||||||
timeout 7200 mongod --dbpath "$dir/db" --port "$port" --bind_ip 127.0.0.1 --noauth \
|
timeout 7200 mongod --dbpath "$dir/db" --port "$port" --bind_ip 127.0.0.1 --noauth \
|
||||||
--wiredTigerCacheSizeGB 0.25 --quiet --logpath "$dir/mongod.log" &
|
--wiredTigerCacheSizeGB 0.25 --replSet rs0 --quiet --logpath "$dir/mongod.log" &
|
||||||
echo $! > "$dir/pid"
|
echo $! > "$dir/pid"
|
||||||
how="mongod $(mongod --version | head -1)"
|
how="mongod $(mongod --version | head -1)"
|
||||||
elif command -v podman >/dev/null 2>&1; then
|
elif command -v podman >/dev/null 2>&1; then
|
||||||
podman run -d --rm -p "127.0.0.1:$port:27017" docker.io/library/mongo:8 --quiet > "$dir/container"
|
podman run -d --rm -p "127.0.0.1:$port:27017" docker.io/library/mongo:8 --quiet --replSet rs0 > "$dir/container"
|
||||||
how="podman mongo:8"
|
how="podman mongo:8"
|
||||||
else
|
else
|
||||||
echo "::error::neither mongod nor podman is available for the test mongod"; exit 1
|
echo "::error::neither mongod nor podman is available for the test mongod"; exit 1
|
||||||
@@ -51,7 +52,17 @@ fi
|
|||||||
|
|
||||||
for _ in $(seq 1 60); do listening "$port" && break; sleep 0.5; done
|
for _ in $(seq 1 60); do listening "$port" && break; sleep 0.5; done
|
||||||
listening "$port" || { echo "::error::the test mongod did not start"; cat "$dir/mongod.log" 2>/dev/null | tail -20; exit 1; }
|
listening "$port" || { echo "::error::the test mongod did not start"; cat "$dir/mongod.log" 2>/dev/null | tail -20; exit 1; }
|
||||||
echo "test mongod: $how on 127.0.0.1:$port"
|
|
||||||
|
|
||||||
export PRIVAPUB_TEST_MONGOD=1 PRIVAPUB_TEST_REQUIRE_MONGOD=1 PRIVAPUB_TEST_MONGO="mongodb://127.0.0.1:$port"
|
# the one-member replica set, then wait until it is its primary
|
||||||
|
shell() {
|
||||||
|
if [ -f "$dir/container" ]; then podman exec "$(cat "$dir/container")" mongosh --quiet --eval "$1"
|
||||||
|
else mongosh --quiet --port "$port" --eval "$1"; fi
|
||||||
|
}
|
||||||
|
member=$([ -f "$dir/container" ] && echo "127.0.0.1:27017" || echo "127.0.0.1:$port")
|
||||||
|
shell "rs.initiate({_id: 'rs0', members: [{_id: 0, host: '$member'}]})" >/dev/null
|
||||||
|
for _ in $(seq 1 60); do [ "$(shell 'db.hello().isWritablePrimary' 2>/dev/null)" = "true" ] && break; sleep 0.5; done
|
||||||
|
[ "$(shell 'db.hello().isWritablePrimary' 2>/dev/null)" = "true" ] || { echo "::error::the test mongod never became its replica set's primary"; exit 1; }
|
||||||
|
echo "test mongod: $how on 127.0.0.1:$port, replica set rs0"
|
||||||
|
|
||||||
|
export PRIVAPUB_TEST_MONGOD=1 PRIVAPUB_TEST_REQUIRE_MONGOD=1 PRIVAPUB_TEST_MONGO="mongodb://127.0.0.1:$port/?directConnection=true"
|
||||||
"$@"
|
"$@"
|
||||||
@@ -2,7 +2,7 @@
|
|||||||
"MongoSettings": {
|
"MongoSettings": {
|
||||||
"Database": "PrivaPub",
|
"Database": "PrivaPub",
|
||||||
"LogsDatabase": "logs",
|
"LogsDatabase": "logs",
|
||||||
"ConnectionString": "mongodb://mongo:27017"
|
"ConnectionString": "mongodb://mongo:27017/?directConnection=true"
|
||||||
},
|
},
|
||||||
"AppConfiguration": {
|
"AppConfiguration": {
|
||||||
"Version": "0.0.0",
|
"Version": "0.0.0",
|
||||||
|
|||||||
@@ -21,7 +21,11 @@ pasture_base_up() {
|
|||||||
podman volume exists pasture-caddy-data || podman volume create pasture-caddy-data >/dev/null
|
podman volume exists pasture-caddy-data || podman volume create pasture-caddy-data >/dev/null
|
||||||
# (PrivaPub's uploads live in a volume: a container made again must not lose them)
|
# (PrivaPub's uploads live in a volume: a container made again must not lose them)
|
||||||
podman volume exists pasture-privapub-media || podman volume create --label pasture=1 pasture-privapub-media >/dev/null
|
podman volume exists pasture-privapub-media || podman volume create --label pasture=1 pasture-privapub-media >/dev/null
|
||||||
podman run -d --replace --name pasture-mongo --network $net --network-alias mongo docker.io/library/mongo:8 --quiet >/dev/null
|
# a one-member replica set, as production's, so backups read at one instant
|
||||||
|
podman run -d --replace --name pasture-mongo --network $net --network-alias mongo docker.io/library/mongo:8 --quiet --replSet rs0 >/dev/null
|
||||||
|
for _ in $(seq 1 60); do podman exec pasture-mongo mongosh --quiet --eval 'db.hello()' >/dev/null 2>&1 && break; sleep 1; done
|
||||||
|
podman exec pasture-mongo mongosh --quiet --eval "rs.initiate({_id: 'rs0', members: [{_id: 0, host: 'mongo:27017'}]})" >/dev/null
|
||||||
|
for _ in $(seq 1 60); do [ "$(podman exec pasture-mongo mongosh --quiet --eval 'db.hello().isWritablePrimary')" = "true" ] && break; sleep 1; done
|
||||||
caddy_up
|
caddy_up
|
||||||
local extra=()
|
local extra=()
|
||||||
for setting in ${PRIVAPUB_ENV:-}; do extra+=(-e "$setting"); done
|
for setting in ${PRIVAPUB_ENV:-}; do extra+=(-e "$setting"); done
|
||||||
|
|||||||
Reference in new issue
Block a user