A private fediverse on the workstation: PrivaPub against a real GoToSocial

tools/pasture/run.sh starts PrivaPub, GoToSocial and Mongo on one podman network behind Caddy's internal CA, and
interop.sh drives both through their own client APIs: follows (one to a locked account), posts, CW, replies, likes,
boosts, DMs, edits, deletes and unfollow. All 25 checks pass, three fresh runs in a row.

- Federation:AcceptAnyCertificate joins the two test-network switches; startup refuses all three in Production.
- WebFinger falls back to http only when AllowPlainHttp is on.
- A bootstrap logger, so a failure before the host is built is no longer silent.
- P4 is ticked in the roadmap, with what has not been run live (Lemmy, a Mastodon circle member).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB

.gitignore

@@ -401,3 +401,4 @@ FodyWeavers.xsd
 
 PrivaPub/media-store/
 PrivaPub/media-store-proxy/
+tools/pasture/.publish/

CLAUDE.md

@@ -69,7 +69,7 @@ PrivaPub/                         ASP.NET Core Web API, net10.0
   Infrastructure/
     Http/                         FederationHttp + SafeHttpHandlerFactory + IpRangeGuard: the only way out
     Jobs/                         JobQueue (leases), JobWorker, Backoff, HostCircuitBreaker
-    Ids/                          PrivacyIds (day-only ids for personas and groups, published-time ids for remote posts)
+    Ids/                          PrivacyIds (day-only ids for personas and groups, arrival-ordered ids for remote posts)
     Data/                         Indexes (created at start), EntityMaps.Warm, Migrations/_NNN_*.cs
     Cli/                          AdminCommands (`PrivaPub admin promote|demote <root>`)
     RateLimiting.cs               accounts (per client address) and inbox (per sending origin) policies
@@ -129,7 +129,8 @@ cd PrivaPub && ASPNETCORE_ENVIRONMENT=Development \
 
 Development config (`appsettings.Development.json`) binds HTTPS 7195 with HTTP/2 only; the overrides above make it
 curl-able. Outbound fetches only go to https DNS names resolving to public addresses; a test network (Pasture) sets
-`Federation__AllowPrivateNetworks=true` and `Federation__AllowPlainHttp=true`, which startup refuses in Production.
+`Federation__AllowPrivateNetworks=true`, `Federation__AllowPlainHttp=true` and `Federation__AcceptAnyCertificate=true`,
+which startup refuses in Production.
 
 Promoting an admin on the box (signing up as "admin" grants nothing):
 
@@ -226,6 +227,15 @@ cd /var/www/privapub.thepra.dev && sudo -u www-data ASPNETCORE_ENVIRONMENT=Produ
 - **Collection name = class name, so never rename an entity class.**
 - `Entity.ID` is a 24-character lowercase hex string; `GenerateNewID()` returns `object`, so cast it.
 - New fields must be additive: a deploy rollback restores the binary, not the database.
+- **Post ids are the timeline order, so they follow arrival, not `published`.** `PrivacyIds.Arrived` gives a remote post
+  published within the last hour (or in the future) a fresh `ObjectId`, which sorts after every post already stored,
+  and only backfill keeps a `published`-derived id. With `published` ids a reply arriving in the same second could sort
+  under the post it answers, and a late arrival landed behind a client's `since_id` and was never seen. `created_at`
+  comes from `CreationDate`, never from the id.
+- **The same ordering problem exists on the other side, so `published` carries milliseconds**
+  (`ActivityPubRenderer.Timestamp`). GoToSocial (ULIDs) and Mastodon (Snowflakes) derive a remote status's id from
+  `published` at millisecond resolution. With whole seconds, two of our posts from the same second sorted at random
+  there.
 - **Startup order:** `EntityMaps.Warm()` (every entity's class map, one at a time; two mapped at once throw "An item
   with the same key has already been added" and stay broken), then `MigrateAsync` (`Infrastructure/Data/Migrations`,
   `_NNN_` order, each runs once), then `Indexes.Create()`. A new entity needs nothing; a new unique index needs a
@@ -243,6 +253,9 @@ cd /var/www/privapub.thepra.dev && sudo -u www-data ASPNETCORE_ENVIRONMENT=Produ
 - Localised strings go through `IStringLocalizer<GenericRes>`.
 - Every `Display`/`ErrorMessage` resource key must exist in `FieldsNameResource`/`ErrorsResource`, including the
   `Designer.cs`, which the CLI build doesn't regenerate. A missing key throws at validation time.
+- **`cond ? value : default` with a value-type branch is the type's default, not null:** `false`, `0`, or year one
+  stored as an edit date on every remote post. Write `(T?)null`. It has shipped four times (`MastodonParams.Bool/Int`,
+  `NoteParser.Int`, `NoteParser.Time`).
 - ActivityPub output is built with `System.Text.Json.Nodes` in `ActivityPubRenderer`, not typed models. Inbound
   documents are read through `InboxService.Id`/`Value`, which handle string, object and array.
 - Libraries chosen for the roadmap: HtmlSanitizer, Markdig (`DisableHtml`), NSign (RFC 9421 inbound), OpenIddict +
@@ -261,8 +274,39 @@ Beyond the tests, verify by building, running locally, and exercising:
 - the client API (sign up, create an avatar, a group, a post);
 - the ActivityPub endpoints with curl and `Accept: application/activity+json`.
 
-Interop is checked against real servers: Fediverse Pasture with podman on the workstation, verify.funfedi.dev, and
-the owner's GoToSocial at social.arasaka.software. **Ask before acting from the owner's GoToSocial account.**
+Interop is checked against real servers, starting with the workstation's own pasture:
+
+```bash
+DOTNET=~/.dotnet/dotnet tools/pasture/run.sh up   # podman: PrivaPub + the latest GoToSocial + Mongo, behind Caddy
+tools/pasture/interop.sh                          # 25 checks, each side driven through its own Mastodon API
+tools/pasture/run.sh down
+```
+
+- **Two sites on one podman network, one Caddy in front.** `privapub.test` and `gts.test` are network aliases of the
+  Caddy container, which serves both with its internal CA (`tls internal`). Both servers are told to accept any
+  certificate: PrivaPub through `appsettings.Pasture.json`, GoToSocial through `GTS_HTTP_CLIENT_TLS_INSECURE_SKIP_VERIFY`.
+  GoToSocial WebFingers and fetches over https only, so plain http between them is not an option.
+- From the workstation, PrivaPub's API is `http://127.0.0.1:6971`. GoToSocial is reached as `https://gts.test:6443`
+  with `curl -k --resolve gts.test:6443:127.0.0.1`, because its sign-in cookie is bound to the host name.
+- **GoToSocial's cached home timeline can stop taking new posts after its first read**, its owner's own included, while
+  a `min_id` query shows them all. So a delivery is checked by looking the object up by URI with `resolve=false`
+  (`on_gts`), which answers from GoToSocial's database and never fetches from us. A home-timeline check there proves
+  nothing, in either direction. A deleted status still turns up in that search as a "deleted status" stub, so a delete
+  is checked as a 404 on `/api/v1/statuses/{id}`.
+- **GoToSocial creates its accounts locked**, so `interop.sh` approves alice's request through
+  `/api/v1/follow_requests`. That also checks our pending (`requested`) state and the manual Accept.
+- The scenario covers:
+  - discovery and follows both ways;
+  - posts and CW;
+  - a reply and its notification;
+  - likes and boosts both ways;
+  - DMs both ways, and the DM staying off public timelines;
+  - edit, delete both ways, and unfollow.
+- `run.sh up` replaces every container, Mongo included, so each run starts clean. To keep the data, republish into
+  `tools/pasture/.publish` and `podman restart pasture-privapub`; that is how a migration is tried on dirty data.
+
+After the pasture, verify.funfedi.dev and the owner's GoToSocial at social.arasaka.software. **Ask before acting from
+the owner's GoToSocial account.**
 
 ## Deploy
 

FEDERATION.md

@@ -10,6 +10,20 @@ PrivaPub is an ActivityPub server written in C#. This document follows
 - [HTTP Signatures](https://datatracker.ietf.org/doc/html/draft-cavage-http-signatures), `rsa-sha256` / `hs2019` with RSA keys
 - [NodeInfo](https://nodeinfo.diaspora.software/) 2.0 and 2.1
 
+## Tested against
+
+- **GoToSocial 0.22.1, end to end.** It runs in a private network on the workstation (`tools/pasture/`) and is driven
+  through its own client API. Checked both ways:
+  - follows, including to a locked account;
+  - public posts with a content warning;
+  - replies with notifications;
+  - likes and boosts;
+  - direct messages;
+  - edits and deletes;
+  - unfollow.
+- **Mastodon, Misskey, Lemmy and PeerTube, by unit tests only.** The tests feed the parser documents written in each
+  server's shape. No live exchange with any of them has run yet.
+
 ## Supported FEPs
 
 - [FEP-67ff: FEDERATION.md](https://codeberg.org/fediverse/fep/src/branch/main/fep/67ff/fep-67ff.md)

PrivaPub/Federation/Actors/RemoteActorService.cs

@@ -6,6 +6,7 @@ using PrivaPub.StaticServices;
 using System.Text.Json;
 
 using Microsoft.Extensions.Caching.Memory;
+using Microsoft.Extensions.Options;
 
 using PrivaPub.Federation.Objects;
 using PrivaPub.Federation.Signing;
@@ -32,13 +33,16 @@ namespace PrivaPub.Federation.Actors
 		readonly ILocalActorService _localActors;
 		readonly IMemoryCache _cache;
 		readonly DbEntities _dbEntities;
+		readonly IOptionsMonitor<FederationOptions> _options;
 
-		public RemoteActorService(IFederationHttp http, ILocalActorService localActors, IMemoryCache cache, DbEntities dbEntities)
+		public RemoteActorService(IFederationHttp http, ILocalActorService localActors, IMemoryCache cache, DbEntities dbEntities,
+			IOptionsMonitor<FederationOptions> options = default)
 		{
 			_http = http;
 			_localActors = localActors;
 			_cache = cache;
 			_dbEntities = dbEntities;
+			_options = options;
 		}
 
 		public async Task<FetchedJson> FetchObject(string uri, CancellationToken token)
@@ -122,8 +126,11 @@ namespace PrivaPub.Federation.Actors
 			if (parts is not { Length: 2 } || string.IsNullOrEmpty(parts[0]) || string.IsNullOrEmpty(parts[1]))
 				return default;
 
-			var url = $"https://{parts[1]}/.well-known/webfinger?resource={Uri.EscapeDataString($"acct:{parts[0]}@{parts[1]}")}";
-			using var fetched = await _http.GetJson(url, "application/jrd+json, application/json", sign: default, token);
+			var query = $"/.well-known/webfinger?resource={Uri.EscapeDataString($"acct:{parts[0]}@{parts[1]}")}";
+			using var fetched = await _http.GetJson($"https://{parts[1]}{query}", "application/jrd+json, application/json", sign: default, token)
+				?? (_options?.CurrentValue.AllowPlainHttp == true
+					? await _http.GetJson($"http://{parts[1]}{query}", "application/jrd+json, application/json", sign: default, token)
+					: default);
 			if (fetched == default)
 				return default;
 			var document = fetched.Document;

PrivaPub/Infrastructure/Http/FederationOptions.cs

@@ -5,5 +5,6 @@ namespace PrivaPub.Infrastructure.Http
 		public bool AllowPrivateNetworks { get; set; }
 		public bool AllowPlainHttp { get; set; }
 		public bool SecureMode { get; set; }
+		public bool AcceptAnyCertificate { get; set; }
 	}
 }

PrivaPub/Infrastructure/Http/SafeHttpHandlerFactory.cs

@@ -1,4 +1,5 @@
 using System.Net;
+using System.Net.Security;
 using System.Net.Sockets;
 
 namespace PrivaPub.Infrastructure.Http
@@ -12,6 +13,9 @@ namespace PrivaPub.Infrastructure.Http
 	{
 		public static SocketsHttpHandler Create(FederationOptions options) => new()
 		{
+			SslOptions = options.AcceptAnyCertificate
+				? new SslClientAuthenticationOptions { RemoteCertificateValidationCallback = (_, _, _, _) => true }
+				: new SslClientAuthenticationOptions(),
 			AllowAutoRedirect = false,
 			UseProxy = false,
 			UseCookies = false,

PrivaPub/Program.cs

@@ -21,6 +21,8 @@ using PrivaPub.Models;
 using PrivaPub.Services;
 using PrivaPub.StaticServices;
 
+Log.Logger = new LoggerConfiguration().WriteTo.Console().CreateBootstrapLogger();
+
 try
 {
 	var builder = WebApplication.CreateBuilder(args);
@@ -71,8 +73,8 @@ try
 	}
 
 	var federationOptions = builder.Configuration.GetSection("Federation").Get<FederationOptions>() ?? new();
-	if (builder.Environment.IsProduction() && (federationOptions.AllowPrivateNetworks || federationOptions.AllowPlainHttp))
-		throw new InvalidOperationException("Federation:AllowPrivateNetworks and Federation:AllowPlainHttp are for test networks and must stay off in Production.");
+	if (builder.Environment.IsProduction() && (federationOptions.AllowPrivateNetworks || federationOptions.AllowPlainHttp || federationOptions.AcceptAnyCertificate))
+		throw new InvalidOperationException("Federation:AllowPrivateNetworks, AllowPlainHttp and AcceptAnyCertificate are for test networks and must stay off in Production.");
 
 	try
 	{
@@ -189,4 +191,8 @@ catch (Exception ex)
 	Log.ForContext<Program>().Fatal(ex, $"{nameof(Program)}.{nameof(Program)}()");
 	Environment.ExitCode = 1;
 }
+finally
+{
+	await Log.CloseAndFlushAsync();
+}
 

docs/ROADMAP.md

@@ -9,7 +9,7 @@ Written 2026-10-01 from the original 2023 code, the decePubClient UI, a federati
 - [x] P1.2 Social graph and timelines: v1.3.0, deployed and verified 2026-10-01
 - [x] P2 Mastodon client API: v1.4.0, deployed 2026-10-01; OAuth and the anonymous API verified on production, the signed-in API verified locally (no real-client login on production yet)
 - [x] P3 Social features: v1.5.0, deployed 2026-10-01; media, proxy, blocks, mutes, bookmarks, pins and reports verified by tests and locally (no upload on production yet)
-- [ ] P4 Groups and privacy features
+- [x] P4 Groups and privacy features: v1.6.0, deployed 2026-10-01; communities, circles and local-only located posts verified by tests. v1.6.1 adds the pasture (`tools/pasture/`): live interop with GoToSocial 0.22.1 passes all 25 checks, three runs in a row. Lemmy and a live Mastodon circle member are not run yet; the pasture has GoToSocial only
 - [ ] P5 FEPs and polish
 
 ## Intent
@@ -417,8 +417,10 @@ The first refactor commit is a pure move with namespaces only. Logic changes fol
   `Fixtures/{mastodon,gotosocial,misskey,akkoma,lemmy}/`. Renderers are checked against golden JSON. Inbound
   scenarios run as integration tests on a throwaway mongod.
 - **Interop:** **Fediverse Pasture** runs on the workstation with podman: Mastodon, GoToSocial, Misskey/Sharkey,
-  Akkoma, plus Lemmy for P4. `tools/pasture/docker-compose.privapub.yml` runs PrivaPub with `AllowPrivateNetworks`;
-  startup asserts that flag is false in Production. The matrix, per peer:
+  Akkoma, plus Lemmy for P4. As built, `tools/pasture/run.sh` runs PrivaPub and GoToSocial behind one Caddy on a
+  podman network, with `AllowPrivateNetworks`, `AllowPlainHttp` and `AcceptAnyCertificate`; startup refuses all three in
+  Production. `tools/pasture/interop.sh` drives the matrix for GoToSocial; the other peers are still to be added. The
+  matrix, per peer:
   - follow both ways;
   - public, unlisted, followers-only and direct posts both ways;
   - a reply landing in the remote thread;

tools/pasture/Caddyfile

@@ -0,0 +1,14 @@
+{
+	local_certs
+	skip_install_trust
+}
+
+privapub.test {
+	tls internal
+	reverse_proxy pasture-privapub:80
+}
+
+gts.test {
+	tls internal
+	reverse_proxy pasture-gts:80
+}

tools/pasture/appsettings.Pasture.json

@@ -0,0 +1,33 @@
+{
+	"MongoSettings": {
+		"Database": "PrivaPub",
+		"LogsDatabase": "logs",
+		"ConnectionString": "mongodb://mongo:27017"
+	},
+	"AppConfiguration": {
+		"Version": "0.0.0",
+		"MaxAllowedUploadFiles": 3,
+		"MaxAllowedFileSize": 2097152,
+		"SupportedLanguages": [ "en" ],
+		"BackendBaseAddress": "https://privapub.test",
+		"FrontendBaseAddress": "https://privapub.test",
+		"HashingOptions": { "Iterations": 10101 },
+		"Jwt": {
+			"Key": "pasture-only-key-not-a-secret-pasture-only-key-not-a-secret-0123456789",
+			"Issuer": "http://privapub.test",
+			"Audience": "http://privapub.test",
+			"HoursTimeout": 24
+		}
+	},
+	"Federation": {
+		"AllowPrivateNetworks": true,
+		"AllowPlainHttp": true,
+		"AcceptAnyCertificate": true
+	},
+	"Media": { "Root": "/tmp/privapub-media" },
+	"Kestrel": { "Endpoints": { "Http": { "Url": "http://0.0.0.0:80", "Protocols": "Http1AndHttp2" } } },
+	"Serilog": {
+		"MinimumLevel": { "Default": "Information", "Override": { "Microsoft": "Warning", "System": "Warning" } },
+		"WriteTo": [ { "Name": "Console" } ]
+	}
+}

tools/pasture/interop.sh

@@ -0,0 +1,117 @@
+#!/usr/bin/env bash
+# Drives PrivaPub and GoToSocial (started by run.sh) through each other's federation, as their users would.
+set -uo pipefail
+P=http://127.0.0.1:6971; G=https://gts.test:6443
+gcurl() { curl -k --resolve gts.test:6443:127.0.0.1 "$@"; }
+work=$(mktemp -d); trap 'rm -rf "$work"' EXIT
+pass=0; fail=0
+ok() { echo "  ok   $*"; pass=$((pass+1)); }
+ko() { echo "  FAIL $*"; fail=$((fail+1)); }
+j() { python3 -c "import sys,json
+try: d=json.load(sys.stdin)
+except Exception: d=None
+$1" 2>/dev/null; }
+until_true() { local tries=$1; shift; for i in $(seq 1 "$tries"); do if eval "$@"; then return 0; fi; sleep 2; done; return 1; }
+# GoToSocial's cached home timeline can stop taking new posts after its first read, so a delivery is checked by looking
+# the object up by URI with resolve=false: that answers from GoToSocial's database and never fetches from us.
+on_gts() { gcurl -s -G -H "$GH" "$G/api/v2/search" --data-urlencode "q=$1" -d resolve=false -d type=statuses; }
+
+# --- PrivaPub persona and token
+root=$(curl -s -X POST $P/clientapi/user/signup -H 'Content-Type: application/json' -d '{"userName":"pastureroot","password":"Pasture-Pass-1!"}')
+jwt=$(echo "$root" | j "print(d['token'])"); rootid=$(echo "$root" | j "print(d['userId'])")
+if [ -z "$jwt" ]; then jwt=$(curl -s -X POST $P/clientapi/user/login -H 'Content-Type: application/json' -d '{"userName":"pastureroot","password":"Pasture-Pass-1!"}' | j "print(d['token'])"); rootid=$(curl -s -X POST $P/clientapi/user/login -H 'Content-Type: application/json' -d '{"userName":"pastureroot","password":"Pasture-Pass-1!"}' | j "print(d['userId'])"); fi
+curl -s -o /dev/null -X POST $P/clientapi/avatar/private/insert -H 'Content-Type: application/json' -H "Authorization: Bearer $jwt" \
+	-d "{\"rootId\":\"$rootid\",\"userName\":\"alice\",\"name\":\"Alice of PrivaPub\",\"biography\":\"testing federation\"}"
+app=$(curl -s -X POST $P/api/v1/apps -d 'client_name=pasture&redirect_uris=urn:ietf:wg:oauth:2.0:oob&scopes=read+write+follow')
+cid=$(echo "$app" | j "print(d['client_id'])"); cs=$(echo "$app" | j "print(d['client_secret'])")
+q="client_id=$cid&redirect_uri=urn:ietf:wg:oauth:2.0:oob&response_type=code&scope=read+write+follow"
+xt=$(curl -s -c $work/pj -b $work/pj "$P/oauth/login?returnUrl=/oauth/authorize?$q" | grep -o 'name="__RequestVerificationToken" type="hidden" value="[^"]*"' | sed 's/.*value="//;s/"//')
+curl -s -o /dev/null -c $work/pj -b $work/pj -X POST $P/oauth/login --data-urlencode "returnUrl=/oauth/authorize?$q" --data-urlencode "__RequestVerificationToken=$xt" -d 'userName=pastureroot&password=Pasture-Pass-1!'
+curl -s -c $work/pj -b $work/pj "$P/oauth/authorize?$q&signed_in=1" > $work/choose.html
+form=$(python3 - "$work/choose.html" <<'PY'
+import re,sys,urllib.parse,html
+s=open(sys.argv[1]).read()
+pairs=[(k,html.unescape(v)) for k,v in re.findall(r'<input type="hidden" name="([^"]*)" value="([^"]*)"',s)]
+print(urllib.parse.urlencode(pairs+[("avatarId",re.findall(r'name="avatarId" value="([^"]*)"',s)[0]),("decision","allow")]))
+PY
+)
+code=$(curl -s -c $work/pj -b $work/pj -X POST $P/oauth/authorize --data "$form" | grep -o '<code>[^<]*</code>' | sed 's/<[^>]*>//g')
+PT=$(curl -s -X POST $P/oauth/token -d "grant_type=authorization_code&code=$code&client_id=$cid&client_secret=$cs&redirect_uri=urn:ietf:wg:oauth:2.0:oob" | j "print(d['access_token'])")
+PH="Authorization: Bearer $PT"
+[ -n "$PT" ] && ok "PrivaPub token for alice" || { ko "PrivaPub token"; exit 1; }
+
+# --- GoToSocial token
+gapp=$(gcurl -s -X POST $G/api/v1/apps -d 'client_name=pasture&redirect_uris=urn:ietf:wg:oauth:2.0:oob&scopes=read+write+follow')
+gid=$(echo "$gapp" | j "print(d['client_id'])"); gs=$(echo "$gapp" | j "print(d['client_secret'])")
+gcurl -s -o /dev/null -c $work/gj -b $work/gj "$G/oauth/authorize?client_id=$gid&redirect_uri=urn:ietf:wg:oauth:2.0:oob&response_type=code&scope=read+write+follow"
+gcurl -s -o /dev/null -c $work/gj -b $work/gj -X POST $G/auth/sign_in --data-urlencode 'username=gtsuser@gts.test' --data-urlencode 'password=Gts-Pasture-Pass-1!'
+gcode=$(gcurl -s -o /dev/null -w '%{redirect_url}' -c $work/gj -b $work/gj -X POST "$G/oauth/authorize" | sed -n 's/.*[?&]code=\([^&]*\).*/\1/p')
+GT=$(gcurl -s -X POST $G/oauth/token -d "grant_type=authorization_code&code=$gcode&client_id=$gid&client_secret=$gs&redirect_uri=urn:ietf:wg:oauth:2.0:oob&scope=read+write+follow" | j "print(d['access_token'])")
+GH="Authorization: Bearer $GT"
+[ -n "$GT" ] && ok "GoToSocial token for gtsuser" || { ko "GoToSocial token (code '$gcode')"; exit 1; }
+
+echo "discovery"
+alice_on_gts=$(gcurl -s -H "$GH" "$G/api/v2/search?q=@alice@privapub.test&resolve=true&type=accounts" | j "print(d['accounts'][0]['id'])")
+[ -n "$alice_on_gts" ] && ok "GoToSocial resolves @alice@privapub.test" || ko "GoToSocial cannot resolve alice"
+gts_on_pp=$(curl -s -H "$PH" "$P/api/v2/search?q=gtsuser@gts.test&resolve=true&type=accounts" | j "print(d['accounts'][0]['id'])")
+[ -n "$gts_on_pp" ] && ok "PrivaPub resolves @gtsuser@gts.test" || ko "PrivaPub cannot resolve gtsuser"
+
+echo "follows"
+gcurl -s -o /dev/null -X POST -H "$GH" $G/api/v1/accounts/$alice_on_gts/follow
+until_true 20 '[ "$(gcurl -s -H "$GH" "$G/api/v1/accounts/relationships?id[]=$alice_on_gts" | j "print(d[0][\"following\"])")" = "True" ]' && ok "gtsuser follows alice (Accept arrived)" || ko "gtsuser's follow of alice not accepted"
+curl -s -o /dev/null -X POST -H "$PH" $P/api/v1/accounts/$gts_on_pp/follow
+until_true 10 '[ "$(curl -s -H "$PH" "$P/api/v1/accounts/relationships?id[]=$gts_on_pp" | j "print(d[0][\"requested\"])")" = "True" ]' && ok "alice's follow of locked gtsuser waits as a request" || ko "alice's follow is not shown as requested"
+until_true 20 'gcurl -s -H "$GH" "$G/api/v1/follow_requests" | j "print(any(a[\"id\"]==\"$alice_on_gts\" for a in d))" | grep -q True' && ok "the request reaches gtsuser's follow requests" || ko "follow request missing on GoToSocial"
+gcurl -s -o /dev/null -X POST -H "$GH" $G/api/v1/follow_requests/$alice_on_gts/authorize
+until_true 20 '[ "$(curl -s -H "$PH" "$P/api/v1/accounts/relationships?id[]=$gts_on_pp" | j "print(d[0][\"following\"])")" = "True" ]' && ok "alice follows gtsuser (Accept arrived)" || ko "alice's follow of gtsuser not accepted"
+
+echo "posts"
+pp_status=$(curl -s -X POST -H "$PH" $P/api/v1/statuses -d 'status=Hello from PrivaPub #pasture&visibility=public')
+pp_post=$(echo "$pp_status" | j "print(d['id'])"); pp_uri=$(echo "$pp_status" | j "print(d['uri'])")
+until_true 20 'on_gts "$pp_uri" | j "print(len(d[\"statuses\"]))" | grep -q 1' && ok "alice's post reaches GoToSocial" || ko "alice's post missing on GoToSocial"
+gts_post=$(gcurl -s -X POST -H "$GH" $G/api/v1/statuses -d 'status=Hello from GoToSocial&visibility=public' | j "print(d['id'])")
+until_true 20 'curl -s -H "$PH" "$P/api/v1/timelines/home" | grep -q "Hello from GoToSocial"' && ok "gtsuser's post reaches alice's home" || ko "gtsuser's post missing on PrivaPub"
+[ "$(curl -s -H "$PH" "$P/api/v1/timelines/home" | j "print(next(s['edited_at'] for s in d if 'Hello from GoToSocial' in s['content']))")" = "None" ] && ok "an unedited remote post carries no edited_at" || ko "unedited remote post reports an edit"
+cw_status=$(curl -s -X POST -H "$PH" $P/api/v1/statuses -d 'status=behind a warning&spoiler_text=spoilers&visibility=public')
+cw=$(echo "$cw_status" | j "print(d['id'])"); cw_uri=$(echo "$cw_status" | j "print(d['uri'])")
+until_true 20 'on_gts "$cw_uri" | j "print(any(s[\"spoiler_text\"]==\"spoilers\" and s[\"sensitive\"] for s in d[\"statuses\"]))" | grep -q True' && ok "content warning survives to GoToSocial" || ko "content warning lost"
+cw_on_gts=$(on_gts "$cw_uri" | j "print(d['statuses'][0]['id'])")
+
+echo "replies and mentions"
+pp_on_gts=$(on_gts "$pp_uri" | j "print(d['statuses'][0]['id'])")
+gcurl -s -o /dev/null -X POST -H "$GH" $G/api/v1/statuses -d "status=@alice@privapub.test nice to meet you&in_reply_to_id=$pp_on_gts&visibility=public"
+until_true 20 'curl -s -H "$PH" "$P/api/v1/notifications" | j "print(any(n[\"type\"]==\"mention\" for n in d))" | grep -q True' && ok "gtsuser's reply notifies alice" || ko "reply did not notify alice"
+until_true 10 '[ "$(curl -s -H "$PH" "$P/api/v1/statuses/$pp_post/context" | j "print(len(d[\"descendants\"]))")" -ge 1 ]' && ok "the reply threads under alice's post" || ko "reply not threaded"
+
+echo "likes and boosts"
+gts_on_pp_post=$(curl -s -H "$PH" "$P/api/v1/timelines/home" | j "print(next(s['id'] for s in d if 'Hello from GoToSocial' in s['content']))")
+curl -s -o /dev/null -X POST -H "$PH" $P/api/v1/statuses/$gts_on_pp_post/favourite
+curl -s -o /dev/null -X POST -H "$PH" $P/api/v1/statuses/$gts_on_pp_post/reblog
+until_true 20 'gcurl -s -H "$GH" "$G/api/v1/notifications" | j "print(any(n[\"type\"]==\"favourite\" for n in d))" | grep -q True' && ok "alice's like reaches GoToSocial" || ko "like not received"
+until_true 20 'gcurl -s -H "$GH" "$G/api/v1/notifications" | j "print(any(n[\"type\"]==\"reblog\" for n in d))" | grep -q True' && ok "alice's boost reaches GoToSocial" || ko "boost not received"
+gcurl -s -o /dev/null -X POST -H "$GH" $G/api/v1/statuses/$pp_on_gts/favourite
+gcurl -s -o /dev/null -X POST -H "$GH" $G/api/v1/statuses/$pp_on_gts/reblog
+until_true 20 '[ "$(curl -s -H "$PH" "$P/api/v1/statuses/$pp_post" | j "print(d[\"favourites_count\"])")" = "1" ]' && ok "gtsuser's like counts on PrivaPub" || ko "like not counted"
+until_true 20 'curl -s -H "$PH" "$P/api/v1/notifications" | j "print(any(n[\"type\"]==\"reblog\" for n in d))" | grep -q True' && ok "gtsuser's boost notifies alice" || ko "boost not notified"
+
+echo "direct messages"
+curl -s -o /dev/null -X POST -H "$PH" $P/api/v1/statuses -d 'status=@gtsuser@gts.test a secret&visibility=direct'
+until_true 20 'gcurl -s -H "$GH" "$G/api/v1/conversations" | grep -q "a secret"' && ok "alice's DM reaches gtsuser" || ko "DM missing on GoToSocial"
+gcurl -s -o /dev/null -X POST -H "$GH" $G/api/v1/statuses -d 'status=@alice@privapub.test a secret back&visibility=direct'
+until_true 20 'curl -s -H "$PH" "$P/api/v1/conversations" | grep -q "a secret back"' && ok "gtsuser's DM reaches alice" || ko "DM missing on PrivaPub"
+until_true 5 '! gcurl -s -H "$GH" "$G/api/v1/timelines/public?local=false" | grep -q "a secret"' && ok "the DM is not public on GoToSocial" || ko "DM leaked to a public timeline"
+
+echo "edits and deletes"
+curl -s -o /dev/null -X PUT -H "$PH" $P/api/v1/statuses/$pp_post -d 'status=Hello from PrivaPub, edited'
+until_true 20 'gcurl -s -H "$GH" "$G/api/v1/statuses/$pp_on_gts" | grep -q "edited"' && ok "alice's edit reaches GoToSocial" || ko "edit not applied"
+curl -s -o /dev/null -X DELETE -H "$PH" $P/api/v1/statuses/$cw
+[ -n "$cw_on_gts" ] && until_true 20 '[ "$(gcurl -s -o /dev/null -w "%{http_code}" -H "$GH" "$G/api/v1/statuses/$cw_on_gts")" = "404" ]' && ok "alice's delete reaches GoToSocial" || ko "delete not applied"
+gcurl -s -o /dev/null -X DELETE -H "$GH" $G/api/v1/statuses/$gts_post
+until_true 20 '! curl -s -H "$PH" "$P/api/v1/timelines/home" | grep -q "Hello from GoToSocial"' && ok "gtsuser's delete reaches PrivaPub" || ko "remote delete not applied"
+
+echo "unfollow"
+curl -s -o /dev/null -X POST -H "$PH" $P/api/v1/accounts/$gts_on_pp/unfollow
+until_true 20 '[ "$(gcurl -s -H "$GH" "$G/api/v1/accounts/relationships?id[]=$alice_on_gts" | j "print(d[0][\"followed_by\"])")" = "False" ]' && ok "alice's unfollow reaches GoToSocial" || ko "unfollow not applied"
+
+echo; echo "$pass passed, $fail failed"
+[ "$fail" = 0 ]

tools/pasture/run.sh

@@ -0,0 +1,44 @@
+#!/usr/bin/env bash
+# A private test fediverse on one podman network: PrivaPub (privapub.test) and GoToSocial (gts.test) behind Caddy,
+# whose internal CA both sides are told to accept. From the workstation: PrivaPub's API at http://127.0.0.1:6971,
+# both sites at https://{privapub,gts}.test:6443 (curl --resolve ...:6443:127.0.0.1 -k).
+# usage: tools/pasture/run.sh up | down | logs <name>
+set -euo pipefail
+here="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"; repo="$(cd "$here/../.." && pwd)"
+net=privapub-pasture; publish="$here/.publish"
+
+case "${1:-up}" in
+up)
+	dotnet=${DOTNET:-$(command -v dotnet || echo ~/.dotnet/dotnet)}
+	"$dotnet" publish "$repo/PrivaPub/PrivaPub.csproj" -c Release -r linux-x64 --self-contained true -o "$publish" -v quiet
+	cp "$here/appsettings.Pasture.json" "$publish/"
+	podman network exists $net || podman network create $net >/dev/null
+	podman run -d --replace --name pasture-mongo --network $net --network-alias mongo docker.io/library/mongo:8 --quiet >/dev/null
+	podman run -d --replace --name pasture-caddy --network $net --network-alias privapub.test --network-alias gts.test \
+		-p 127.0.0.1:6443:443 --sysctl net.ipv4.ip_unprivileged_port_start=0 -v "$here/Caddyfile:/etc/caddy/Caddyfile:Z,ro" \
+		docker.io/library/caddy:2 >/dev/null
+	podman run -d --replace --name pasture-privapub --network $net -p 127.0.0.1:6971:80 \
+		--sysctl net.ipv4.ip_unprivileged_port_start=0 -e ASPNETCORE_ENVIRONMENT=Pasture -w /app -v "$publish:/app:Z,ro" \
+		mcr.microsoft.com/dotnet/runtime-deps:10.0 /app/PrivaPub >/dev/null
+	podman run -d --replace --name pasture-gts --network $net -p 127.0.0.1:6972:80 \
+		--sysctl net.ipv4.ip_unprivileged_port_start=0 \
+		-e GTS_HOST=gts.test -e GTS_PROTOCOL=https -e GTS_PORT=80 -e GTS_BIND_ADDRESS=0.0.0.0 -e GTS_HTTP_CLIENT_TLS_INSECURE_SKIP_VERIFY=true \
+		-e GTS_DB_TYPE=sqlite -e GTS_DB_ADDRESS=/gotosocial/storage/sqlite.db -e GTS_STORAGE_LOCAL_BASE_PATH=/gotosocial/storage \
+		-e GTS_LETSENCRYPT_ENABLED=false -e GTS_HTTP_CLIENT_ALLOW_IPS=10.0.0.0/8,172.16.0.0/12,192.168.0.0/16 \
+		-e GTS_TRUSTED_PROXIES=0.0.0.0/0 -e GTS_LOG_LEVEL=info -e GTS_INSTANCE_EXPOSE_PUBLIC_TIMELINE=true \
+		docker.io/superseriousbusiness/gotosocial:latest >/dev/null
+	for i in $(seq 1 60); do curl -fs -o /dev/null http://127.0.0.1:6971/build.json && curl -fs -o /dev/null http://127.0.0.1:6972/api/v1/instance && break; sleep 2; done
+	podman exec pasture-gts /gotosocial/gotosocial admin account create --username gtsuser --email gtsuser@gts.test --password 'Gts-Pasture-Pass-1!' >/dev/null 2>&1 || true
+	podman exec pasture-gts /gotosocial/gotosocial admin account confirm --username gtsuser >/dev/null 2>&1 || true
+	podman restart pasture-gts >/dev/null
+	for i in $(seq 1 60); do curl -fs -o /dev/null http://127.0.0.1:6972/api/v1/instance && break; sleep 2; done
+	echo "privapub: http://127.0.0.1:6971, https://privapub.test:6443   gotosocial: https://gts.test:6443"
+	;;
+down)
+	podman rm -f pasture-caddy pasture-privapub pasture-gts pasture-mongo >/dev/null 2>&1 || true
+	podman network rm $net >/dev/null 2>&1 || true
+	;;
+logs)
+	podman logs --tail 200 "pasture-${2:-privapub}"
+	;;
+esac