From 8f4d6cbdf9b961e88d092be2ebe74a2eda2edcac Mon Sep 17 00:00:00 2001 From: thepra Date: Thu, 1 Oct 2026 13:19:16 +0200 Subject: [PATCH] A private fediverse on the workstation: PrivaPub against a real GoToSocial tools/pasture/run.sh starts PrivaPub, GoToSocial and Mongo on one podman network behind Caddy's internal CA, and interop.sh drives both through their own client APIs: follows (one to a locked account), posts, CW, replies, likes, boosts, DMs, edits, deletes and unfollow. All 25 checks pass, three fresh runs in a row. - Federation:AcceptAnyCertificate joins the two test-network switches; startup refuses all three in Production. - WebFinger falls back to http only when AllowPlainHttp is on. - A bootstrap logger, so a failure before the host is built is no longer silent. - P4 is ticked in the roadmap, with what has not been run live (Lemmy, a Mastodon circle member). Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB --- .gitignore | 1 + CLAUDE.md | 52 +++++++- FEDERATION.md | 14 +++ .../Federation/Actors/RemoteActorService.cs | 13 +- .../Infrastructure/Http/FederationOptions.cs | 1 + .../Http/SafeHttpHandlerFactory.cs | 4 + PrivaPub/Program.cs | 10 +- docs/ROADMAP.md | 8 +- tools/pasture/Caddyfile | 14 +++ tools/pasture/appsettings.Pasture.json | 33 +++++ tools/pasture/interop.sh | 117 ++++++++++++++++++ tools/pasture/run.sh | 44 +++++++ 12 files changed, 299 insertions(+), 12 deletions(-) create mode 100644 tools/pasture/Caddyfile create mode 100644 tools/pasture/appsettings.Pasture.json create mode 100755 tools/pasture/interop.sh create mode 100755 tools/pasture/run.sh diff --git a/.gitignore b/.gitignore index 7bb4345..fe790b0 100644 --- a/.gitignore +++ b/.gitignore @@ -401,3 +401,4 @@ FodyWeavers.xsd PrivaPub/media-store/ PrivaPub/media-store-proxy/ +tools/pasture/.publish/ diff --git a/CLAUDE.md b/CLAUDE.md index 4663ddb..c727e07 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -69,7 +69,7 @@ PrivaPub/ ASP.NET Core Web API, net10.0 Infrastructure/ Http/ FederationHttp + SafeHttpHandlerFactory + IpRangeGuard: the only way out Jobs/ JobQueue (leases), JobWorker, Backoff, HostCircuitBreaker - Ids/ PrivacyIds (day-only ids for personas and groups, published-time ids for remote posts) + Ids/ PrivacyIds (day-only ids for personas and groups, arrival-ordered ids for remote posts) Data/ Indexes (created at start), EntityMaps.Warm, Migrations/_NNN_*.cs Cli/ AdminCommands (`PrivaPub admin promote|demote `) RateLimiting.cs accounts (per client address) and inbox (per sending origin) policies @@ -129,7 +129,8 @@ cd PrivaPub && ASPNETCORE_ENVIRONMENT=Development \ Development config (`appsettings.Development.json`) binds HTTPS 7195 with HTTP/2 only; the overrides above make it curl-able. Outbound fetches only go to https DNS names resolving to public addresses; a test network (Pasture) sets -`Federation__AllowPrivateNetworks=true` and `Federation__AllowPlainHttp=true`, which startup refuses in Production. +`Federation__AllowPrivateNetworks=true`, `Federation__AllowPlainHttp=true` and `Federation__AcceptAnyCertificate=true`, +which startup refuses in Production. Promoting an admin on the box (signing up as "admin" grants nothing): @@ -226,6 +227,15 @@ cd /var/www/privapub.thepra.dev && sudo -u www-data ASPNETCORE_ENVIRONMENT=Produ - **Collection name = class name, so never rename an entity class.** - `Entity.ID` is a 24-character lowercase hex string; `GenerateNewID()` returns `object`, so cast it. - New fields must be additive: a deploy rollback restores the binary, not the database. +- **Post ids are the timeline order, so they follow arrival, not `published`.** `PrivacyIds.Arrived` gives a remote post + published within the last hour (or in the future) a fresh `ObjectId`, which sorts after every post already stored, + and only backfill keeps a `published`-derived id. With `published` ids a reply arriving in the same second could sort + under the post it answers, and a late arrival landed behind a client's `since_id` and was never seen. `created_at` + comes from `CreationDate`, never from the id. +- **The same ordering problem exists on the other side, so `published` carries milliseconds** + (`ActivityPubRenderer.Timestamp`). GoToSocial (ULIDs) and Mastodon (Snowflakes) derive a remote status's id from + `published` at millisecond resolution. With whole seconds, two of our posts from the same second sorted at random + there. - **Startup order:** `EntityMaps.Warm()` (every entity's class map, one at a time; two mapped at once throw "An item with the same key has already been added" and stay broken), then `MigrateAsync` (`Infrastructure/Data/Migrations`, `_NNN_` order, each runs once), then `Indexes.Create()`. A new entity needs nothing; a new unique index needs a @@ -243,6 +253,9 @@ cd /var/www/privapub.thepra.dev && sudo -u www-data ASPNETCORE_ENVIRONMENT=Produ - Localised strings go through `IStringLocalizer`. - Every `Display`/`ErrorMessage` resource key must exist in `FieldsNameResource`/`ErrorsResource`, including the `Designer.cs`, which the CLI build doesn't regenerate. A missing key throws at validation time. +- **`cond ? value : default` with a value-type branch is the type's default, not null:** `false`, `0`, or year one + stored as an edit date on every remote post. Write `(T?)null`. It has shipped four times (`MastodonParams.Bool/Int`, + `NoteParser.Int`, `NoteParser.Time`). - ActivityPub output is built with `System.Text.Json.Nodes` in `ActivityPubRenderer`, not typed models. Inbound documents are read through `InboxService.Id`/`Value`, which handle string, object and array. - Libraries chosen for the roadmap: HtmlSanitizer, Markdig (`DisableHtml`), NSign (RFC 9421 inbound), OpenIddict + @@ -261,8 +274,39 @@ Beyond the tests, verify by building, running locally, and exercising: - the client API (sign up, create an avatar, a group, a post); - the ActivityPub endpoints with curl and `Accept: application/activity+json`. -Interop is checked against real servers: Fediverse Pasture with podman on the workstation, verify.funfedi.dev, and -the owner's GoToSocial at social.arasaka.software. **Ask before acting from the owner's GoToSocial account.** +Interop is checked against real servers, starting with the workstation's own pasture: + +```bash +DOTNET=~/.dotnet/dotnet tools/pasture/run.sh up # podman: PrivaPub + the latest GoToSocial + Mongo, behind Caddy +tools/pasture/interop.sh # 25 checks, each side driven through its own Mastodon API +tools/pasture/run.sh down +``` + +- **Two sites on one podman network, one Caddy in front.** `privapub.test` and `gts.test` are network aliases of the + Caddy container, which serves both with its internal CA (`tls internal`). Both servers are told to accept any + certificate: PrivaPub through `appsettings.Pasture.json`, GoToSocial through `GTS_HTTP_CLIENT_TLS_INSECURE_SKIP_VERIFY`. + GoToSocial WebFingers and fetches over https only, so plain http between them is not an option. +- From the workstation, PrivaPub's API is `http://127.0.0.1:6971`. GoToSocial is reached as `https://gts.test:6443` + with `curl -k --resolve gts.test:6443:127.0.0.1`, because its sign-in cookie is bound to the host name. +- **GoToSocial's cached home timeline can stop taking new posts after its first read**, its owner's own included, while + a `min_id` query shows them all. So a delivery is checked by looking the object up by URI with `resolve=false` + (`on_gts`), which answers from GoToSocial's database and never fetches from us. A home-timeline check there proves + nothing, in either direction. A deleted status still turns up in that search as a "deleted status" stub, so a delete + is checked as a 404 on `/api/v1/statuses/{id}`. +- **GoToSocial creates its accounts locked**, so `interop.sh` approves alice's request through + `/api/v1/follow_requests`. That also checks our pending (`requested`) state and the manual Accept. +- The scenario covers: + - discovery and follows both ways; + - posts and CW; + - a reply and its notification; + - likes and boosts both ways; + - DMs both ways, and the DM staying off public timelines; + - edit, delete both ways, and unfollow. +- `run.sh up` replaces every container, Mongo included, so each run starts clean. To keep the data, republish into + `tools/pasture/.publish` and `podman restart pasture-privapub`; that is how a migration is tried on dirty data. + +After the pasture, verify.funfedi.dev and the owner's GoToSocial at social.arasaka.software. **Ask before acting from +the owner's GoToSocial account.** ## Deploy diff --git a/FEDERATION.md b/FEDERATION.md index 54bb560..5cf0da8 100644 --- a/FEDERATION.md +++ b/FEDERATION.md @@ -10,6 +10,20 @@ PrivaPub is an ActivityPub server written in C#. This document follows - [HTTP Signatures](https://datatracker.ietf.org/doc/html/draft-cavage-http-signatures), `rsa-sha256` / `hs2019` with RSA keys - [NodeInfo](https://nodeinfo.diaspora.software/) 2.0 and 2.1 +## Tested against + +- **GoToSocial 0.22.1, end to end.** It runs in a private network on the workstation (`tools/pasture/`) and is driven + through its own client API. Checked both ways: + - follows, including to a locked account; + - public posts with a content warning; + - replies with notifications; + - likes and boosts; + - direct messages; + - edits and deletes; + - unfollow. +- **Mastodon, Misskey, Lemmy and PeerTube, by unit tests only.** The tests feed the parser documents written in each + server's shape. No live exchange with any of them has run yet. + ## Supported FEPs - [FEP-67ff: FEDERATION.md](https://codeberg.org/fediverse/fep/src/branch/main/fep/67ff/fep-67ff.md) diff --git a/PrivaPub/Federation/Actors/RemoteActorService.cs b/PrivaPub/Federation/Actors/RemoteActorService.cs index 99f71f7..3be98dc 100644 --- a/PrivaPub/Federation/Actors/RemoteActorService.cs +++ b/PrivaPub/Federation/Actors/RemoteActorService.cs @@ -6,6 +6,7 @@ using PrivaPub.StaticServices; using System.Text.Json; using Microsoft.Extensions.Caching.Memory; +using Microsoft.Extensions.Options; using PrivaPub.Federation.Objects; using PrivaPub.Federation.Signing; @@ -32,13 +33,16 @@ namespace PrivaPub.Federation.Actors readonly ILocalActorService _localActors; readonly IMemoryCache _cache; readonly DbEntities _dbEntities; + readonly IOptionsMonitor _options; - public RemoteActorService(IFederationHttp http, ILocalActorService localActors, IMemoryCache cache, DbEntities dbEntities) + public RemoteActorService(IFederationHttp http, ILocalActorService localActors, IMemoryCache cache, DbEntities dbEntities, + IOptionsMonitor options = default) { _http = http; _localActors = localActors; _cache = cache; _dbEntities = dbEntities; + _options = options; } public async Task FetchObject(string uri, CancellationToken token) @@ -122,8 +126,11 @@ namespace PrivaPub.Federation.Actors if (parts is not { Length: 2 } || string.IsNullOrEmpty(parts[0]) || string.IsNullOrEmpty(parts[1])) return default; - var url = $"https://{parts[1]}/.well-known/webfinger?resource={Uri.EscapeDataString($"acct:{parts[0]}@{parts[1]}")}"; - using var fetched = await _http.GetJson(url, "application/jrd+json, application/json", sign: default, token); + var query = $"/.well-known/webfinger?resource={Uri.EscapeDataString($"acct:{parts[0]}@{parts[1]}")}"; + using var fetched = await _http.GetJson($"https://{parts[1]}{query}", "application/jrd+json, application/json", sign: default, token) + ?? (_options?.CurrentValue.AllowPlainHttp == true + ? await _http.GetJson($"http://{parts[1]}{query}", "application/jrd+json, application/json", sign: default, token) + : default); if (fetched == default) return default; var document = fetched.Document; diff --git a/PrivaPub/Infrastructure/Http/FederationOptions.cs b/PrivaPub/Infrastructure/Http/FederationOptions.cs index cb68e48..96996d3 100644 --- a/PrivaPub/Infrastructure/Http/FederationOptions.cs +++ b/PrivaPub/Infrastructure/Http/FederationOptions.cs @@ -5,5 +5,6 @@ namespace PrivaPub.Infrastructure.Http public bool AllowPrivateNetworks { get; set; } public bool AllowPlainHttp { get; set; } public bool SecureMode { get; set; } + public bool AcceptAnyCertificate { get; set; } } } diff --git a/PrivaPub/Infrastructure/Http/SafeHttpHandlerFactory.cs b/PrivaPub/Infrastructure/Http/SafeHttpHandlerFactory.cs index df38b7e..2f20b92 100644 --- a/PrivaPub/Infrastructure/Http/SafeHttpHandlerFactory.cs +++ b/PrivaPub/Infrastructure/Http/SafeHttpHandlerFactory.cs @@ -1,4 +1,5 @@ using System.Net; +using System.Net.Security; using System.Net.Sockets; namespace PrivaPub.Infrastructure.Http @@ -12,6 +13,9 @@ namespace PrivaPub.Infrastructure.Http { public static SocketsHttpHandler Create(FederationOptions options) => new() { + SslOptions = options.AcceptAnyCertificate + ? new SslClientAuthenticationOptions { RemoteCertificateValidationCallback = (_, _, _, _) => true } + : new SslClientAuthenticationOptions(), AllowAutoRedirect = false, UseProxy = false, UseCookies = false, diff --git a/PrivaPub/Program.cs b/PrivaPub/Program.cs index 57d79cb..e635b62 100644 --- a/PrivaPub/Program.cs +++ b/PrivaPub/Program.cs @@ -21,6 +21,8 @@ using PrivaPub.Models; using PrivaPub.Services; using PrivaPub.StaticServices; +Log.Logger = new LoggerConfiguration().WriteTo.Console().CreateBootstrapLogger(); + try { var builder = WebApplication.CreateBuilder(args); @@ -71,8 +73,8 @@ try } var federationOptions = builder.Configuration.GetSection("Federation").Get() ?? new(); - if (builder.Environment.IsProduction() && (federationOptions.AllowPrivateNetworks || federationOptions.AllowPlainHttp)) - throw new InvalidOperationException("Federation:AllowPrivateNetworks and Federation:AllowPlainHttp are for test networks and must stay off in Production."); + if (builder.Environment.IsProduction() && (federationOptions.AllowPrivateNetworks || federationOptions.AllowPlainHttp || federationOptions.AcceptAnyCertificate)) + throw new InvalidOperationException("Federation:AllowPrivateNetworks, AllowPlainHttp and AcceptAnyCertificate are for test networks and must stay off in Production."); try { @@ -189,4 +191,8 @@ catch (Exception ex) Log.ForContext().Fatal(ex, $"{nameof(Program)}.{nameof(Program)}()"); Environment.ExitCode = 1; } +finally +{ + await Log.CloseAndFlushAsync(); +} diff --git a/docs/ROADMAP.md b/docs/ROADMAP.md index 539e754..8e0e4da 100644 --- a/docs/ROADMAP.md +++ b/docs/ROADMAP.md @@ -9,7 +9,7 @@ Written 2026-10-01 from the original 2023 code, the decePubClient UI, a federati - [x] P1.2 Social graph and timelines: v1.3.0, deployed and verified 2026-10-01 - [x] P2 Mastodon client API: v1.4.0, deployed 2026-10-01; OAuth and the anonymous API verified on production, the signed-in API verified locally (no real-client login on production yet) - [x] P3 Social features: v1.5.0, deployed 2026-10-01; media, proxy, blocks, mutes, bookmarks, pins and reports verified by tests and locally (no upload on production yet) -- [ ] P4 Groups and privacy features +- [x] P4 Groups and privacy features: v1.6.0, deployed 2026-10-01; communities, circles and local-only located posts verified by tests. v1.6.1 adds the pasture (`tools/pasture/`): live interop with GoToSocial 0.22.1 passes all 25 checks, three runs in a row. Lemmy and a live Mastodon circle member are not run yet; the pasture has GoToSocial only - [ ] P5 FEPs and polish ## Intent @@ -417,8 +417,10 @@ The first refactor commit is a pure move with namespaces only. Logic changes fol `Fixtures/{mastodon,gotosocial,misskey,akkoma,lemmy}/`. Renderers are checked against golden JSON. Inbound scenarios run as integration tests on a throwaway mongod. - **Interop:** **Fediverse Pasture** runs on the workstation with podman: Mastodon, GoToSocial, Misskey/Sharkey, - Akkoma, plus Lemmy for P4. `tools/pasture/docker-compose.privapub.yml` runs PrivaPub with `AllowPrivateNetworks`; - startup asserts that flag is false in Production. The matrix, per peer: + Akkoma, plus Lemmy for P4. As built, `tools/pasture/run.sh` runs PrivaPub and GoToSocial behind one Caddy on a + podman network, with `AllowPrivateNetworks`, `AllowPlainHttp` and `AcceptAnyCertificate`; startup refuses all three in + Production. `tools/pasture/interop.sh` drives the matrix for GoToSocial; the other peers are still to be added. The + matrix, per peer: - follow both ways; - public, unlisted, followers-only and direct posts both ways; - a reply landing in the remote thread; diff --git a/tools/pasture/Caddyfile b/tools/pasture/Caddyfile new file mode 100644 index 0000000..b6a2487 --- /dev/null +++ b/tools/pasture/Caddyfile @@ -0,0 +1,14 @@ +{ + local_certs + skip_install_trust +} + +privapub.test { + tls internal + reverse_proxy pasture-privapub:80 +} + +gts.test { + tls internal + reverse_proxy pasture-gts:80 +} diff --git a/tools/pasture/appsettings.Pasture.json b/tools/pasture/appsettings.Pasture.json new file mode 100644 index 0000000..b9358eb --- /dev/null +++ b/tools/pasture/appsettings.Pasture.json @@ -0,0 +1,33 @@ +{ + "MongoSettings": { + "Database": "PrivaPub", + "LogsDatabase": "logs", + "ConnectionString": "mongodb://mongo:27017" + }, + "AppConfiguration": { + "Version": "0.0.0", + "MaxAllowedUploadFiles": 3, + "MaxAllowedFileSize": 2097152, + "SupportedLanguages": [ "en" ], + "BackendBaseAddress": "https://privapub.test", + "FrontendBaseAddress": "https://privapub.test", + "HashingOptions": { "Iterations": 10101 }, + "Jwt": { + "Key": "pasture-only-key-not-a-secret-pasture-only-key-not-a-secret-0123456789", + "Issuer": "http://privapub.test", + "Audience": "http://privapub.test", + "HoursTimeout": 24 + } + }, + "Federation": { + "AllowPrivateNetworks": true, + "AllowPlainHttp": true, + "AcceptAnyCertificate": true + }, + "Media": { "Root": "/tmp/privapub-media" }, + "Kestrel": { "Endpoints": { "Http": { "Url": "http://0.0.0.0:80", "Protocols": "Http1AndHttp2" } } }, + "Serilog": { + "MinimumLevel": { "Default": "Information", "Override": { "Microsoft": "Warning", "System": "Warning" } }, + "WriteTo": [ { "Name": "Console" } ] + } +} diff --git a/tools/pasture/interop.sh b/tools/pasture/interop.sh new file mode 100755 index 0000000..6c236aa --- /dev/null +++ b/tools/pasture/interop.sh @@ -0,0 +1,117 @@ +#!/usr/bin/env bash +# Drives PrivaPub and GoToSocial (started by run.sh) through each other's federation, as their users would. +set -uo pipefail +P=http://127.0.0.1:6971; G=https://gts.test:6443 +gcurl() { curl -k --resolve gts.test:6443:127.0.0.1 "$@"; } +work=$(mktemp -d); trap 'rm -rf "$work"' EXIT +pass=0; fail=0 +ok() { echo " ok $*"; pass=$((pass+1)); } +ko() { echo " FAIL $*"; fail=$((fail+1)); } +j() { python3 -c "import sys,json +try: d=json.load(sys.stdin) +except Exception: d=None +$1" 2>/dev/null; } +until_true() { local tries=$1; shift; for i in $(seq 1 "$tries"); do if eval "$@"; then return 0; fi; sleep 2; done; return 1; } +# GoToSocial's cached home timeline can stop taking new posts after its first read, so a delivery is checked by looking +# the object up by URI with resolve=false: that answers from GoToSocial's database and never fetches from us. +on_gts() { gcurl -s -G -H "$GH" "$G/api/v2/search" --data-urlencode "q=$1" -d resolve=false -d type=statuses; } + +# --- PrivaPub persona and token +root=$(curl -s -X POST $P/clientapi/user/signup -H 'Content-Type: application/json' -d '{"userName":"pastureroot","password":"Pasture-Pass-1!"}') +jwt=$(echo "$root" | j "print(d['token'])"); rootid=$(echo "$root" | j "print(d['userId'])") +if [ -z "$jwt" ]; then jwt=$(curl -s -X POST $P/clientapi/user/login -H 'Content-Type: application/json' -d '{"userName":"pastureroot","password":"Pasture-Pass-1!"}' | j "print(d['token'])"); rootid=$(curl -s -X POST $P/clientapi/user/login -H 'Content-Type: application/json' -d '{"userName":"pastureroot","password":"Pasture-Pass-1!"}' | j "print(d['userId'])"); fi +curl -s -o /dev/null -X POST $P/clientapi/avatar/private/insert -H 'Content-Type: application/json' -H "Authorization: Bearer $jwt" \ + -d "{\"rootId\":\"$rootid\",\"userName\":\"alice\",\"name\":\"Alice of PrivaPub\",\"biography\":\"testing federation\"}" +app=$(curl -s -X POST $P/api/v1/apps -d 'client_name=pasture&redirect_uris=urn:ietf:wg:oauth:2.0:oob&scopes=read+write+follow') +cid=$(echo "$app" | j "print(d['client_id'])"); cs=$(echo "$app" | j "print(d['client_secret'])") +q="client_id=$cid&redirect_uri=urn:ietf:wg:oauth:2.0:oob&response_type=code&scope=read+write+follow" +xt=$(curl -s -c $work/pj -b $work/pj "$P/oauth/login?returnUrl=/oauth/authorize?$q" | grep -o 'name="__RequestVerificationToken" type="hidden" value="[^"]*"' | sed 's/.*value="//;s/"//') +curl -s -o /dev/null -c $work/pj -b $work/pj -X POST $P/oauth/login --data-urlencode "returnUrl=/oauth/authorize?$q" --data-urlencode "__RequestVerificationToken=$xt" -d 'userName=pastureroot&password=Pasture-Pass-1!' +curl -s -c $work/pj -b $work/pj "$P/oauth/authorize?$q&signed_in=1" > $work/choose.html +form=$(python3 - "$work/choose.html" <<'PY' +import re,sys,urllib.parse,html +s=open(sys.argv[1]).read() +pairs=[(k,html.unescape(v)) for k,v in re.findall(r'[^<]*' | sed 's/<[^>]*>//g') +PT=$(curl -s -X POST $P/oauth/token -d "grant_type=authorization_code&code=$code&client_id=$cid&client_secret=$cs&redirect_uri=urn:ietf:wg:oauth:2.0:oob" | j "print(d['access_token'])") +PH="Authorization: Bearer $PT" +[ -n "$PT" ] && ok "PrivaPub token for alice" || { ko "PrivaPub token"; exit 1; } + +# --- GoToSocial token +gapp=$(gcurl -s -X POST $G/api/v1/apps -d 'client_name=pasture&redirect_uris=urn:ietf:wg:oauth:2.0:oob&scopes=read+write+follow') +gid=$(echo "$gapp" | j "print(d['client_id'])"); gs=$(echo "$gapp" | j "print(d['client_secret'])") +gcurl -s -o /dev/null -c $work/gj -b $work/gj "$G/oauth/authorize?client_id=$gid&redirect_uri=urn:ietf:wg:oauth:2.0:oob&response_type=code&scope=read+write+follow" +gcurl -s -o /dev/null -c $work/gj -b $work/gj -X POST $G/auth/sign_in --data-urlencode 'username=gtsuser@gts.test' --data-urlencode 'password=Gts-Pasture-Pass-1!' +gcode=$(gcurl -s -o /dev/null -w '%{redirect_url}' -c $work/gj -b $work/gj -X POST "$G/oauth/authorize" | sed -n 's/.*[?&]code=\([^&]*\).*/\1/p') +GT=$(gcurl -s -X POST $G/oauth/token -d "grant_type=authorization_code&code=$gcode&client_id=$gid&client_secret=$gs&redirect_uri=urn:ietf:wg:oauth:2.0:oob&scope=read+write+follow" | j "print(d['access_token'])") +GH="Authorization: Bearer $GT" +[ -n "$GT" ] && ok "GoToSocial token for gtsuser" || { ko "GoToSocial token (code '$gcode')"; exit 1; } + +echo "discovery" +alice_on_gts=$(gcurl -s -H "$GH" "$G/api/v2/search?q=@alice@privapub.test&resolve=true&type=accounts" | j "print(d['accounts'][0]['id'])") +[ -n "$alice_on_gts" ] && ok "GoToSocial resolves @alice@privapub.test" || ko "GoToSocial cannot resolve alice" +gts_on_pp=$(curl -s -H "$PH" "$P/api/v2/search?q=gtsuser@gts.test&resolve=true&type=accounts" | j "print(d['accounts'][0]['id'])") +[ -n "$gts_on_pp" ] && ok "PrivaPub resolves @gtsuser@gts.test" || ko "PrivaPub cannot resolve gtsuser" + +echo "follows" +gcurl -s -o /dev/null -X POST -H "$GH" $G/api/v1/accounts/$alice_on_gts/follow +until_true 20 '[ "$(gcurl -s -H "$GH" "$G/api/v1/accounts/relationships?id[]=$alice_on_gts" | j "print(d[0][\"following\"])")" = "True" ]' && ok "gtsuser follows alice (Accept arrived)" || ko "gtsuser's follow of alice not accepted" +curl -s -o /dev/null -X POST -H "$PH" $P/api/v1/accounts/$gts_on_pp/follow +until_true 10 '[ "$(curl -s -H "$PH" "$P/api/v1/accounts/relationships?id[]=$gts_on_pp" | j "print(d[0][\"requested\"])")" = "True" ]' && ok "alice's follow of locked gtsuser waits as a request" || ko "alice's follow is not shown as requested" +until_true 20 'gcurl -s -H "$GH" "$G/api/v1/follow_requests" | j "print(any(a[\"id\"]==\"$alice_on_gts\" for a in d))" | grep -q True' && ok "the request reaches gtsuser's follow requests" || ko "follow request missing on GoToSocial" +gcurl -s -o /dev/null -X POST -H "$GH" $G/api/v1/follow_requests/$alice_on_gts/authorize +until_true 20 '[ "$(curl -s -H "$PH" "$P/api/v1/accounts/relationships?id[]=$gts_on_pp" | j "print(d[0][\"following\"])")" = "True" ]' && ok "alice follows gtsuser (Accept arrived)" || ko "alice's follow of gtsuser not accepted" + +echo "posts" +pp_status=$(curl -s -X POST -H "$PH" $P/api/v1/statuses -d 'status=Hello from PrivaPub #pasture&visibility=public') +pp_post=$(echo "$pp_status" | j "print(d['id'])"); pp_uri=$(echo "$pp_status" | j "print(d['uri'])") +until_true 20 'on_gts "$pp_uri" | j "print(len(d[\"statuses\"]))" | grep -q 1' && ok "alice's post reaches GoToSocial" || ko "alice's post missing on GoToSocial" +gts_post=$(gcurl -s -X POST -H "$GH" $G/api/v1/statuses -d 'status=Hello from GoToSocial&visibility=public' | j "print(d['id'])") +until_true 20 'curl -s -H "$PH" "$P/api/v1/timelines/home" | grep -q "Hello from GoToSocial"' && ok "gtsuser's post reaches alice's home" || ko "gtsuser's post missing on PrivaPub" +[ "$(curl -s -H "$PH" "$P/api/v1/timelines/home" | j "print(next(s['edited_at'] for s in d if 'Hello from GoToSocial' in s['content']))")" = "None" ] && ok "an unedited remote post carries no edited_at" || ko "unedited remote post reports an edit" +cw_status=$(curl -s -X POST -H "$PH" $P/api/v1/statuses -d 'status=behind a warning&spoiler_text=spoilers&visibility=public') +cw=$(echo "$cw_status" | j "print(d['id'])"); cw_uri=$(echo "$cw_status" | j "print(d['uri'])") +until_true 20 'on_gts "$cw_uri" | j "print(any(s[\"spoiler_text\"]==\"spoilers\" and s[\"sensitive\"] for s in d[\"statuses\"]))" | grep -q True' && ok "content warning survives to GoToSocial" || ko "content warning lost" +cw_on_gts=$(on_gts "$cw_uri" | j "print(d['statuses'][0]['id'])") + +echo "replies and mentions" +pp_on_gts=$(on_gts "$pp_uri" | j "print(d['statuses'][0]['id'])") +gcurl -s -o /dev/null -X POST -H "$GH" $G/api/v1/statuses -d "status=@alice@privapub.test nice to meet you&in_reply_to_id=$pp_on_gts&visibility=public" +until_true 20 'curl -s -H "$PH" "$P/api/v1/notifications" | j "print(any(n[\"type\"]==\"mention\" for n in d))" | grep -q True' && ok "gtsuser's reply notifies alice" || ko "reply did not notify alice" +until_true 10 '[ "$(curl -s -H "$PH" "$P/api/v1/statuses/$pp_post/context" | j "print(len(d[\"descendants\"]))")" -ge 1 ]' && ok "the reply threads under alice's post" || ko "reply not threaded" + +echo "likes and boosts" +gts_on_pp_post=$(curl -s -H "$PH" "$P/api/v1/timelines/home" | j "print(next(s['id'] for s in d if 'Hello from GoToSocial' in s['content']))") +curl -s -o /dev/null -X POST -H "$PH" $P/api/v1/statuses/$gts_on_pp_post/favourite +curl -s -o /dev/null -X POST -H "$PH" $P/api/v1/statuses/$gts_on_pp_post/reblog +until_true 20 'gcurl -s -H "$GH" "$G/api/v1/notifications" | j "print(any(n[\"type\"]==\"favourite\" for n in d))" | grep -q True' && ok "alice's like reaches GoToSocial" || ko "like not received" +until_true 20 'gcurl -s -H "$GH" "$G/api/v1/notifications" | j "print(any(n[\"type\"]==\"reblog\" for n in d))" | grep -q True' && ok "alice's boost reaches GoToSocial" || ko "boost not received" +gcurl -s -o /dev/null -X POST -H "$GH" $G/api/v1/statuses/$pp_on_gts/favourite +gcurl -s -o /dev/null -X POST -H "$GH" $G/api/v1/statuses/$pp_on_gts/reblog +until_true 20 '[ "$(curl -s -H "$PH" "$P/api/v1/statuses/$pp_post" | j "print(d[\"favourites_count\"])")" = "1" ]' && ok "gtsuser's like counts on PrivaPub" || ko "like not counted" +until_true 20 'curl -s -H "$PH" "$P/api/v1/notifications" | j "print(any(n[\"type\"]==\"reblog\" for n in d))" | grep -q True' && ok "gtsuser's boost notifies alice" || ko "boost not notified" + +echo "direct messages" +curl -s -o /dev/null -X POST -H "$PH" $P/api/v1/statuses -d 'status=@gtsuser@gts.test a secret&visibility=direct' +until_true 20 'gcurl -s -H "$GH" "$G/api/v1/conversations" | grep -q "a secret"' && ok "alice's DM reaches gtsuser" || ko "DM missing on GoToSocial" +gcurl -s -o /dev/null -X POST -H "$GH" $G/api/v1/statuses -d 'status=@alice@privapub.test a secret back&visibility=direct' +until_true 20 'curl -s -H "$PH" "$P/api/v1/conversations" | grep -q "a secret back"' && ok "gtsuser's DM reaches alice" || ko "DM missing on PrivaPub" +until_true 5 '! gcurl -s -H "$GH" "$G/api/v1/timelines/public?local=false" | grep -q "a secret"' && ok "the DM is not public on GoToSocial" || ko "DM leaked to a public timeline" + +echo "edits and deletes" +curl -s -o /dev/null -X PUT -H "$PH" $P/api/v1/statuses/$pp_post -d 'status=Hello from PrivaPub, edited' +until_true 20 'gcurl -s -H "$GH" "$G/api/v1/statuses/$pp_on_gts" | grep -q "edited"' && ok "alice's edit reaches GoToSocial" || ko "edit not applied" +curl -s -o /dev/null -X DELETE -H "$PH" $P/api/v1/statuses/$cw +[ -n "$cw_on_gts" ] && until_true 20 '[ "$(gcurl -s -o /dev/null -w "%{http_code}" -H "$GH" "$G/api/v1/statuses/$cw_on_gts")" = "404" ]' && ok "alice's delete reaches GoToSocial" || ko "delete not applied" +gcurl -s -o /dev/null -X DELETE -H "$GH" $G/api/v1/statuses/$gts_post +until_true 20 '! curl -s -H "$PH" "$P/api/v1/timelines/home" | grep -q "Hello from GoToSocial"' && ok "gtsuser's delete reaches PrivaPub" || ko "remote delete not applied" + +echo "unfollow" +curl -s -o /dev/null -X POST -H "$PH" $P/api/v1/accounts/$gts_on_pp/unfollow +until_true 20 '[ "$(gcurl -s -H "$GH" "$G/api/v1/accounts/relationships?id[]=$alice_on_gts" | j "print(d[0][\"followed_by\"])")" = "False" ]' && ok "alice's unfollow reaches GoToSocial" || ko "unfollow not applied" + +echo; echo "$pass passed, $fail failed" +[ "$fail" = 0 ] diff --git a/tools/pasture/run.sh b/tools/pasture/run.sh new file mode 100755 index 0000000..668ae11 --- /dev/null +++ b/tools/pasture/run.sh @@ -0,0 +1,44 @@ +#!/usr/bin/env bash +# A private test fediverse on one podman network: PrivaPub (privapub.test) and GoToSocial (gts.test) behind Caddy, +# whose internal CA both sides are told to accept. From the workstation: PrivaPub's API at http://127.0.0.1:6971, +# both sites at https://{privapub,gts}.test:6443 (curl --resolve ...:6443:127.0.0.1 -k). +# usage: tools/pasture/run.sh up | down | logs +set -euo pipefail +here="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"; repo="$(cd "$here/../.." && pwd)" +net=privapub-pasture; publish="$here/.publish" + +case "${1:-up}" in +up) + dotnet=${DOTNET:-$(command -v dotnet || echo ~/.dotnet/dotnet)} + "$dotnet" publish "$repo/PrivaPub/PrivaPub.csproj" -c Release -r linux-x64 --self-contained true -o "$publish" -v quiet + cp "$here/appsettings.Pasture.json" "$publish/" + podman network exists $net || podman network create $net >/dev/null + podman run -d --replace --name pasture-mongo --network $net --network-alias mongo docker.io/library/mongo:8 --quiet >/dev/null + podman run -d --replace --name pasture-caddy --network $net --network-alias privapub.test --network-alias gts.test \ + -p 127.0.0.1:6443:443 --sysctl net.ipv4.ip_unprivileged_port_start=0 -v "$here/Caddyfile:/etc/caddy/Caddyfile:Z,ro" \ + docker.io/library/caddy:2 >/dev/null + podman run -d --replace --name pasture-privapub --network $net -p 127.0.0.1:6971:80 \ + --sysctl net.ipv4.ip_unprivileged_port_start=0 -e ASPNETCORE_ENVIRONMENT=Pasture -w /app -v "$publish:/app:Z,ro" \ + mcr.microsoft.com/dotnet/runtime-deps:10.0 /app/PrivaPub >/dev/null + podman run -d --replace --name pasture-gts --network $net -p 127.0.0.1:6972:80 \ + --sysctl net.ipv4.ip_unprivileged_port_start=0 \ + -e GTS_HOST=gts.test -e GTS_PROTOCOL=https -e GTS_PORT=80 -e GTS_BIND_ADDRESS=0.0.0.0 -e GTS_HTTP_CLIENT_TLS_INSECURE_SKIP_VERIFY=true \ + -e GTS_DB_TYPE=sqlite -e GTS_DB_ADDRESS=/gotosocial/storage/sqlite.db -e GTS_STORAGE_LOCAL_BASE_PATH=/gotosocial/storage \ + -e GTS_LETSENCRYPT_ENABLED=false -e GTS_HTTP_CLIENT_ALLOW_IPS=10.0.0.0/8,172.16.0.0/12,192.168.0.0/16 \ + -e GTS_TRUSTED_PROXIES=0.0.0.0/0 -e GTS_LOG_LEVEL=info -e GTS_INSTANCE_EXPOSE_PUBLIC_TIMELINE=true \ + docker.io/superseriousbusiness/gotosocial:latest >/dev/null + for i in $(seq 1 60); do curl -fs -o /dev/null http://127.0.0.1:6971/build.json && curl -fs -o /dev/null http://127.0.0.1:6972/api/v1/instance && break; sleep 2; done + podman exec pasture-gts /gotosocial/gotosocial admin account create --username gtsuser --email gtsuser@gts.test --password 'Gts-Pasture-Pass-1!' >/dev/null 2>&1 || true + podman exec pasture-gts /gotosocial/gotosocial admin account confirm --username gtsuser >/dev/null 2>&1 || true + podman restart pasture-gts >/dev/null + for i in $(seq 1 60); do curl -fs -o /dev/null http://127.0.0.1:6972/api/v1/instance && break; sleep 2; done + echo "privapub: http://127.0.0.1:6971, https://privapub.test:6443 gotosocial: https://gts.test:6443" + ;; +down) + podman rm -f pasture-caddy pasture-privapub pasture-gts pasture-mongo >/dev/null 2>&1 || true + podman network rm $net >/dev/null 2>&1 || true + ;; +logs) + podman logs --tail 200 "pasture-${2:-privapub}" + ;; +esac