T11: Mastodon 4.7.3 in the pasture

peers/mastodon.sh runs Mastodon's web and sidekiq containers on a shared Postgres and Redis
(peers/shared.sh). They trust Caddy's CA through SSL_CERT_FILE and reach private addresses
through ALLOWED_PRIVATE_ADDRESSES. Its users and tokens come from tootctl and rails runner.

scenarios/mastodon.sh adds 49 checks, as listed in docs/INTEROP.md: follows, posts,
replies, likes and boosts with undos, DMs, polls, FEP-044f quotes both ways, media through
the proxy, edits, deletes, locked follows, a circle request, reports, blocks and statistics.
Two are expected failures:
- inbound Block (P7);
- circle posts. Mastodon 4.7 loses the recipient of deliveries to its numeric
  /ap/users/<id>/inbox and then drops a post that names no local account. The fix on our
  side changes what a circle reveals, so it waits for the owner.

GoToSocial and Mastodon together: 86 passed, 0 failed.

The pasture now copies Caddy's root certificate reliably, readable by the peers, and
rebuilds the bundle each time. The CA directory is mounted shared (:z), because a private
:Z label locks out every container but the last. pfetch reaches PrivaPub's own https URIs
through Caddy. PRIVAPUB_ENV passes settings to PrivaPub, which scenarios/crawler.sh uses to
check the opt-in crawler against Mastodon: it visits, describes and reads the peers list.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ELjqpznMFMNrJoJUj6K5p2
This commit is contained in:
thepraandClaude Opus 5.5 committed 2026-10-03 12:30:05 +02:00
1 parent d0a13d9cc2
commit e0f2eb7da7
9 files changed
+367 -28

No files matched your search

+38 -24
View File
@@ -183,7 +183,9 @@ cd /var/www/privapub.thepra.dev && sudo -u www-data ASPNETCORE_ENVIRONMENT=Produ
10. **Nothing slow happens inside a request.** Deliveries and inbox processing are `Job`s (`Infrastructure/Jobs`): 10. **Nothing slow happens inside a request.** Deliveries and inbox processing are `Job`s (`Infrastructure/Jobs`):
leased, retried on Mastodon's curve, at most two per host, paused per host by `RemoteInstance`. The inbox answers leased, retried on Mastodon's curve, at most two per host, paused per host by `RemoteInstance`. The inbox answers
202 once it has verified and queued; a handler must be idempotent (unique `ObjectURI`, job `DedupeKey`). 202 once it has verified and queued; a handler must be idempotent (unique `ObjectURI`, job `DedupeKey`).
11. **Every "may anyone see this" goes through `VisibilityPolicy.IsPublic`;** a persona-specific read uses `CanSee`. 11. **Every "may anyone see this" goes through `VisibilityPolicy.IsPublic`;** a persona-specific read uses `CanSee`, and
any other read of stored posts filters by `IsShown`. All three hide deleted posts and the posts of a remote account
that deleted itself (`Post.AuthorGone`: kept, hidden everywhere, owner decision).
12. **Remote content is stored only when someone here asked for it:** a persona follows the author, is addressed or 12. **Remote content is stored only when someone here asked for it:** a persona follows the author, is addressed or
mentioned, it replies to a local post, or it is addressed to a community the author follows; a public parent is mentioned, it replies to a local post, or it is addressed to a community the author follows; a public parent is
fetched as context. Followers-only is detected by the author's stored `followers` URL. fetched as context. Followers-only is detected by the author's stored `followers` URL.
@@ -372,31 +374,43 @@ Beyond the tests, verify by building, running locally, and exercising:
Interop is checked against real servers, starting with the workstation's own pasture: Interop is checked against real servers, starting with the workstation's own pasture:
```bash ```bash
DOTNET=~/.dotnet/dotnet tools/pasture/run.sh up # podman: PrivaPub + the latest GoToSocial + Mongo, behind Caddy DOTNET=~/.dotnet/dotnet tools/pasture/run.sh up [gts mastodon ...] # podman: PrivaPub + the named peers (default gts) + Mongo, behind Caddy
tools/pasture/interop.sh # 33 checks, each side driven through its own Mastodon API tools/pasture/interop.sh [gts mastodon ...] # each peer's scenario, then what PrivaPub's statistics saw of it
tools/pasture/run.sh down tools/pasture/run.sh down # removes every pasture container and volume
``` ```
- **Two sites on one podman network, one Caddy in front.** `privapub.test` and `gts.test` are network aliases of the - **Layout:** `lib/pasture.sh` (network, Caddy, Mongo, PrivaPub), `peers/<name>.sh` (`<name>_up`, plus `peers/shared.sh`
Caddy container, which serves both with its internal CA (`tls internal`). Both servers are told to accept any for the Postgres and Redis several peers share), `lib/interop.sh` (`ok`, `ko`, `xf` for a check expected to fail until
certificate: PrivaPub through `appsettings.Pasture.json`, GoToSocial through `GTS_HTTP_CLIENT_TLS_INSECURE_SKIP_VERIFY`. a later phase, `privapub_token <persona>`, `stats_check <host> <software>`), `scenarios/<name>.sh`. Each peer talks to
GoToSocial WebFingers and fetches over https only, so plain http between them is not an option. its own PrivaPub persona under one root, so OAuth's persona choice is exercised too. Images are pinned.
- From the workstation, PrivaPub's API is `http://127.0.0.1:6971`. GoToSocial is reached as `https://gts.test:6443` - Caddy's CA lives in the `pasture-caddy-data` volume and is copied to `tools/pasture/.ca/root.crt` (and `bundle.pem`
with `curl -k --resolve gts.test:6443:127.0.0.1`, because its sign-in cookie is bound to the host name. with the system roots) for peers that must trust it instead of skipping verification.
- **GoToSocial's cached home timeline can stop taking new posts after its first read**, its owner's own included, while
a `min_id` query shows them all. So a delivery is checked by looking the object up by URI with `resolve=false` - **All sites on one podman network, one Caddy in front.** `privapub.test`, `gts.test`, `mastodon.test` and the other
(`on_gts`), which answers from GoToSocial's database and never fetches from us. A home-timeline check there proves peers are network aliases of the Caddy container, which serves them all with its internal CA (`tls internal`).
nothing, in either direction. A deleted status still turns up in that search as a "deleted status" stub, so a delete PrivaPub accepts any certificate (`appsettings.Pasture.json`) and GoToSocial is told to skip verification
is checked as a 404 on `/api/v1/statuses/{id}`. (`GTS_HTTP_CLIENT_TLS_INSECURE_SKIP_VERIFY`); Mastodon trusts the copied CA through `SSL_CERT_FILE`. Peers fetch over
- **GoToSocial creates its accounts locked**, so `interop.sh` approves alice's request through https only, so plain http between them is not an option.
`/api/v1/follow_requests`. That also checks our pending (`requested`) state and the manual Accept. - From the workstation, PrivaPub's API is `http://127.0.0.1:6971`. A peer is reached as `https://<name>.test:6443` with
- The scenario covers: `curl -k --resolve <name>.test:6443:127.0.0.1`, because sign-in cookies are bound to the host name.
- discovery and follows both ways; - **GoToSocial (0.22.1):**
- posts and CW; - Its cached home timeline can stop taking new posts after its first read, its owner's own included, while a
- a reply and its notification; `min_id` query shows them all. So a delivery is checked by looking the object up by URI with `resolve=false`
- likes and boosts both ways; (`on_gts`), which answers from GoToSocial's database and never fetches from us. A home-timeline check there proves
- DMs both ways, and the DM staying off public timelines; nothing, in either direction. A deleted status still turns up in that search as a "deleted status" stub, so a
- edit, delete both ways, and unfollow. delete is checked as a 404 on `/api/v1/statuses/{id}`.
- It creates its accounts locked, so the scenario approves alice's request through `/api/v1/follow_requests`, which
also checks our pending (`requested`) state and the manual Accept.
- 37 checks: discovery and follows both ways; posts and CW; a reply and its notification; likes and boosts both
ways; DMs both ways and off public timelines; polls both ways; quote policy; link cards; edits and deletes both
ways; unfollow; block and unblock; statistics.
- **Mastodon (4.7.3):** web and sidekiq on the shared Postgres and Redis, `ALLOWED_PRIVATE_ADDRESSES` for the network.
Its token comes from `rails runner` (no password grant). Without Elasticsearch its status search finds nothing, so
deliveries are checked through `/api/v1/accounts/:id/statuses` of the sender as Mastodon knows them, or
`Status.exists?` through `rails runner`. Its actors are numbered (`/ap/users/<id>`), so look URIs up rather than
build them. 49 checks; circle posts are an expected failure (see `docs/INTEROP.md`, Mastodon).
- **Crawler:** `PRIVAPUB_ENV="Statistics__Crawler__Enabled=true Statistics__Crawler__Seeds__0=mastodon.test"
run.sh up mastodon`, then `interop.sh crawler`. `PRIVAPUB_ENV` passes any setting to the PrivaPub container.
- `run.sh up` replaces every container, Mongo included, so each run starts clean. To keep the data, republish into - `run.sh up` replaces every container, Mongo included, so each run starts clean. To keep the data, republish into
`tools/pasture/.publish` and `podman restart pasture-privapub`; that is how a migration is tried on dirty data. `tools/pasture/.publish` and `podman restart pasture-privapub`; that is how a migration is tried on dirty data.
+34
View File
@@ -150,6 +150,36 @@ Priorities, used throughout:
| Publish `context` and a paged `replies` | P2 | — | | Publish `context` and a paged `replies` | P2 | — |
| `FeatureRequest`: send `Reject` (or implement FEP-7aa9) | P3 | — | | `FeatureRequest`: send `Reject` (or implement FEP-7aa9) | P3 | — |
**Pasture evidence (2026-10-03, Mastodon v4.7.3, `tools/pasture/scenarios/mastodon.sh`):** 49 checks pass. They cover:
- discovery, follows and locked follows both ways;
- public, CW and followers-only posts (the last answering 404 unsigned);
- replies threading both ways;
- likes, boosts and their undos both ways, with counts;
- DMs both ways;
- polls and votes both ways;
- FEP-044f quotes approved both ways;
- images with alt text both ways, ours through `/media/proxy`;
- edits with history both ways;
- deletes both ways, ours answering 410;
- a Flag reaching Mastodon's moderators from the instance actor;
- a circle request held for its owner and approved;
- unfollow and block;
- statistics naming `mastodon.test` as mastodon with no account names.
Findings:
- **Mastodon 4.7 names its actors by number**: `https://mastodon.test/ap/users/<id>`, inbox `…/ap/users/<id>/inbox`. Nothing here may assume
`/users/<name>`.
- **It drops circle posts** (expected failure in the scenario).
- A post addressed only to `[circle, circle/flock]` parses as `direct` there, and Mastodon keeps a `direct` post
only if it names a local account or arrived in a known account's inbox (`Create#addresses_local_accounts?`).
- But `ActivityPub::InboxesController#account_required?` looks only at `params[:account_username]`. A delivery to
the numeric `/ap/users/:account_id/inbox` it now advertises therefore reaches the worker with no recipient and
is rejected.
- DMs are unaffected because they name the recipient.
- Fixing it on our side means naming the member (or that server's members) in each copy's `cc`, which changes what a
circle reveals: **owner decision pending**. Reporting it upstream is the other half.
- Inbound `Block` from Mastodon is not enforced yet (P7).
### GoToSocial: 0.22.1 (2026-07-20) ### GoToSocial: 0.22.1 (2026-07-20)
**Emits** **Emits**
@@ -201,6 +231,10 @@ Priorities, used throughout:
| Measure media size when proxying | P2 | `MediaAttachment.meta` | | Measure media size when proxying | P2 | `MediaAttachment.meta` |
| Only advertise the policies we enforce | P2 | — | | Only advertise the policies we enforce | P2 | — |
**Pasture evidence (2026-10-03, GoToSocial 0.22.1, `tools/pasture/scenarios/gts.sh`):** 37 checks pass, three runs in a
row. That is the original 33 plus four on statistics: described as gotosocial, inbound and outbound traffic counted,
no account named.
### Misskey family: Misskey 2026.10.0, Sharkey 2025.4.7, Iceshrimp.NET 2026.1.2-beta, CherryPick 4.17 ### Misskey family: Misskey 2026.10.0, Sharkey 2025.4.7, Iceshrimp.NET 2026.1.2-beta, CherryPick 4.17
Firefish is dead (its site has answered 410 since February 2025). Firefish is dead (its site has answered 410 since February 2025).
+5
View File
@@ -12,3 +12,8 @@ gts.test {
tls internal tls internal
reverse_proxy pasture-gts:80 reverse_proxy pasture-gts:80
} }
mastodon.test {
tls internal
reverse_proxy pasture-mastodon:3000
}
+2
View File
@@ -11,6 +11,8 @@ except Exception: d=None
$1" 2>/dev/null; } $1" 2>/dev/null; }
until_true() { local tries=$1; shift; for _ in $(seq 1 "$tries"); do if eval "$@"; then return 0; fi; sleep 2; done; return 1; } until_true() { local tries=$1; shift; for _ in $(seq 1 "$tries"); do if eval "$@"; then return 0; fi; sleep 2; done; return 1; }
site() { curl -k --resolve "$1:6443:127.0.0.1" "${@:2}"; } site() { curl -k --resolve "$1:6443:127.0.0.1" "${@:2}"; }
# fetches one of PrivaPub's own https URIs (ids, scribbles) from the workstation, through Caddy
pfetch() { curl -sk --connect-to privapub.test:443:127.0.0.1:6443 "$@"; }
ROOT_USER=pastureroot; ROOT_PASS='Pasture-Pass-1!' ROOT_USER=pastureroot; ROOT_PASS='Pasture-Pass-1!'
privapub_root() { privapub_root() {
+15 -4
View File
@@ -20,17 +20,28 @@ pasture_base_up() {
podman run -d --replace --name pasture-caddy --network $net $aliases \ podman run -d --replace --name pasture-caddy --network $net $aliases \
-p 127.0.0.1:6443:443 --sysctl net.ipv4.ip_unprivileged_port_start=0 -v "$here/Caddyfile:/etc/caddy/Caddyfile:Z,ro" \ -p 127.0.0.1:6443:443 --sysctl net.ipv4.ip_unprivileged_port_start=0 -v "$here/Caddyfile:/etc/caddy/Caddyfile:Z,ro" \
-v pasture-caddy-data:/data docker.io/library/caddy:2 >/dev/null -v pasture-caddy-data:/data docker.io/library/caddy:2 >/dev/null
local extra=()
for setting in ${PRIVAPUB_ENV:-}; do extra+=(-e "$setting"); done
podman run -d --replace --name pasture-privapub --network $net -p 127.0.0.1:6971:80 \ podman run -d --replace --name pasture-privapub --network $net -p 127.0.0.1:6971:80 \
--sysctl net.ipv4.ip_unprivileged_port_start=0 -e ASPNETCORE_ENVIRONMENT=Pasture -w /app -v "$publish:/app:Z,ro" \ --sysctl net.ipv4.ip_unprivileged_port_start=0 -e ASPNETCORE_ENVIRONMENT=Pasture "${extra[@]}" -w /app -v "$publish:/app:Z,ro" \
mcr.microsoft.com/dotnet/runtime-deps:10.0 /app/PrivaPub >/dev/null mcr.microsoft.com/dotnet/runtime-deps:10.0 /app/PrivaPub >/dev/null
wait_http http://127.0.0.1:6971/build.json wait_http http://127.0.0.1:6971/build.json
mkdir -p "$ca" rm -rf "$ca"; mkdir -p "$ca"
for _ in $(seq 1 30); do podman cp pasture-caddy:/data/caddy/pki/authorities/local/root.crt "$ca/root.crt" 2>/dev/null && break; sleep 1; done for _ in $(seq 1 60); do
podman cp pasture-caddy:/data/caddy/pki/authorities/local/root.crt "$ca/root.crt" 2>/dev/null && [ -s "$ca/root.crt" ] && break
curl -sk -o /dev/null --resolve privapub.test:6443:127.0.0.1 https://privapub.test:6443/build.json || true
sleep 1
done
[ -s "$ca/root.crt" ] || { echo "Caddy's root certificate could not be copied" >&2; return 1; }
chmod 644 "$ca/root.crt"
cat /etc/pki/tls/certs/ca-bundle.crt /etc/ssl/certs/ca-certificates.crt 2>/dev/null > "$ca/bundle.pem" || true
cat "$ca/root.crt" >> "$ca/bundle.pem"
chmod 644 "$ca/bundle.pem"
} }
pasture_down() { pasture_down() {
podman ps -a --format '{{.Names}}' | grep '^pasture-' | xargs -r podman rm -f >/dev/null 2>&1 || true podman ps -a --format '{{.Names}}' | grep '^pasture-' | xargs -r podman rm -f >/dev/null 2>&1 || true
podman volume ls --format '{{.Name}}' | grep '^pasture-' | xargs -r podman volume rm -f >/dev/null 2>&1 || true podman volume ls --format '{{.Name}}' | grep '^pasture-' | xargs -r podman volume rm -f >/dev/null 2>&1 || true
podman network rm $net >/dev/null 2>&1 || true podman network rm $net >/dev/null 2>&1 || true
rm -rf "$here/.state" rm -rf "$here/.state" "$ca"
} }
+56
View File
@@ -0,0 +1,56 @@
# Mastodon: web (puma) and sidekiq from one image, on the shared Postgres and Redis. Streaming is not needed.
MASTODON_IMAGE=ghcr.io/mastodon/mastodon:v4.7.3
. "$here/peers/shared.sh"
mastodon_env() {
cat <<ENV
LOCAL_DOMAIN=mastodon.test
RAILS_ENV=production
NODE_ENV=production
DB_HOST=postgres
DB_USER=pasture
DB_PASS=pasture
DB_NAME=mastodon
REDIS_URL=redis://redis:6379/1
SECRET_KEY_BASE=pasture0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
OTP_SECRET=pasture000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
ACTIVE_RECORD_ENCRYPTION_DETERMINISTIC_KEY=pasturedeterministickey0000000000
ACTIVE_RECORD_ENCRYPTION_KEY_DERIVATION_SALT=pasturederivationsalt000000000000
ACTIVE_RECORD_ENCRYPTION_PRIMARY_KEY=pastureprimarykey0000000000000000
ALLOWED_PRIVATE_ADDRESSES=10.0.0.0/8,172.16.0.0/12,192.168.0.0/16
SSL_CERT_FILE=/pasture/ca/bundle.pem
SKIP_POST_DEPLOYMENT_MIGRATIONS=false
WEB_CONCURRENCY=0
MAX_THREADS=8
RAILS_LOG_LEVEL=warn
${MASTODON_EXTRA_ENV:-}
ENV
}
mastodon_up() {
shared_postgres_up
shared_redis_up
mastodon_env > "$here/.ca/mastodon.env"
local common=(--network $net --env-file "$here/.ca/mastodon.env" -v "$ca:/pasture/ca:z,ro")
podman run --rm "${common[@]}" -e SAFETY_ASSURED=1 $MASTODON_IMAGE bundle exec rails db:prepare >/dev/null
podman run -d --replace --name pasture-mastodon "${common[@]}" -p 127.0.0.1:6973:3000 $MASTODON_IMAGE bundle exec puma -C config/puma.rb >/dev/null
podman run -d --replace --name pasture-mastodon-sidekiq "${common[@]}" $MASTODON_IMAGE bundle exec sidekiq >/dev/null
wait_http http://127.0.0.1:6973/health 90
podman exec pasture-mastodon bin/tootctl accounts create mastouser --email mastouser@mastodon.test --confirmed --role Owner >/dev/null 2>&1 || true
podman exec pasture-mastodon bin/tootctl accounts approve mastouser >/dev/null 2>&1 || true
echo "mastodon: https://mastodon.test:6443"
}
# mastodon_token [user]: a token for a Mastodon user, made directly (Mastodon offers no password grant). Owners get
# admin:read too, for reading reports.
mastodon_token() {
podman exec pasture-mastodon bin/rails runner '
app = Doorkeeper::Application.find_or_create_by!(name: "pasture") { |a| a.redirect_uri = "urn:ietf:wg:oauth:2.0:oob"; a.scopes = "read write follow admin:read" }
user = Account.find_local("'"${1:-mastouser}"'").user
puts Doorkeeper::AccessToken.create!(application_id: app.id, resource_owner_id: user.id, scopes: "read write follow admin:read").token' 2>/dev/null | tail -1
}
mastodon_user() { # name
podman exec pasture-mastodon bin/tootctl accounts create "$1" --email "$1@mastodon.test" --confirmed >/dev/null 2>&1 || true
podman exec pasture-mastodon bin/tootctl accounts approve "$1" >/dev/null 2>&1 || true
}
+13
View File
@@ -0,0 +1,13 @@
# Services several peers share: one Postgres (each peer gets its own database) and one Redis (each its own db number).
shared_postgres_up() {
podman container exists pasture-postgres && return 0
podman run -d --replace --name pasture-postgres --network $net --network-alias postgres \
-e POSTGRES_USER=pasture -e POSTGRES_PASSWORD=pasture -e POSTGRES_DB=pasture docker.io/library/postgres:17-alpine >/dev/null
for _ in $(seq 1 60); do podman exec pasture-postgres pg_isready -U pasture >/dev/null 2>&1 && return 0; sleep 1; done
echo "postgres did not start" >&2; return 1
}
shared_redis_up() {
podman container exists pasture-redis && return 0
podman run -d --replace --name pasture-redis --network $net --network-alias redis docker.io/library/redis:7-alpine >/dev/null
}
+21
View File
@@ -0,0 +1,21 @@
# The opt-in crawler against the pasture's own peers. Needs PrivaPub started with the crawler on:
# PRIVAPUB_ENV="Statistics__Crawler__Enabled=true Statistics__Crawler__Seeds__0=mastodon.test" tools/pasture/run.sh up mastodon
# PrivaPub starts before its peers, so a first visit can find robots.txt answering 502 and rightly keep out for a week;
# the scenario then forgets that visit and restarts PrivaPub, which plans again.
echo "crawler"
privapub_root >/dev/null
podman exec -w /app pasture-privapub /app/PrivaPub admin promote "$ROOT_USER" >/dev/null 2>&1 || true
admin=$(curl -s -X POST $P/clientapi/user/login -H 'Content-Type: application/json' -d "{\"userName\":\"$ROOT_USER\",\"password\":\"$ROOT_PASS\"}" | j "print(d['token'])")
crawler() { curl -s -H "Authorization: Bearer $admin" "$P/clientapi/admin/statistics/crawler"; }
if [ "$(crawler | j "print(any(r['robotsDisallowed'] for r in d['recent']))")" = "True" ]; then
podman exec pasture-mongo mongosh --quiet PrivaPub --eval 'db.Job.deleteMany({Kind:{$in:[8,9]}}); db.RemoteInstance.updateMany({}, {$unset:{LastCrawledAt:1, RobotsDisallowed:1, CrawlError:1}})' >/dev/null
podman restart pasture-privapub >/dev/null
until_true 60 'curl -fs -o /dev/null $P/build.json'
fi
[ "$(crawler | j "print(d['enabled'])")" = "True" ] && ok "the crawler is on" || { ko "the crawler is off: start PrivaPub with PRIVAPUB_ENV=Statistics__Crawler__Enabled=true"; return 0; }
until_true 60 '[ "$(crawler | j "print(any(r[\"host\"]==\"mastodon.test\" and r[\"lastCrawledAt\"] for r in d[\"recent\"]))")" = "True" ]' \
&& ok "the crawler visited mastodon.test" || ko "the crawler never visited mastodon.test"
[ "$(crawler | j "print(next(r['peersCount'] is not None for r in d['recent'] if r['host']=='mastodon.test'))")" = "True" ] \
&& ok "it read mastodon.test's peers list" || ko "no peers list read from mastodon.test"
[ "$(curl -s -H "Authorization: Bearer $admin" "$P/clientapi/admin/statistics/hosts/mastodon.test?days=1" | j "print((d['instance'] or {}).get('software'))")" = "mastodon" ] \
&& ok "mastodon.test is described" || ko "mastodon.test is not described"
+183
View File
@@ -0,0 +1,183 @@
# Mastodon 4.7: discovery, follows, posts, CW, replies both ways, likes and boosts with their undos, DMs, polls,
# FEP-044f quotes both ways, edits and deletes both ways, media with alt text, locked follows, blocks, statistics.
M=https://mastodon.test:6443
mcurl() { curl -sk --resolve mastodon.test:6443:127.0.0.1 "$@"; }
. "$here/peers/mastodon.sh"
echo "mastodon"
MT=$(mastodon_token)
MH="Authorization: Bearer $MT"
[ -n "$MT" ] && ok "Mastodon token for mastouser" || { ko "Mastodon token"; return 1; }
AT=$(privapub_token alice_masto)
AH="Authorization: Bearer $AT"
[ -n "$AT" ] && ok "PrivaPub token for alice_masto" || { ko "PrivaPub token for alice_masto"; return 1; }
# what Mastodon holds of an account, newest first; and the one status whose uri is given
m_statuses() { mcurl -H "$MH" "$M/api/v1/accounts/$1/statuses?limit=40"; }
m_status_by_uri() { m_statuses "$1" | j "print(json.dumps(next((s for s in d if s['uri']=='$2'), None)))"; }
echo " discovery"
alice_on_m=$(mcurl -H "$MH" "$M/api/v2/search?q=@alice_masto@privapub.test&resolve=true&type=accounts" | j "print(d['accounts'][0]['id'])")
[ -n "$alice_on_m" ] && ok "Mastodon resolves @alice_masto@privapub.test" || ko "Mastodon cannot resolve alice_masto"
masto_on_p=$(curl -s -H "$AH" "$P/api/v2/search?q=mastouser@mastodon.test&resolve=true&type=accounts" | j "print(d['accounts'][0]['id'])")
[ -n "$masto_on_p" ] && ok "PrivaPub resolves @mastouser@mastodon.test" || ko "PrivaPub cannot resolve mastouser"
echo " follows"
mcurl -o /dev/null -X POST -H "$MH" "$M/api/v1/accounts/$alice_on_m/follow"
until_true 30 '[ "$(mcurl -H "$MH" "$M/api/v1/accounts/relationships?id[]=$alice_on_m" | j "print(d[0][\"following\"])")" = "True" ]' && ok "mastouser follows alice_masto (Accept arrived)" || ko "Mastodon's follow not accepted"
curl -s -o /dev/null -X POST -H "$AH" "$P/api/v1/accounts/$masto_on_p/follow"
until_true 30 '[ "$(curl -s -H "$AH" "$P/api/v1/accounts/relationships?id[]=$masto_on_p" | j "print(d[0][\"following\"])")" = "True" ]' && ok "alice_masto follows mastouser (Accept arrived)" || ko "PrivaPub's follow not accepted"
echo " posts"
a_post=$(curl -s -X POST -H "$AH" $P/api/v1/statuses -d 'status=Hello Mastodon from PrivaPub&visibility=public')
a_post_id=$(echo "$a_post" | j "print(d['id'])"); a_post_uri=$(echo "$a_post" | j "print(d['uri'])")
until_true 30 '[ "$(m_status_by_uri "$alice_on_m" "$a_post_uri")" != "null" ]' && ok "alice_masto's post reaches Mastodon" || ko "post missing on Mastodon"
a_post_on_m=$(m_status_by_uri "$alice_on_m" "$a_post_uri" | j "print(d['id'])")
m_post=$(mcurl -X POST -H "$MH" "$M/api/v1/statuses" -d 'status=Hello PrivaPub from Mastodon&visibility=public' | j "print(d['id'])")
until_true 30 'curl -s -H "$AH" "$P/api/v1/timelines/home" | grep -q "Hello PrivaPub from Mastodon"' && ok "mastouser's post reaches alice_masto's home" || ko "Mastodon's post missing on PrivaPub"
m_post_on_p=$(curl -s -H "$AH" "$P/api/v1/timelines/home" | j "print(next(s['id'] for s in d if 'Hello PrivaPub from Mastodon' in s['content']))")
cw_uri=$(curl -s -X POST -H "$AH" $P/api/v1/statuses -d 'status=behind a warning&spoiler_text=spoilers&visibility=public' | j "print(d['uri'])")
until_true 30 '[ "$(m_status_by_uri "$alice_on_m" "$cw_uri" | j "print(d and d[\"spoiler_text\"]==\"spoilers\" and d[\"sensitive\"])")" = "True" ]' && ok "a content warning survives to Mastodon" || ko "content warning lost on Mastodon"
fo_uri=$(curl -s -X POST -H "$AH" $P/api/v1/statuses -d 'status=only for followers&visibility=private' | j "print(d['uri'])")
until_true 30 '[ "$(m_status_by_uri "$alice_on_m" "$fo_uri" | j "print(d and d[\"visibility\"])")" = "private" ]' && ok "a followers-only post reaches Mastodon as private" || ko "followers-only post missing or widened on Mastodon"
[ "$(pfetch -o /dev/null -w '%{http_code}' -H 'Accept: application/activity+json' "$fo_uri")" = "404" ] && ok "the followers-only post is not served unsigned" || ko "followers-only post served unsigned"
echo " replies"
mcurl -o /dev/null -X POST -H "$MH" "$M/api/v1/statuses" -d "status=@alice_masto@privapub.test replying from Mastodon&in_reply_to_id=$a_post_on_m&visibility=public"
until_true 30 'curl -s -H "$AH" "$P/api/v1/notifications" | j "print(any(n[\"type\"]==\"mention\" for n in d))" | grep -q True' && ok "mastouser's reply notifies alice_masto" || ko "reply did not notify"
until_true 15 '[ "$(curl -s -H "$AH" "$P/api/v1/statuses/$a_post_id/context" | j "print(len(d[\"descendants\"]))")" -ge 1 ]' && ok "the reply threads under alice_masto's post" || ko "reply not threaded on PrivaPub"
curl -s -o /dev/null -X POST -H "$AH" $P/api/v1/statuses -d "status=@mastouser@mastodon.test replying from PrivaPub&in_reply_to_id=$m_post_on_p&visibility=public"
until_true 30 '[ "$(mcurl -H "$MH" "$M/api/v1/statuses/$m_post/context" | j "print(len(d[\"descendants\"]))")" -ge 1 ]' && ok "alice_masto's reply threads under mastouser's post" || ko "outbound reply not threaded on Mastodon"
echo " likes and boosts"
curl -s -o /dev/null -X POST -H "$AH" "$P/api/v1/statuses/$m_post_on_p/favourite"
curl -s -o /dev/null -X POST -H "$AH" "$P/api/v1/statuses/$m_post_on_p/reblog"
until_true 30 '[ "$(mcurl -H "$MH" "$M/api/v1/statuses/$m_post" | j "print(d[\"favourites_count\"])")" = "1" ]' && ok "alice_masto's like counts on Mastodon" || ko "like not counted on Mastodon"
until_true 30 '[ "$(mcurl -H "$MH" "$M/api/v1/statuses/$m_post" | j "print(d[\"reblogs_count\"])")" = "1" ]' && ok "alice_masto's boost counts on Mastodon" || ko "boost not counted on Mastodon"
curl -s -o /dev/null -X POST -H "$AH" "$P/api/v1/statuses/$m_post_on_p/unfavourite"
curl -s -o /dev/null -X POST -H "$AH" "$P/api/v1/statuses/$m_post_on_p/unreblog"
until_true 30 '[ "$(mcurl -H "$MH" "$M/api/v1/statuses/$m_post" | j "print(d[\"favourites_count\"], d[\"reblogs_count\"])")" = "0 0" ]' && ok "alice_masto's unlike and unboost reach Mastodon" || ko "undo of like or boost not applied on Mastodon"
mcurl -o /dev/null -X POST -H "$MH" "$M/api/v1/statuses/$a_post_on_m/favourite"
mcurl -o /dev/null -X POST -H "$MH" "$M/api/v1/statuses/$a_post_on_m/reblog"
until_true 30 '[ "$(curl -s -H "$AH" "$P/api/v1/statuses/$a_post_id" | j "print(d[\"favourites_count\"], d[\"reblogs_count\"])")" = "1 1" ]' && ok "mastouser's like and boost count on PrivaPub" || ko "like or boost not counted on PrivaPub"
mcurl -o /dev/null -X POST -H "$MH" "$M/api/v1/statuses/$a_post_on_m/unfavourite"
mcurl -o /dev/null -X POST -H "$MH" "$M/api/v1/statuses/$a_post_on_m/unreblog"
until_true 30 '[ "$(curl -s -H "$AH" "$P/api/v1/statuses/$a_post_id" | j "print(d[\"favourites_count\"], d[\"reblogs_count\"])")" = "0 0" ]' && ok "mastouser's unlike and unboost reach PrivaPub" || ko "undo of like or boost not applied on PrivaPub"
echo " direct messages"
curl -s -o /dev/null -X POST -H "$AH" $P/api/v1/statuses -d 'status=@mastouser@mastodon.test a secret for Mastodon&visibility=direct'
until_true 30 'mcurl -H "$MH" "$M/api/v1/conversations" | grep -q "a secret for Mastodon"' && ok "alice_masto's DM reaches mastouser" || ko "DM missing on Mastodon"
mcurl -o /dev/null -X POST -H "$MH" "$M/api/v1/statuses" -d 'status=@alice_masto@privapub.test a secret for PrivaPub&visibility=direct'
until_true 30 'curl -s -H "$AH" "$P/api/v1/conversations" | grep -q "a secret for PrivaPub"' && ok "mastouser's DM reaches alice_masto" || ko "DM missing on PrivaPub"
! curl -s "$P/api/v1/timelines/public" | grep -q "a secret for PrivaPub" && ok "the DM is not on PrivaPub's public timeline" || ko "DM leaked to the public timeline"
echo " polls"
a_poll_uri=$(curl -s -X POST -H "$AH" $P/api/v1/statuses -d 'status=cats or dogs&visibility=public&poll[options][]=cats&poll[options][]=dogs&poll[expires_in]=3600' | j "print(d['uri'])")
until_true 30 '[ "$(m_status_by_uri "$alice_on_m" "$a_poll_uri" | j "print(len(d[\"poll\"][\"options\"]))")" = "2" ]' && ok "alice_masto's poll reaches Mastodon as a poll" || ko "poll missing on Mastodon"
a_poll_on_m=$(m_status_by_uri "$alice_on_m" "$a_poll_uri" | j "print(d['poll']['id'])")
mcurl -o /dev/null -X POST -H "$MH" "$M/api/v1/polls/$a_poll_on_m/votes" -d 'choices[]=1'
a_poll_id=$(curl -s -H "$AH" "$P/api/v1/accounts/verify_credentials" | j "print(d['id'])")
until_true 30 '[ "$(curl -s -H "$AH" "$P/api/v1/accounts/$a_poll_id/statuses" | j "print(next(s[\"poll\"][\"options\"][1][\"votes_count\"] for s in d if s[\"uri\"]==\"$a_poll_uri\"))")" = "1" ]' && ok "mastouser's vote counts on PrivaPub" || ko "vote not counted on PrivaPub"
m_poll=$(mcurl -X POST -H "$MH" "$M/api/v1/statuses" -d 'status=tea or coffee, Mastodon asks&visibility=public&poll[options][]=tea&poll[options][]=coffee&poll[expires_in]=3600' | j "print(d['id'])")
until_true 30 'curl -s -H "$AH" "$P/api/v1/timelines/home" | j "print(any(s[\"poll\"] and \"Mastodon asks\" in s[\"content\"] for s in d))" | grep -q True' && ok "mastouser's poll reaches PrivaPub as a poll" || ko "poll missing on PrivaPub"
m_poll_on_p=$(curl -s -H "$AH" "$P/api/v1/timelines/home" | j "print(next(s['poll']['id'] for s in d if s['poll'] and 'Mastodon asks' in s['content']))")
curl -s -o /dev/null -X POST -H "$AH" "$P/api/v1/polls/$m_poll_on_p/votes" -d 'choices[]=0'
until_true 30 '[ "$(mcurl -H "$MH" "$M/api/v1/statuses/$m_poll" | j "print(d[\"poll\"][\"options\"][0][\"votes_count\"])")" = "1" ]' && ok "alice_masto's vote counts on Mastodon" || ko "vote not counted on Mastodon"
echo " quotes"
q_target=$(mcurl -X POST -H "$MH" "$M/api/v1/statuses" -d 'status=quote me if you like&visibility=public' | j "print(d['id'])")
until_true 30 'curl -s -H "$AH" "$P/api/v1/timelines/home" | grep -q "quote me if you like"' || true
q_target_on_p=$(curl -s -H "$AH" "$P/api/v1/timelines/home" | j "print(next(s['id'] for s in d if 'quote me if you like' in s['content']))")
a_quote=$(curl -s -X POST -H "$AH" $P/api/v1/statuses -d "status=quoting Mastodon&visibility=public&quoted_status_id=$q_target_on_p" | j "print(d['id'])")
until_true 30 '[ "$(curl -s -H "$AH" "$P/api/v1/statuses/$a_quote" | j "print((d.get(\"quote\") or {}).get(\"state\"))")" = "accepted" ]' && ok "Mastodon approves alice_masto's quote (FEP-044f)" || ko "quote of a Mastodon post not approved"
m_quote=$(mcurl -X POST -H "$MH" "$M/api/v1/statuses" -d "status=quoting PrivaPub&visibility=public&quoted_status_id=$a_post_on_m" | j "print(d['id'])")
until_true 30 '[ "$(mcurl -H "$MH" "$M/api/v1/statuses/$m_quote" | j "print((d.get(\"quote\") or {}).get(\"state\"))")" = "accepted" ]' && ok "PrivaPub approves mastouser's quote (FEP-044f)" || ko "quote of a PrivaPub post not approved on Mastodon"
echo " media"
python3 -c "
import struct,zlib
w=h=8
raw=b''.join(b'\x00'+bytes([200,60,60])*w for _ in range(h))
png=b'\x89PNG\r\n\x1a\n'+b''.join(struct.pack('>I',len(c))+t+c+struct.pack('>I',zlib.crc32(t+c)&0xffffffff) for t,c in [(b'IHDR',struct.pack('>IIBBBBB',w,h,8,2,0,0,0)),(b'IDAT',zlib.compress(raw)),(b'IEND',b'')])
open('$work/red.png','wb').write(png)"
a_media=$(curl -s -X POST -H "$AH" "$P/api/v2/media" -F "file=@$work/red.png;type=image/png" -F 'description=a red square' | j "print(d['id'])")
a_media_uri=$(curl -s -X POST -H "$AH" $P/api/v1/statuses -d "status=a picture&visibility=public&media_ids[]=$a_media" | j "print(d['uri'])")
until_true 30 '[ "$(m_status_by_uri "$alice_on_m" "$a_media_uri" | j "print(d[\"media_attachments\"][0][\"description\"])")" = "a red square" ]' && ok "an image with alt text reaches Mastodon" || ko "image or alt text missing on Mastodon"
m_media=$(mcurl -X POST -H "$MH" "$M/api/v2/media" -F "file=@$work/red.png;type=image/png" -F 'description=a red square from Mastodon' | j "print(d['id'])")
until_true 10 '[ "$(mcurl -o /dev/null -w "%{http_code}" -H "$MH" "$M/api/v1/media/$m_media")" = "200" ]' || true
mcurl -o /dev/null -X POST -H "$MH" "$M/api/v1/statuses" -d "status=a picture from Mastodon&visibility=public&media_ids[]=$m_media"
until_true 30 'curl -s -H "$AH" "$P/api/v1/timelines/home" | j "print(any(\"a picture from Mastodon\" in s[\"content\"] and s[\"media_attachments\"] and \"/media/proxy/\" in s[\"media_attachments\"][0][\"url\"] and s[\"media_attachments\"][0][\"description\"]==\"a red square from Mastodon\" for s in d))" | grep -q True' \
&& ok "an image with alt text from Mastodon arrives through our proxy" || ko "Mastodon's image missing, unproxied or without alt text"
echo " edits and deletes"
curl -s -o /dev/null -X PUT -H "$AH" "$P/api/v1/statuses/$a_post_id" -d 'status=Hello Mastodon from PrivaPub, edited'
until_true 30 'mcurl -H "$MH" "$M/api/v1/statuses/$a_post_on_m" | grep -q "edited"' && ok "alice_masto's edit reaches Mastodon" || ko "edit not applied on Mastodon"
mcurl -o /dev/null -X PUT -H "$MH" "$M/api/v1/statuses/$m_post" -d 'status=Hello PrivaPub from Mastodon, edited'
until_true 30 '[ "$(curl -s -H "$AH" "$P/api/v1/statuses/$m_post_on_p/history" | j "print(len(d))")" = "2" ]' && ok "mastouser's edit reaches PrivaPub with its history" || ko "Mastodon's edit not applied on PrivaPub"
cw_on_m=$(m_status_by_uri "$alice_on_m" "$cw_uri" | j "print(d['id'])")
cw_id=$(curl -s -H "$AH" "$P/api/v1/accounts/$a_poll_id/statuses" | j "print(next(s['id'] for s in d if s['uri']=='$cw_uri'))")
curl -s -o /dev/null -X DELETE -H "$AH" "$P/api/v1/statuses/$cw_id"
until_true 30 '[ "$(mcurl -o /dev/null -w "%{http_code}" -H "$MH" "$M/api/v1/statuses/$cw_on_m")" = "404" ]' && ok "alice_masto's delete reaches Mastodon" || ko "delete not applied on Mastodon"
[ "$(pfetch -o /dev/null -w '%{http_code}' -H 'Accept: application/activity+json' "$cw_uri")" = "410" ] && ok "the deleted post answers 410" || ko "deleted post does not answer 410"
mcurl -o /dev/null -X DELETE -H "$MH" "$M/api/v1/statuses/$m_poll"
until_true 30 '! curl -s -H "$AH" "$P/api/v1/timelines/home" | grep -q "Mastodon asks"' && ok "mastouser's delete reaches PrivaPub" || ko "Mastodon's delete not applied on PrivaPub"
echo " locked follows"
mcurl -o /dev/null -X PATCH -H "$MH" "$M/api/v1/accounts/update_credentials" -d 'locked=true'
BT=$(privapub_token bob_masto); BH="Authorization: Bearer $BT"
masto_on_p_b=$(curl -s -H "$BH" "$P/api/v2/search?q=mastouser@mastodon.test&resolve=true&type=accounts" | j "print(d['accounts'][0]['id'])")
curl -s -o /dev/null -X POST -H "$BH" "$P/api/v1/accounts/$masto_on_p_b/follow"
until_true 30 'mcurl -H "$MH" "$M/api/v1/follow_requests" | j "print(any(a[\"acct\"]==\"bob_masto@privapub.test\" for a in d))" | grep -q True' && ok "bob_masto's follow waits as a request on locked mastouser" || ko "follow request missing on Mastodon"
bob_on_m=$(mcurl -H "$MH" "$M/api/v1/follow_requests" | j "print(next(a['id'] for a in d if a['acct']=='bob_masto@privapub.test'))")
mcurl -o /dev/null -X POST -H "$MH" "$M/api/v1/follow_requests/$bob_on_m/authorize"
until_true 30 '[ "$(curl -s -H "$BH" "$P/api/v1/accounts/relationships?id[]=$masto_on_p_b" | j "print(d[0][\"following\"])")" = "True" ]' && ok "mastouser's approval reaches PrivaPub" || ko "approval not applied on PrivaPub"
echo " circles"
jwt=$(privapub_root)
alice_id=$(curl -s -H "$AH" "$P/api/v1/accounts/verify_credentials" | j "print(d['id'])")
circle_name="circle$(date +%s)"
circle=$(curl -s -X POST $P/clientapi/group/insert -H 'Content-Type: application/json' -H "Authorization: Bearer $jwt" \
-d "{\"avatarId\":\"$alice_id\",\"userName\":\"$circle_name\",\"name\":\"a circle\",\"description\":\"just us\",\"isCommunity\":false}" | j "print(d['id'])")
circle_on_m=$(mcurl -H "$MH" "$M/api/v2/search?q=@$circle_name@privapub.test&resolve=true&type=accounts" | j "print(d['accounts'][0]['id'])")
mcurl -o /dev/null -X POST -H "$MH" "$M/api/v1/accounts/$circle_on_m/follow"
until_true 15 '[ "$(mcurl -H "$MH" "$M/api/v1/accounts/relationships?id[]=$circle_on_m" | j "print(d[0][\"requested\"])")" = "True" ]' && ok "mastouser's request to join a circle waits for its owner" || ko "joining the circle was not held for approval"
# Mastodon 4.7 names its actors by number (https://mastodon.test/ap/users/<id>), so the pending request says who asked
requester=$(podman exec pasture-mongo mongosh --quiet PrivaPub --eval "print(db.Follower.findOne({LocalActorId:'$circle', IsAccepted:false}).ActorURI)")
curl -s -o /dev/null -X POST $P/clientapi/group/approve -H 'Content-Type: application/json' -H "Authorization: Bearer $jwt" \
-d "{\"avatarId\":\"$alice_id\",\"groupId\":\"$circle\",\"memberActorURI\":\"$requester\"}"
until_true 30 '[ "$(mcurl -H "$MH" "$M/api/v1/accounts/relationships?id[]=$circle_on_m" | j "print(d[0][\"following\"])")" = "True" ]' && ok "the owner's approval makes mastouser a circle member" || ko "circle approval did not reach Mastodon"
circle_post=$(curl -s -X POST $P/clientapi/post/insert -H 'Content-Type: application/json' -H "Authorization: Bearer $jwt" \
-d "{\"avatarId\":\"$alice_id\",\"text\":\"only the circle sees this\",\"groupId\":\"$circle\"}")
circle_uri=$(podman exec pasture-mongo mongosh --quiet PrivaPub --eval 'print(db.Post.findOne({Text:/only the circle sees this/},{ObjectURI:1}).ObjectURI)')
# Mastodon 4.7 learns whose personal inbox a delivery reached only from /users/<name>/inbox, never from the numeric
# /ap/users/<id>/inbox it now advertises, and keeps a post naming no local account only for that recipient
# (InboxesController#account_required?, Create#addresses_local_accounts?). A circle post names only the circle.
m_stored() { podman exec pasture-mastodon bin/rails runner "puts Status.exists?(uri: '$1')" 2>/dev/null | tail -1; }
if until_true 15 '[ "$(m_stored "$circle_uri")" = "true" ]'; then
ok "a circle post reaches its Mastodon member"
else
xf "a circle post reaches its Mastodon member (Mastodon 4.7 loses the recipient of numeric-inbox deliveries; owner decision pending)"
fi
mastodon_user outsider
OT=$(mastodon_token outsider)
alice_on_m_o=$(mcurl -H "Authorization: Bearer $OT" "$M/api/v2/search?q=@alice_masto@privapub.test&resolve=true&type=accounts" | j "print(d['accounts'][0]['id'])")
[ "$(mcurl -H "Authorization: Bearer $OT" "$M/api/v1/accounts/$alice_on_m_o/statuses?limit=40" | j "print(any(s['uri']=='$circle_uri' for s in d))")" = "False" ] \
&& ok "another Mastodon user does not see the circle post" || ko "the circle post leaked to a non-member"
[ "$(pfetch -o /dev/null -w '%{http_code}' -H 'Accept: application/activity+json' "$circle_uri")" = "404" ] && ok "the circle post is not served unsigned" || ko "circle post served unsigned"
echo " reports"
curl -s -o /dev/null -X POST -H "$AH" "$P/api/v1/reports" -d "account_id=$masto_on_p&comment=pasture-report-$circle_name&forward=true"
until_true 30 'mcurl -H "$MH" "$M/api/v1/admin/reports" | grep -q "pasture-report-$circle_name"' && ok "a report reaches Mastodon's moderators" || ko "report missing on Mastodon"
[ "$(mcurl -H "$MH" "$M/api/v1/admin/reports" | j "print(next((r['account']['username'] for r in d if 'pasture-report-$circle_name' in r['comment']), ''))")" != "alice_masto" ] \
&& ok "the report does not come from the reporting persona" || ko "the report names the reporting persona"
echo " blocks and unfollows"
curl -s -o /dev/null -X POST -H "$AH" "$P/api/v1/accounts/$masto_on_p/unfollow"
until_true 30 '[ "$(mcurl -H "$MH" "$M/api/v1/accounts/relationships?id[]=$alice_on_m" | j "print(d[0][\"followed_by\"])")" = "False" ]' && ok "alice_masto's unfollow reaches Mastodon" || ko "unfollow not applied on Mastodon"
curl -s -o /dev/null -X POST -H "$AH" "$P/api/v1/accounts/$masto_on_p/block"
until_true 30 '[ "$(mcurl -H "$MH" "$M/api/v1/accounts/relationships?id[]=$alice_on_m" | j "print(d[0][\"blocked_by\"])")" = "True" ]' && ok "alice_masto's block reaches Mastodon" || ko "block not applied on Mastodon"
curl -s -o /dev/null -X POST -H "$AH" "$P/api/v1/accounts/$masto_on_p/unblock"
mcurl -o /dev/null -X POST -H "$MH" "$M/api/v1/accounts/$alice_on_m/block"
sleep 5; xf "Mastodon's block is enforced on PrivaPub (inbound Block is P7)"
echo " statistics"
stats_check mastodon.test mastodon