diff --git a/CLAUDE.md b/CLAUDE.md index 55c2170..68aa35a 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -471,6 +471,10 @@ tools/pasture/interop.sh [gts mastodon ...] # each peer tools/pasture/run.sh down # removes every pasture container and volume ``` +- **A change to what PrivaPub sends** (the `@context`, an actor's properties, an activity's shape, a delivery's headers) + is run against every peer's scenario before it is committed, not only the peers it was written for. One reader + stricter than the rest fails on it alone: the 2026-10-06 `proof` term passed every peer but Smithereen, whose JSON-LD + 1.0 reader refused every document carrying it. - **Layout:** `lib/pasture.sh` (network, Caddy, Mongo, PrivaPub), `peers/.sh` (`_up`, plus `peers/shared.sh` for the Postgres and Redis several peers share), `lib/interop.sh` (`ok`, `ko`, `xf` for a check expected to fail until a later phase, `privapub_token `, `stats_check `), `scenarios/.sh`. Each peer talks to @@ -612,6 +616,12 @@ tools/pasture/run.sh down # removes e `.env` the image rewrites at each start, `spark fediverse:broadcast` looping in the container. The podcast `@pod` is made and published through its admin pages (`cp_web`, CSRF on each form). `scenarios/castopod.sh`, 15 checks; the episode's sound is made with ffmpeg on the workstation. +- **Ktistec (3.13.0, `peers/ktistec.sh`):** built from its tag by `images/ktistec` (its own Dockerfile, pinned: a static + Crystal server, its assets with node), SQLite in the `pasture-ktistec` volume, the pasture bundle mounted as + `/etc/ssl/cert.pem`. The server's name and ktuser come from its setup API (every field required, `summary` too); + `ktistec_token` signs in on `/sessions`. `scenarios/ktistec.sh`, 27 checks: its Mastodon API where it has the route, + its outbox API (`kn`) for follows, polls (`poll-duration` is seconds from now), edits, deletions and quotes, and a + copy of its SQLite database (`kt_sql`) for what it holds. - **Forte (26.9.10, `peers/forte.sh`):** built from its tag by `images/forte` (composer on Hubzilla's PHP image, an nginx routing through `index.php?req=`), on the shared MySQL (database `forte`), its `.htconfig.php` written to `.state/forte`, a cron sidecar (`src/Daemon/Run.php Cron`). The pasture bundle is mounted over `library/cacert.pem`, diff --git a/FEDERATION.md b/FEDERATION.md index 5ceab84..2861e7c 100644 --- a/FEDERATION.md +++ b/FEDERATION.md @@ -48,6 +48,7 @@ and every run starting clean, with signed fetches required (as privapub.thepra.d - **Hubzilla 11.4.1** with its pubcrawl addon - **Forte 26.9.10** (portable identities: actors served through `/.well-known/apgateway/did:key:…`) - **Castopod 1.15.5** (podcasts as actors, their episodes with the sound) +- **Ktistec 3.13.0** - **Activity-Relay 2.0.9** and **aode-relay 0.3.129**, as relays PrivaPub reads from - in the town only (a seeded community checked server by server): **Hollo 0.9.19**, **Iceshrimp.NET 2026.1.2-beta**, **Pleroma 2.10.2** diff --git a/docs/INTEROP.md b/docs/INTEROP.md index 7aaeb76..a02b1bb 100644 --- a/docs/INTEROP.md +++ b/docs/INTEROP.md @@ -872,6 +872,21 @@ without a port, before it gives out its OAuth client. - **Pasture evidence (2026-10-06, `tools/pasture/scenarios/castopod.sh`):** 15 checks pass: alice follows the podcast, an episode reaches her with its sound, her like, boost and comment land there, the unfollow, statistics. +### Ktistec 3.13.0 + +- **A small server in Crystal on SQLite**, with its own JSON API (an outbox taking `Publish`, `Follow`, `Like`, + `Announce`, `Undo`, `Delete`) and part of Mastodon's (posts, replies, likes, boosts, timelines, relationships; no + search, no edits, no polls, no counts: its `favourites_count` and `reblogs_count` stay at 0). Both take the bearer + token of a session signed in on `/sessions`. +- **Polls** are `Question`s, votes FEP-9967's `Note{name, inReplyTo}`, taken once FEP-9967's checks pass. +- **Quotes (FEP-044f) both ways.** With its default manual approval of quotes, its posts' `canQuote` names only their + author and offers no `manualApproval`, so other servers (PrivaPub, Mastodon) may not even ask; with it off, + `automaticApproval` is `Public` and its `QuoteRequest`s and `Accept`s work with ours. +- **Its settings are written whole** (`/settings/actor`): a flag left out of the request is turned off. +- **Pasture evidence (2026-10-06, `tools/pasture/scenarios/ktistec.sh`):** 27 checks pass, twice in a row on the same + pasture: follows both ways, posts, replies, likes and boosts with their undos both ways, its poll and alice's vote, + quotes both ways, edits and deletions both ways, the unfollow, statistics. + ### Forte 26.9.10 - **Portable identities (FEP-ef61):** a channel's actor is `https:///.well-known/apgateway/did:key:z6Mk…/actor`, diff --git a/tools/pasture/Caddyfile b/tools/pasture/Caddyfile index 89570c4..8e8b77b 100644 --- a/tools/pasture/Caddyfile +++ b/tools/pasture/Caddyfile @@ -113,6 +113,11 @@ castopod.test { reverse_proxy pasture-castopod:8080 } +ktistec.test { + tls internal + reverse_proxy pasture-ktistec:3000 +} + forte.test { tls internal reverse_proxy pasture-forte:8080 diff --git a/tools/pasture/images/ktistec/Containerfile b/tools/pasture/images/ktistec/Containerfile new file mode 100644 index 0000000..e9812ba --- /dev/null +++ b/tools/pasture/images/ktistec/Containerfile @@ -0,0 +1,24 @@ +# Ktistec 3.13.0 from its tag: the project publishes no image. Its own Dockerfile, pinned: a static server built with +# Crystal, its scripts and styles with node +FROM docker.io/crystallang/crystal:1.19.1-alpine AS builder +RUN apk -U --no-progress add make sqlite-static yaml-dev musl-dev gc-dev gc-static pcre2-dev pcre2-static gmp-dev gmp-static +WORKDIR /build +RUN git clone --branch v3.13.0 --depth 1 https://github.com/toddsundsted/ktistec . +RUN shards install --production +RUN crystal build src/ktistec/server.cr --static --no-debug --release + +FROM docker.io/library/node:22-alpine AS nodebuilder +WORKDIR /build +COPY --from=builder /build /build +RUN npm ci && npm run build + +FROM docker.io/library/alpine:3.22 +RUN apk -U --no-progress --no-cache add tzdata +WORKDIR /app +COPY --from=nodebuilder /build/etc /app/etc +COPY --from=nodebuilder /build/public /app/public +COPY --from=nodebuilder /build/server /bin/server +RUN mkdir /db && ln -s /app/public/uploads /uploads +ENV KTISTEC_DB=/db/ktistec.db +EXPOSE 3000 +CMD ["/bin/server"] diff --git a/tools/pasture/peers/ktistec.sh b/tools/pasture/peers/ktistec.sh new file mode 100644 index 0000000..130309d --- /dev/null +++ b/tools/pasture/peers/ktistec.sh @@ -0,0 +1,38 @@ +# Ktistec 3.13.0 (Crystal, SQLite): a small ActivityPub server with its own JSON API. Built from its tag +# (images/ktistec), as ktistec.test; its static OpenSSL reads /etc/ssl/cert.pem, where the pasture's bundle goes. The +# server's name and its first account (ktuser) are given through its setup API, which takes them unauthenticated until +# there is one; a script signs in on /sessions, whose answer carries the bearer token both its APIs take (its own, and +# the part of Mastodon's it speaks). +KTISTEC_IMAGE=${KTISTEC_IMAGE:-localhost/pasture-ktistec:3.13.0} +KTISTEC_PASSWORD=Ktistec-Pasture-1 + +ktistec_up() { + podman image exists "$KTISTEC_IMAGE" || podman build -q -t "$KTISTEC_IMAGE" "$here/images/ktistec" >/dev/null + mkdir -p "$here/.state/ktistec" + podman volume exists pasture-ktistec || podman volume create --label pasture=1 pasture-ktistec >/dev/null + podman run -d --replace --name pasture-ktistec --label pasture=1 --network $net -v pasture-ktistec:/db \ + -v "$ca/bundle.pem:/etc/ssl/cert.pem:z,ro" -v "$ca/bundle.pem:/etc/ssl/certs/ca-certificates.crt:z,ro" "$KTISTEC_IMAGE" >/dev/null + for _ in $(seq 1 60); do + site ktistec.test -s -o /dev/null -w '%{http_code}' https://ktistec.test:6443/ 2>/dev/null | grep -qE '200|302' && break + sleep 1 + done + # its setup, once: the server's address and name, then its first account + kt -o /dev/null -X POST https://ktistec.test:6443/ -d '{"host":"https://ktistec.test","site":"Pasture Ktistec"}' + # (every field is required, summary too: a missing one sends the setup back to its first page) + kt -o /dev/null -X POST https://ktistec.test:6443/ -d "{\"username\":\"ktuser\",\"password\":\"$KTISTEC_PASSWORD\",\"name\":\"ktuser\",\"summary\":\"Ktistec in the pasture\",\"language\":\"en\",\"timezone\":\"UTC\",\"auto_approve_followers\":true}" + ktistec_token > "$here/.state/ktistec/ktuser.token" + # quotes approved without asking: with manual approval on, Ktistec's posts name only their author in canQuote and offer + # no manualApproval, so no other server may even ask. (Its settings are written whole: a flag left out is turned off.) + kt -o /dev/null -X POST https://ktistec.test:6443/settings/actor -H "Authorization: Bearer $(cat "$here/.state/ktistec/ktuser.token")" \ + -d '{"manually_approve_quotes":false,"auto_approve_followers":true}' + echo "ktistec: https://ktistec.test:6443" +} + +# kt : Ktistec's JSON API +kt() { site ktistec.test -s -H 'Content-Type: application/json' -H 'Accept: application/json' "$@"; } + +# ktistec_token [user]: the bearer token of a signed-in session (ktuser's) +ktistec_token() { + kt -X POST https://ktistec.test:6443/sessions -d "{\"username\":\"${1:-ktuser}\",\"password\":\"$KTISTEC_PASSWORD\"}" \ + | python3 -c 'import json, sys; d = json.load(sys.stdin); print(d.get("jwt") or d.get("token") or "")' 2>/dev/null +} diff --git a/tools/pasture/scenarios/ktistec.sh b/tools/pasture/scenarios/ktistec.sh new file mode 100644 index 0000000..72e40e4 --- /dev/null +++ b/tools/pasture/scenarios/ktistec.sh @@ -0,0 +1,125 @@ +# Ktistec 3.13.0: follows both ways; posts both ways; replies both ways; likes and boosts both ways with their undos; a +# poll from Ktistec and alice's vote counted there; FEP-044f quotes both ways; edits and deletions both ways; the +# unfollow; statistics. Ktistec speaks part of the Mastodon API (posts, replies, likes, boosts, timelines, +# relationships) with the bearer token of a signed-in session; what it has no route for there (follows, polls, edits, +# deletions, quotes) goes through its own outbox API, and what it holds (likes, boosts and votes included: its API counts +# none) is read from a copy of its SQLite database. +. "$here/peers/ktistec.sh" +K=https://ktistec.test:6443 +KT=$(cat "$here/.state/ktistec/ktuser.token" 2>/dev/null) +kc() { site ktistec.test -s -H "Authorization: Bearer $KT" "$@"; } +# kn : an activity put in ktuser's outbox (Ktistec's own API), answering its status code +kn() { kc -o /dev/null -w '%{http_code}' -X POST -H 'Content-Type: application/json' -H 'Accept: application/json' "$K/actors/ktuser/outbox" -d "$1"; } +# kt_sql : Ktistec's database, read from a copy +kt_sql() { + local d + d=$(mktemp -d) + podman cp pasture-ktistec:/db/ktistec.db "$d/" 2>/dev/null + python3 -c "import sqlite3, sys; print('\n'.join('|'.join(map(str, r)) for r in sqlite3.connect(sys.argv[1]).execute(sys.argv[2])))" "$d/ktistec.db" "$1" + rm -rf "$d" +} +k_home_id() { kc "$K/api/v1/timelines/home?limit=40" | j "print(next((o['id'] for o in ((s.get('reblog') or s) for s in d) if '$1' in (o['content'] or '')), ''))"; } +k_status() { kc "$K/api/v1/statuses/$1" | j "print(d.get('$2'))"; } +# k_by_alice : alice's likes and boosts of it that Ktistec holds (its API's counts stay at 0) +k_by_alice() { kt_sql "select count(*) from activities where type in ('ActivityPub::Activity::Like', 'ActivityPub::Activity::Announce') and actor_iri = '$alice_uri' and object_iri = '$1' and undone_at is null"; } +p_home_id() { curl -s -H "$PH" "$P/api/v1/timelines/home?limit=40" | j "print(next((o['id'] for o in ((s.get('reblog') or s) for s in d) if '$1' in (o['content'] or '')), ''))"; } +p_status() { curl -s -H "$PH" "$P/api/v1/statuses/$1" | j "print(d.get('$2'))"; } +p_replies_have() { [ "$(curl -s -H "$PH" "$P/api/v1/statuses/$1/context" | j "print(any('$2' in s['content'] for s in d['descendants']))")" = "True" ]; } + +echo "ktistec" +[ -n "$(kc "$K/api/v1/accounts/verify_credentials" | j "print(d.get('username') or '')")" ] && ok "Ktistec's API as ktuser" \ + || { ko "Ktistec's API ($(kc "$K/api/v1/accounts/verify_credentials" | head -c 200))"; return 1; } +PT=$(privapub_token alice_ktistec) +PH="Authorization: Bearer $PT" +[ -n "$PT" ] && ok "PrivaPub token for alice_ktistec" || { ko "PrivaPub token for alice_ktistec"; return 1; } +alice_uri="$(curl -s -H "$PH" "$P/api/v1/accounts/verify_credentials" | j "print(d['url'])" | sed 's|/@|/peasants/|')" +kt_uri=https://ktistec.test/actors/ktuser +run=$(date +%s) + +echo " follows" +kt_on_p=$(curl -s -H "$PH" "$P/api/v2/search?q=ktuser@ktistec.test&resolve=true&type=accounts" | j "print(d['accounts'][0]['id'])") +[ -n "$kt_on_p" ] && ok "PrivaPub resolves ktuser" || ko "PrivaPub cannot resolve ktuser" +alice_on_k=$(kc "$K/api/v1/accounts/lookup?acct=alice_ktistec@privapub.test" | j "print(d['id'])") +[ -n "$alice_on_k" ] && ok "Ktistec resolves alice" || ko "Ktistec cannot resolve alice" +# (a run before leaves both following: both follows below are made anew) +curl -s -o /dev/null -X POST -H "$PH" "$P/api/v1/accounts/$kt_on_p/unfollow" +kc -o /dev/null -X POST "$K/api/v1/accounts/$alice_on_k/unfollow" +until_true 30 '[ "$(curl -s -H "$PH" "$P/api/v1/accounts/relationships?id[]=$kt_on_p" | j "print(d[0][\"followed_by\"])")" = "False" ]' >/dev/null +kn "{\"type\":\"Follow\",\"object\":\"$alice_uri\"}" >/dev/null +until_true 60 '[ "$(curl -s -H "$PH" "$P/api/v1/accounts/relationships?id[]=$kt_on_p" | j "print(d[0][\"followed_by\"])")" = "True" ]' \ + && ok "ktuser follows alice" || ko "Ktistec's follow never reached alice" +curl -s -o /dev/null -X POST -H "$PH" "$P/api/v1/accounts/$kt_on_p/follow" +until_true 60 '[ "$(curl -s -H "$PH" "$P/api/v1/accounts/relationships?id[]=$kt_on_p" | j "print(d[0][\"following\"])")" = "True" ]' \ + && ok "alice follows ktuser (Accept arrived)" || ko "Ktistec's Accept never arrived" + +echo " posts" +k_post=$(kc -X POST "$K/api/v1/statuses" -d "status=a Ktistec post $run&visibility=public" | j "print(d['id'])") +until_true 60 '[ -n "$(p_home_id "a Ktistec post $run")" ]' && ok "ktuser's post reaches alice's home" || ko "ktuser's post never reached alice" +k_post_on_p=$(p_home_id "a Ktistec post $run") +k_post_uri=$(p_status "$k_post_on_p" uri) +p_post=$(curl -s -X POST -H "$PH" "$P/api/v1/statuses" -d "status=a PrivaPub post for Ktistec $run&visibility=public") +p_post_id=$(echo "$p_post" | j "print(d['id'])"); p_post_uri=$(echo "$p_post" | j "print(d['uri'])") +until_true 60 '[ -n "$(k_home_id "a PrivaPub post for Ktistec $run")" ]' && ok "alice's post reaches ktuser's home" || ko "alice's post never reached Ktistec" +p_post_on_k=$(k_home_id "a PrivaPub post for Ktistec $run") + +echo " replies" +curl -s -o /dev/null -X POST -H "$PH" "$P/api/v1/statuses" -d "status=@ktuser@ktistec.test a PrivaPub reply $run&in_reply_to_id=$k_post_on_p&visibility=public" +until_true 60 'kc "$K/api/v1/statuses/$k_post/context" | grep -q "a PrivaPub reply $run"' \ + && ok "alice's reply threads under ktuser's post" || ko "alice's reply never reached Ktistec" +kc -o /dev/null -X POST "$K/api/v1/statuses" -d "status=@alice_ktistec@privapub.test a Ktistec reply $run&in_reply_to_id=$p_post_on_k&visibility=public" +until_true 60 'p_replies_have "$p_post_id" "a Ktistec reply $run"' && ok "ktuser's reply threads under alice's post" || ko "ktuser's reply missing on PrivaPub" + +echo " likes and boosts" +curl -s -o /dev/null -X POST -H "$PH" "$P/api/v1/statuses/$k_post_on_p/favourite" +curl -s -o /dev/null -X POST -H "$PH" "$P/api/v1/statuses/$k_post_on_p/reblog" +until_true 60 '[ "$(k_by_alice "$k_post_uri")" = "2" ]' && ok "alice's like and boost land on Ktistec" || ko "alice's like or boost never reached Ktistec" +kc -o /dev/null -X POST "$K/api/v1/statuses/$p_post_on_k/favourite" +kc -o /dev/null -X POST "$K/api/v1/statuses/$p_post_on_k/reblog" +until_true 60 '[ "$(p_status "$p_post_id" favourites_count) $(p_status "$p_post_id" reblogs_count)" = "1 1" ]' \ + && ok "ktuser's like and boost count on PrivaPub" || ko "ktuser's like or boost not counted on PrivaPub" +curl -s -o /dev/null -X POST -H "$PH" "$P/api/v1/statuses/$k_post_on_p/unfavourite" +curl -s -o /dev/null -X POST -H "$PH" "$P/api/v1/statuses/$k_post_on_p/unreblog" +until_true 60 '[ "$(k_by_alice "$k_post_uri")" = "0" ]' && ok "alice's undos reach Ktistec" || ko "alice's undos not applied on Ktistec" +kc -o /dev/null -X POST "$K/api/v1/statuses/$p_post_on_k/unfavourite" +kc -o /dev/null -X POST "$K/api/v1/statuses/$p_post_on_k/unreblog" +until_true 60 '[ "$(p_status "$p_post_id" favourites_count) $(p_status "$p_post_id" reblogs_count)" = "0 0" ]' \ + && ok "ktuser's undos reach PrivaPub" || ko "ktuser's undos not applied on PrivaPub" + +echo " polls" +kn "{\"type\":\"Publish\",\"content\":\"tea or coffee, Ktistec asks $run\",\"poll-options\":[\"tea\",\"coffee\"],\"poll-duration\":\"3600\",\"visibility\":\"public\"}" >/dev/null +until_true 60 '[ "$(curl -s -H "$PH" "$P/api/v1/timelines/home?limit=40" | j "print(next((len(s[\"poll\"][\"options\"]) for s in d if s[\"poll\"] and \"Ktistec asks $run\" in s[\"content\"]), 0))")" = "2" ]' \ + && ok "ktuser's poll reaches alice as a poll" || ko "ktuser's poll never reached alice as one" +k_poll_on_p=$(curl -s -H "$PH" "$P/api/v1/timelines/home?limit=40" | j "print(next((s['poll']['id'] for s in d if s['poll'] and 'Ktistec asks $run' in s['content']), ''))") +curl -s -o /dev/null -X POST -H "$PH" "$P/api/v1/polls/$k_poll_on_p/votes" -d 'choices[]=1' +# (Ktistec counts a poll of its own from the votes it keeps, which its API does not show) +until_true 60 '[ "$(kt_sql "select v.name from objects v join objects q on q.iri = v.in_reply_to_iri where v.special = '"'vote'"' and v.attributed_to_iri = '"'$alice_uri'"' and q.content like '"'%Ktistec asks $run%'"'")" = "coffee" ]' \ + && ok "alice's vote counts on Ktistec" || ko "alice's vote never counted on Ktistec" + +echo " quotes" +a_quote=$(curl -s -X POST -H "$PH" "$P/api/v1/statuses" -d "status=quoting Ktistec $run&visibility=public"ed_status_id=$k_post_on_p" | j "print(d['id'])") +until_true 60 '[ "$(curl -s -H "$PH" "$P/api/v1/statuses/$a_quote" | j "print((d.get(\"quote\") or {}).get(\"state\"))")" = "accepted" ]' \ + && ok "Ktistec approves alice's quote (FEP-044f)" || ko "alice's quote of a Ktistec post not approved ($(curl -s -H "$PH" "$P/api/v1/statuses/$a_quote" | j "print(d.get('quote'))"))" +kc -o /dev/null -X POST -H 'Content-Type: application/json' -H 'Accept: application/json' "$K/remote/objects/$p_post_on_k/quote" \ + -d "{\"content\":\"Ktistec quotes PrivaPub $run\",\"quote\":\"$p_post_uri\",\"visibility\":\"public\"}" +until_true 60 '[ "$(curl -s -H "$PH" "$P/api/v1/timelines/home?limit=40" | j "print(next((((s.get(\"quote\") or {}).get(\"quoted_status\") or {}).get(\"id\") for s in d if \"Ktistec quotes PrivaPub $run\" in s[\"content\"]), None))")" = "$p_post_id" ]' \ + && ok "PrivaPub approves ktuser's quote, which reaches alice quoting her post" || ko "ktuser's quote of alice's post missing or unapproved" + +echo " edits and deletions" +kn "{\"type\":\"Publish\",\"object\":\"$k_post_uri\",\"content\":\"a Ktistec post $run, edited\",\"visibility\":\"public\"}" >/dev/null +until_true 60 'p_status "$k_post_on_p" content | grep -q "edited"' && ok "ktuser's edit reaches PrivaPub" || ko "ktuser's edit never reached PrivaPub" +curl -s -o /dev/null -X PUT -H "$PH" "$P/api/v1/statuses/$p_post_id" -d "status=a PrivaPub post for Ktistec $run, edited" +until_true 60 'k_status "$p_post_on_k" content | grep -q "edited"' && ok "alice's edit reaches Ktistec" || ko "alice's edit never reached Ktistec" +kn "{\"type\":\"Delete\",\"object\":\"$k_post_uri\"}" >/dev/null +until_true 60 '[ "$(curl -s -o /dev/null -w "%{http_code}" -H "$PH" "$P/api/v1/statuses/$k_post_on_p")" = "404" ]' \ + && ok "ktuser's deletion reaches PrivaPub" || ko "ktuser's deleted post still on PrivaPub" +curl -s -o /dev/null -X DELETE -H "$PH" "$P/api/v1/statuses/$p_post_id" +until_true 60 '[ "$(kt_sql "select count(*) from objects where iri = '"'$p_post_uri'"' and deleted_at is null")" = "0" ]' \ + && ok "alice's deletion reaches Ktistec" || ko "alice's deleted post still on Ktistec" + +echo " unfollow" +curl -s -o /dev/null -X POST -H "$PH" "$P/api/v1/accounts/$kt_on_p/unfollow" +until_true 60 '[ "$(kt_sql "select count(*) from relationships where type = '"'Relationship::Social::Follow'"' and from_iri = '"'$alice_uri'"' and to_iri = '"'$kt_uri'"'")" = "0" ]' \ + && ok "alice's unfollow reaches Ktistec" || ko "Ktistec still counts alice following ktuser" + +echo " statistics" +stats_check ktistec.test ktistec