From aa7e43a61e79ff9216540acd94637c299431047b Mon Sep 17 00:00:00 2001 From: thepra Date: Sat, 3 Oct 2026 14:24:15 +0200 Subject: [PATCH] T14: Lemmy 1.0 in the pasture peers/lemmy.sh runs the Lemmy 1.0.0-beta.2 backend on the shared Postgres. It trusts Caddy's CA through SSL_CERT_FILE and reaches the pasture through DANGER_FEDERATION_ALLOW_LOCAL_IP. Lemmy 0.19 cannot join: its rustls trusts only its bundled roots. LEMMY_LOG sets RUST_LOG. scenarios/lemmy.sh drives Lemmy through its v4 API. 20 checks pass, three runs in a row: - communities both ways; - a titled thread each way, and alice's mention of a Lemmy community becoming a thread there; - the Lemmy community's announce reaching alice's home; - comments both ways and alice's like as an upvote; - private messages both ways; - statistics. Two expected failures: votes, which Lemmy sends only through the community as Announce{Like|Dislike}, and a moderator's removal. Both belong to P7. What it showed, in docs/INTEROP.md: - Lemmy 1.0 keeps its user's thread in a remote community pending until the community announces it back. It clears the flag before answering that echo 400 ("Object is not remote"). Leaving the author's server out of the Announce, as tried here, left every such thread pending, so GroupDistributor now says why the echo stays. - Every bare Announce{object} is answered 400, as Lemmy answers its own compatibility Announce(Page). Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01ELjqpznMFMNrJoJUj6K5p2 --- CLAUDE.md | 7 ++ .../Federation/Outbox/GroupDistributor.cs | 2 + docs/INTEROP.md | 27 ++++- tools/pasture/interop.sh | 2 +- tools/pasture/peers/lemmy.sh | 36 ++++++ tools/pasture/run.sh | 2 +- tools/pasture/scenarios/lemmy.sh | 109 ++++++++++++++++++ 7 files changed, 182 insertions(+), 3 deletions(-) create mode 100644 tools/pasture/peers/lemmy.sh create mode 100644 tools/pasture/scenarios/lemmy.sh diff --git a/CLAUDE.md b/CLAUDE.md index d866d53..3cbc629 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -414,6 +414,13 @@ tools/pasture/run.sh down # removes e (`federation: none`) until `admin/update-meta` says `all`, which `misskey_up` does. Its API is `POST /api/` with the token as `i` (`mk` in the scenario); `users/relation` answers a list, `users/notes` leaves replies out unless asked, and a user has one reaction per note. Never verify a delivery with `ap/show`: it fetches. 35 checks. +- **Lemmy (1.0.0-beta.2):** the backend alone on the shared Postgres, its admin made by `setup` in the generated + `config.hjson`. It trusts Caddy's CA through `SSL_CERT_FILE` and reaches the network through + `DANGER_FEDERATION_ALLOW_LOCAL_IP=1`; 0.19 cannot join (its rustls trusts only its bundled roots). Its API is + `/api/v4/` with a bearer token (`lm` in the scenario), and `sort` values are lowercase. It logs no refused + activity at `warn` (`LEMMY_LOG` sets `RUST_LOG`); the reason is in the 400's body. It answers our community's echo of + its own activity and every bare `Announce{object}` 400 by design, and the echo is still needed (see + `docs/INTEROP.md`, Lemmy). 20 checks; relayed votes and a moderator's removal are expected failures (P7). - **Crawler:** `PRIVAPUB_ENV="Statistics__Crawler__Enabled=true Statistics__Crawler__Seeds__0=mastodon.test" run.sh up mastodon`, then `interop.sh crawler`. `PRIVAPUB_ENV` passes any setting to the PrivaPub container. - `run.sh up` replaces every container, Mongo included, so each run starts clean. To keep the data, republish into diff --git a/PrivaPub/Federation/Outbox/GroupDistributor.cs b/PrivaPub/Federation/Outbox/GroupDistributor.cs index 01dd7c6..caf9f76 100644 --- a/PrivaPub/Federation/Outbox/GroupDistributor.cs +++ b/PrivaPub/Federation/Outbox/GroupDistributor.cs @@ -40,6 +40,8 @@ namespace PrivaPub.Federation.Outbox ["audience"] = group.Uri, ["object"] = embedded }; + // Every follower, the author's own server included: Lemmy 1.0 keeps its user's post pending until the community + // announces it back, and clears that before it answers the echo 400 ("Object is not remote"). await _delivery.EnqueueToFollowers(group, announce, token); if (isNewPost && !string.IsNullOrEmpty(objectUri)) await _delivery.EnqueueToFollowers(group, ActivityPubRenderer.Announce(group, objectUri, $"announce-object-{key}"), token); diff --git a/docs/INTEROP.md b/docs/INTEROP.md index 3e2c37f..68b3fab 100644 --- a/docs/INTEROP.md +++ b/docs/INTEROP.md @@ -427,7 +427,7 @@ on a Page: pins live in `featured`, locks in `Lock`. | Link posts: keep `Link.href`, the thumbnail `image` and alt text; build the card (Lemmy sends no title or description for the link) | P1 | `Status.card` | | `ChatMessage` in and out (out only to Lemmy < 1.0 and Mbin; `Note` to everyone else) | P1 | `visibility: direct` | | Outbound shape for Lemmy: both `to` and `cc`; the community in `to`; Public in the object, Create and Announce; votes and comments sent to the community inbox | P1 | — | -| Communities we host: announce to every follower instance including the author's; pick `Announce(object)` per peer by NodeInfo (as PieFed does) | P1 | — | +| Communities we host: pick `Announce(object)` per peer by NodeInfo (as PieFed does). Announcing to every follower instance, the author's included, is done and needed (pasture evidence below) | P1 | — | | Flags from a `Service`-typed reporter actor with `to: [community]`; the reporter stays anonymous | P2 | — | | `Warn` → `moderation_warning` notification; `Resolve{Flag}` | P2 | AccountWarning | | Remote communities: `description`, `language[]`, private (`locked`), `discoverable`; post tags | P2 | `Account.locked`, own `privapub.flairs[]` | @@ -435,6 +435,31 @@ on a Page: pins live in `featured`, locks in `Lock`. | `Feed` actors | P2 | group-like account | | Read 1.0 `context`, grouped by root post; cross-post detection by URL | P3 | — | +**Pasture evidence (2026-10-03, Lemmy 1.0.0-beta.2, `tools/pasture/scenarios/lemmy.sh`):** 20 checks pass and 3 are +expected failures: +- communities both ways: Lemmy follows ours and alice follows Lemmy's, each Accept arriving; +- a Lemmy thread in our community arrives with its title, and our titled community post reaches Lemmy; +- the Lemmy community's Announce brings its thread to alice's home; +- alice's post mentioning a Lemmy community lands in it, titled from its first line (Lemmy repeats that line in the + body); +- comments both ways, and alice's like counted as an upvote; +- private messages both ways: 1.0 takes our single-recipient direct `Note`, and sends its own as `Note`s; +- statistics. + +What it showed: +- **1.0 keeps its user's thread in a remote community `federation_pending` until the community announces it back**, and + clears the flag *before* answering that echo 400 (`Object is not remote`). Without the echo the thread stays pending, + so `GroupDistributor` sends the `Announce{Create}` to the author's own server on purpose. +- **Every bare `Announce{object}` is answered 400** (`Failed to parse object`: Lemmy dereferences it expecting an + activity), as Lemmy answers the compatibility `Announce(Page)` it sends itself. Both 400s show up as dead deliveries + in the statistics. +- **Votes travel only to the community**, which relays them as `Announce{Like}` and `Announce{Dislike}`. They are + dropped as `unsupported` until P7 (expected failures), as is a moderator's removal. +- Lemmy logs no refused activity at `warn`; the reason is in the 400's body, which our delivery does not keep. The + scenario's API notes: `sort` values are lowercase (`new`), private messages and mentions are in + `account/notification/list`, and `resolve_object` takes both `!community@host` and `@user@host`. +- 0.19 cannot join the pasture: its rustls trusts only its bundled roots, never Caddy's CA. + ### PieFed 1.7.17 and Mbin 1.10.1 - **PieFed sends Lemmy's set plus:** diff --git a/tools/pasture/interop.sh b/tools/pasture/interop.sh index 20a269a..49d9245 100755 --- a/tools/pasture/interop.sh +++ b/tools/pasture/interop.sh @@ -1,7 +1,7 @@ #!/usr/bin/env bash # Drives PrivaPub and each peer (started by run.sh) through each other's federation, as their users would, then checks # that PrivaPub's statistics saw it all without naming anyone. -# usage: tools/pasture/interop.sh [peer...] peers: gts (default) +# usage: tools/pasture/interop.sh [peer...] peers: gts (default), mastodon, misskey, lemmy, crawler set -uo pipefail here="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" . "$here/lib/interop.sh" diff --git a/tools/pasture/peers/lemmy.sh b/tools/pasture/peers/lemmy.sh new file mode 100644 index 0000000..8662200 --- /dev/null +++ b/tools/pasture/peers/lemmy.sh @@ -0,0 +1,36 @@ +# Lemmy 1.0 (beta): the backend alone (no UI, no pict-rs) on the shared Postgres. 0.19's rustls ignores every CA but its +# bundled ones, so only 1.0 can trust Caddy's; DANGER_FEDERATION_ALLOW_LOCAL_IP lets it reach the pasture's addresses. +LEMMY_IMAGE=${LEMMY_IMAGE:-docker.io/dessalines/lemmy:1.0.0-beta.2} +. "$here/peers/shared.sh" + +lemmy_up() { + shared_postgres_up + podman exec pasture-postgres sh -c "psql -U pasture -tc \"select 1 from pg_database where datname='lemmy'\" | grep -q 1 || createdb -U pasture lemmy" + mkdir -p "$here/.state/lemmy" + cat > "$here/.state/lemmy/config.hjson" </dev/null + for _ in $(seq 1 90); do + site lemmy.test -s -o /dev/null -w '%{http_code}' https://lemmy.test:6443/api/v4/site 2>/dev/null | grep -q 200 && break + sleep 2 + done + site lemmy.test -s -X POST https://lemmy.test:6443/api/v4/account/auth/login -H 'Content-Type: application/json' \ + -d '{"username_or_email":"lemmyuser","password":"Lemmy-Pasture-Pass-1"}' \ + | python3 -c "import sys,json; print(json.load(sys.stdin)['jwt'])" > "$here/.state/lemmy.token" 2>/dev/null || true + echo "lemmy: https://lemmy.test:6443" +} diff --git a/tools/pasture/run.sh b/tools/pasture/run.sh index df00a35..80d0c08 100755 --- a/tools/pasture/run.sh +++ b/tools/pasture/run.sh @@ -2,7 +2,7 @@ # A private test fediverse on one podman network: PrivaPub (privapub.test) and the peers asked for, behind one Caddy # whose internal CA every side is told to accept (its root is copied to .ca/root.crt). From the workstation: PrivaPub's # API at http://127.0.0.1:6971, every site at https://.test:6443 (curl --resolve .test:6443:127.0.0.1 -k). -# usage: tools/pasture/run.sh up [peer...] | down | logs | ps peers: gts (default) +# usage: tools/pasture/run.sh up [peer...] | down | logs | ps peers: gts (default), mastodon, misskey, lemmy set -euo pipefail . "$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)/lib/pasture.sh" diff --git a/tools/pasture/scenarios/lemmy.sh b/tools/pasture/scenarios/lemmy.sh new file mode 100644 index 0000000..aae0900 --- /dev/null +++ b/tools/pasture/scenarios/lemmy.sh @@ -0,0 +1,109 @@ +# Lemmy 1.0: communities both ways (FEP-1b12), threads with titles, comments both ways, votes up and down both ways, +# private messages both ways, a moderator's removal (P7), statistics. Lemmy is driven through its v4 API. +LM=https://lemmy.test:6443 +LT=$(cat "$here/.state/lemmy.token" 2>/dev/null) +# lm [json]: a Lemmy v4 API call as lemmyuser +lm() { + local method=$1 path=$2 body=${3:-} + if [ -n "$body" ]; then + site lemmy.test -s -X "$method" "$LM/api/v4/$path" -H "Authorization: Bearer $LT" -H 'Content-Type: application/json' -d "$body" + else + site lemmy.test -s -X "$method" "$LM/api/v4/$path" -H "Authorization: Bearer $LT" + fi +} +# the posts Lemmy holds in a community, and the one whose ap_id is given +lm_posts() { lm GET "post/list?community_id=$1&sort=new&limit=50" | j "print(json.dumps([p['post'] for p in (d.get('items') or d.get('posts') or [])]))"; } +lm_post_by_ap() { lm_posts "$1" | j "print(json.dumps(next((p for p in d if p['ap_id']=='$2'), None)))"; } + +echo "lemmy" +[ -n "$LT" ] && ok "Lemmy token for lemmyuser" || { ko "Lemmy token"; return 1; } +LAT=$(privapub_token alice_lemmy) +LAH="Authorization: Bearer $LAT" +[ -n "$LAT" ] && ok "PrivaPub token for alice_lemmy" || { ko "PrivaPub token for alice_lemmy"; return 1; } +jwt=$(privapub_root) +alice_id=$(curl -s -H "$LAH" "$P/api/v1/accounts/verify_credentials" | j "print(d['id'])") +dogs="dogs$(date +%s)" +cats="cats$(date +%s)" + +echo " communities" +dogs_gid=$(curl -s -X POST $P/clientapi/group/insert -H 'Content-Type: application/json' -H "Authorization: Bearer $jwt" \ + -d "{\"avatarId\":\"$alice_id\",\"userName\":\"$dogs\",\"name\":\"Pasture dogs\",\"description\":\"a PrivaPub community\",\"isCommunity\":true}" | j "print(d['id'])") +dogs_on_lm=$(lm GET "resolve_object?q=!$dogs@privapub.test" | j "print(d['community']['id'])") +[ -n "$dogs_on_lm" ] && ok "Lemmy resolves a PrivaPub community" || ko "Lemmy cannot resolve the PrivaPub community" +lm POST community/follow "{\"community_id\":$dogs_on_lm,\"follow\":true}" >/dev/null +until_true 30 '[ "$(lm GET "community?id=$dogs_on_lm" | j "print(((d[\"community_view\"].get(\"community_actions\") or {}).get(\"follow_state\") or \"\").lower())")" = "accepted" ]' \ + && ok "lemmyuser follows the PrivaPub community (Accept arrived)" || ko "Lemmy's community follow not accepted" +cats_on_lm=$(lm POST community "{\"name\":\"$cats\",\"title\":\"Pasture cats\"}" | j "print(d['community_view']['community']['id'])") +cats_on_p=$(curl -s -H "$LAH" "$P/api/v2/search?q=@$cats@lemmy.test&resolve=true&type=accounts" | j "print(d['accounts'][0]['id'])") +[ -n "$cats_on_p" ] && ok "PrivaPub resolves a Lemmy community" || ko "PrivaPub cannot resolve the Lemmy community" +curl -s -o /dev/null -X POST -H "$LAH" "$P/api/v1/accounts/$cats_on_p/follow" +until_true 30 '[ "$(curl -s -H "$LAH" "$P/api/v1/accounts/relationships?id[]=$cats_on_p" | j "print(d[0][\"following\"])")" = "True" ]' && ok "alice follows the Lemmy community (Accept arrived)" || ko "PrivaPub's community follow not accepted" + +echo " threads" +lm_thread=$(lm POST post "{\"name\":\"A Lemmy thread\",\"body\":\"posted from Lemmy into PrivaPub\",\"community_id\":$dogs_on_lm}" | j "print(d['post_view']['post']['ap_id'])") +dogs_on_p=$(curl -s -H "$LAH" "$P/api/v1/accounts/lookup?acct=$dogs" | j "print(d['id'])") +until_true 30 '[ "$(curl -s -H "$LAH" "$P/api/v1/accounts/$dogs_on_p/statuses" | j "print(any(s[\"uri\"]==\"$lm_thread\" and (s.get(\"privapub\") or {}).get(\"title\")==\"A Lemmy thread\" for s in d))")" = "True" ]' \ + && ok "a Lemmy thread in the PrivaPub community arrives with its title" || ko "Lemmy's thread missing or untitled on PrivaPub" +curl -s -o /dev/null -X POST $P/clientapi/post/insert -H 'Content-Type: application/json' -H "Authorization: Bearer $jwt" \ + -d "{\"avatarId\":\"$alice_id\",\"title\":\"A PrivaPub community thread\",\"text\":\"posted into our own community\",\"groupId\":\"$dogs_gid\"}" +dogs_post=$(podman exec pasture-mongo mongosh --quiet PrivaPub --eval 'print(db.Post.findOne({Text:/posted into our own community/}, {}, {sort:{_id:-1}}).ObjectURI)') +until_true 30 '[ "$(lm_post_by_ap "$dogs_on_lm" "$dogs_post" | j "print(d and d[\"name\"])")" = "A PrivaPub community thread" ]' \ + && ok "a titled post in the PrivaPub community reaches Lemmy through its announce" || ko "PrivaPub community post missing on Lemmy" +# Lemmy 1.0 holds its user's thread in a remote community as pending until the community announces it back, and clears +# the flag even though it then answers that Announce 400 ("Object is not remote"): the echo is what publishes it. +until_true 30 '[ "$(lm_post_by_ap "$dogs_on_lm" "$lm_thread" | j "print(d and d[\"federation_pending\"])")" = "False" ]' \ + && ok "the PrivaPub community's announce back publishes Lemmy's thread there" || ko "Lemmy's thread stays federation_pending" +lm_cats_thread=$(lm POST post "{\"name\":\"Cats only\",\"body\":\"a Lemmy community post\",\"community_id\":$cats_on_lm}" | j "print(d['post_view']['post']['ap_id'])") +until_true 30 'curl -s -H "$LAH" "$P/api/v1/timelines/home" | j "print(any(s[\"uri\"]==\"$lm_cats_thread\" or (s.get(\"reblog\") or {}).get(\"uri\")==\"$lm_cats_thread\" for s in d))" | grep -q True' \ + && ok "the Lemmy community's announce brings its thread to alice's home" || ko "Lemmy community thread missing from alice's home" +p_thread=$(curl -s -X POST -H "$LAH" $P/api/v1/statuses --data-urlencode "status=A PrivaPub thread +@$cats@lemmy.test posted from PrivaPub" -d 'visibility=public') +p_thread_id=$(echo "$p_thread" | j "print(d['id'])"); p_thread_uri=$(echo "$p_thread" | j "print(d['uri'])") +until_true 30 '[ "$(lm_post_by_ap "$cats_on_lm" "$p_thread_uri")" != "null" ]' && ok "alice's thread lands in the Lemmy community" || ko "PrivaPub thread missing from the Lemmy community" +p_thread_on_lm=$(lm_post_by_ap "$cats_on_lm" "$p_thread_uri" | j "print(d['id'])") + +echo " comments" +lm POST comment "{\"post_id\":$p_thread_on_lm,\"content\":\"a Lemmy comment on PrivaPub\"}" >/dev/null +until_true 30 '[ "$(curl -s -H "$LAH" "$P/api/v1/statuses/$p_thread_id/context" | j "print(any(\"a Lemmy comment on PrivaPub\" in s[\"content\"] for s in d[\"descendants\"]))")" = "True" ]' \ + && ok "Lemmy's comment threads under alice's post" || ko "Lemmy's comment missing on PrivaPub" +lm_cats_on_p=$(curl -s -H "$LAH" "$P/api/v1/timelines/home" | j "print(next((s.get('reblog') or s)['id'] for s in d if (s.get('reblog') or s)['uri']=='$lm_cats_thread'))") +curl -s -o /dev/null -X POST -H "$LAH" $P/api/v1/statuses -d "status=a PrivaPub comment on Lemmy&in_reply_to_id=$lm_cats_on_p&visibility=public" +lm_cats_id=$(lm GET "resolve_object?q=$lm_cats_thread" | j "print(d['post']['id'])") +until_true 30 '[ "$(lm GET "comment/list?post_id=$lm_cats_id" | j "print(any(\"a PrivaPub comment on Lemmy\" in c[\"comment\"][\"content\"] for c in (d.get(\"items\") or d.get(\"comments\") or [])))")" = "True" ]' \ + && ok "alice's reply becomes a comment on Lemmy" || ko "PrivaPub reply missing as a Lemmy comment" + +echo " votes" +# Lemmy sends a vote only to the community, which relays it inside an Announce: P7 trusts those for the community's own posts. +lm POST post/like "{\"post_id\":$p_thread_on_lm,\"is_upvote\":true}" >/dev/null +if until_true 15 '[ "$(curl -s -H "$LAH" "$P/api/v1/statuses/$p_thread_id" | j "print(d[\"favourites_count\"])")" = "1" ]'; then + ok "Lemmy's upvote counts as a like on PrivaPub" +else + xf "Lemmy's upvote counts as a like on PrivaPub (votes relayed by a community are P7)" +fi +lm POST post/like "{\"post_id\":$p_thread_on_lm,\"is_upvote\":false}" >/dev/null +if until_true 15 '[ "$(curl -s -H "$LAH" "$P/api/v1/statuses/$p_thread_id" | j "print(d[\"favourites_count\"], ((d.get(\"privapub\") or {}).get(\"votes\") or {}).get(\"down\"))")" = "0 1" ]'; then + ok "Lemmy's change to a downvote reaches PrivaPub" +else + xf "Lemmy's change to a downvote reaches PrivaPub (votes relayed by a community are P7)" +fi +curl -s -o /dev/null -X POST -H "$LAH" "$P/api/v1/statuses/$lm_cats_on_p/favourite" +until_true 30 '[ "$(lm GET "post?id=$lm_cats_id" | j "print(d[\"post_view\"][\"post\"].get(\"upvotes\"))")" = "2" ]' && ok "alice's like is an upvote on Lemmy" || ko "like not counted as an upvote on Lemmy" + +echo " private messages" +alice_on_lm=$(lm GET "resolve_object?q=@alice_lemmy@privapub.test" | j "print(d['person']['id'])") +lm POST private_message "{\"content\":\"a secret from Lemmy\",\"recipient_id\":$alice_on_lm}" >/dev/null +until_true 30 'curl -s -H "$LAH" "$P/api/v1/conversations" | grep -q "a secret from Lemmy"' && ok "Lemmy's private message arrives as a DM" || ko "Lemmy's private message missing on PrivaPub" +curl -s -o /dev/null -X POST -H "$LAH" $P/api/v1/statuses -d 'status=@lemmyuser@lemmy.test a secret from PrivaPub&visibility=direct' +until_true 30 'lm GET "account/notification/list?limit=50" | grep -q "a secret from PrivaPub"' && ok "alice's DM arrives as a Lemmy private message" || ko "DM missing on Lemmy" + +echo " moderation" +lm POST post/remove "{\"post_id\":$lm_cats_id,\"removed\":true,\"reason\":\"pasture\"}" >/dev/null +sleep 10 +if curl -s -H "$LAH" "$P/api/v1/timelines/home" | j "print(any((s.get('reblog') or s)['uri']=='$lm_cats_thread' for s in d))" | grep -q False; then + ok "a moderator's removal reaches PrivaPub" +else + xf "a moderator's removal reaches PrivaPub (the Lemmy moderation set is P7)" +fi + +echo " statistics" +stats_check lemmy.test lemmy