diff --git a/CLAUDE.md b/CLAUDE.md index 53a71e7..f48b557 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -534,6 +534,17 @@ tools/pasture/run.sh down # removes e its CA bundle (`wp-includes/certificates/ca-bundle.crt`) given Caddy's root, and WP-Cron run every five seconds by a sidecar (`DISABLE_WP_CRON`), since the plugin federates from it. Authors are actors; the REST API takes application passwords. `scenarios/wordpress.sh`, 17 checks. +- **Friendica (2026.05):** the official image on the shared MySQL (database friendica) and Redis (db 7, for its cache, + locks and sessions), installed by its autoinstall, with the image's worker daemon (`cron.sh`) as a sidecar sharing + its files. `friendica_settle` repairs what the install leaves: it names the system user (uid 0), or Friendica never + makes the system account that signs its fetches; it declares the daemon running (`worker_daemon_mode` in + `key-value`), or the web container, which cannot see the sidecar's pid, never wakes it and everything waits for its + five-minute cron; it makes the log file and turns logging on. `friendica_user` saves each account through the API + with `locked=0` (a number: "true" reads as 0), which makes it a "soapbox" page that takes followers without asking and + follows nobody back, and reads its outbox once: a Follow that reaches a user Friendica has not cached yet makes it + fetch that user from itself, signed, and checking the signature recurses for about five minutes. Its API takes HTTP + Basic (`nick:password`); an edit through it never federates, so the scenario edits in the web editor. + `scenarios/friendica.sh`, 25 checks; the town's driver and `specs/friendica-pair.json`. - **SecureMode:** `PRIVAPUB_ENV="Federation__SecureMode=true" run.sh up …`, as production runs. A check of what an unsigned reader sees uses `unserved` and `gone_unsigned` (`lib/interop.sh`), which expect 401 when SecureMode is on and 404 or 410 when it is off. @@ -560,12 +571,13 @@ tools/pasture/town.sh selftest [peer...] # each driver against its own server; tools/pasture/town.sh seed village # specs/village.json: 23 accounts on seven servers, about 11 minutes tools/pasture/town.sh check village # out//results.json; --deadline=0 re-sweeps an old run at once tools/pasture/town.sh report village # out//report.html, publishable (fake data, no tokens) -tools/pasture/town.sh report village hollo-pair iceshrimp-pair pleroma-pair # several runs in one matrix +tools/pasture/town.sh report village hollo-pair iceshrimp-pair pleroma-pair pixelfed-pair friendica-pair # one matrix tools/pasture/town.sh backlog --write # docs/INTEROP-BACKLOG.md ``` - **Drivers** (`dialects/`): one class per platform on a dialect base (`mastodon_api`, `misskey_api`, `lemmy_api`, - `privapub`). `stored()` reads the database (shared Postgres through `psql`, GoToSocial's sqlite copied out, Mongo); + `privapub`). `stored()` reads the database (shared Postgres through `psql`, the shared MySQL through `mysql_json`, + GoToSocial's sqlite copied out, Mongo); `seen()` asks the API as one account. A driver that cannot do something raises `Unsupported`, and the planner never asks for it (`gen.CAPS`). - **The plan** (`gen.py`) is a pure function of the spec and its seed; each run keeps its own `plan.json`, `ledger.jsonl` diff --git a/FEDERATION.md b/FEDERATION.md index 1c359ce..6e2ae7f 100644 --- a/FEDERATION.md +++ b/FEDERATION.md @@ -25,6 +25,7 @@ and every run starting clean, with signed fetches required (as privapub.thepra.d - **PeerTube 8.3.1** - **Pixelfed 0.14.4** - **WordPress 6 with ActivityPub 9.3.1** +- **Friendica 2026.05** - in the town only (a seeded community checked server by server): **Hollo 0.9.19**, **Iceshrimp.NET 2026.1.2-beta**, **Pleroma 2.10.2** diff --git a/docs/INTEROP.md b/docs/INTEROP.md index 9aba335..098e86d 100644 --- a/docs/INTEROP.md +++ b/docs/INTEROP.md @@ -592,6 +592,34 @@ What it showed: - `instrument{Service}` names the software. - It sends **`Follow` with a post as the object**, meaning "include me in this thread". Answer that with `Reject` or ignore it, without an error. + - It **forwards every activity in its threads** (comments and their deletions, other servers' included) to the + thread's followers, signed with the thread owner's key. PrivaPub answered them 401 until 2026-10-05; it now takes a + forwarded Create or Update as the object reads at its origin, and a Delete once the origin says it is gone + (`Forwarded`). + - An edit made through its Mastodon API never federates: `Statuses::put` leaves `edited` alone, and only a changed + `edited` notifies. Its web editor's edits do. + - Its own accounts' actors are cached only once something reads them. A Follow that reaches one first makes it fetch + the account from itself, signed as that account, and checking that signature builds the actor again: the request + recurses for about five minutes and holds the sender's Follow past a 15-second timeout (PrivaPub's retry gets it + through). Seen on a fresh install in the pasture, 2026-10-05. + - An incoming boost is kept with ActivityStreams' verb (`as#Announce`); its own are `/share`. + - Its activity ids are `uniqid()`, a three-character prefix and the microsecond: two of its processes answering two + follows at once gave both Accepts one id (town, 2026-10-05). PrivaPub queues an id that comes back carrying another + activity apart, so the second follow is not left pending. + - **It counts a follow as made before the Accept** when the target already follows its account (its "friend" + relation): its API answers `following: true` at once, and it shows the target's followers-only posts to that + account while a locked PrivaPub persona still holds the request (they reach Friendica's shared inbox for the + persona's accepted followers there, and Friendica hands them out by its own relations). PrivaPub refuses that + account's likes on them until the persona accepts. Town, 2026-10-05. + - It shows a reply to an account only inside a thread that account holds, and counts on the thread's owner to relay + the replies in it: a followers-only reply under a post the account cannot see never reaches it, from any server. + - Its "automatic friend" page type follows every follower back; the pasture's accounts are "soapbox" pages, which + take followers without asking and follow nobody back, as an unlocked Mastodon account does. Its Mastodon API makes + them so from `locked`, read as a number: `locked=true` unlocks an account. + - **Pasture evidence (2026-10-05, Friendica 2026.05, `tools/pasture/scenarios/friendica.sh`):** 25 checks pass, + from a clean install. Follows and unfollows both ways; posts both ways, a titled one arriving with its title; + comments both ways, threaded; likes both ways, Friendica's dislike as a downvote, alice's boost counted; edits and + deletes both ways; statistics. - **Gaps:** - **P1:** W2 for NodeBB Articles (`privapub.excerpt`). - **P2:** Groups without `followers`; Announces from an Application; context Move/Remove; paged `context` with ETag; diff --git a/tools/pasture/Caddyfile b/tools/pasture/Caddyfile index 5fc868a..688f288 100644 --- a/tools/pasture/Caddyfile +++ b/tools/pasture/Caddyfile @@ -72,3 +72,8 @@ wordpress.test { tls internal reverse_proxy pasture-wordpress:80 } + +friendica.test { + tls internal + reverse_proxy pasture-friendica:80 +} diff --git a/tools/pasture/peers/friendica.sh b/tools/pasture/peers/friendica.sh new file mode 100644 index 0000000..88b28d1 --- /dev/null +++ b/tools/pasture/peers/friendica.sh @@ -0,0 +1,74 @@ +# Friendica 2026.05: a social network speaking ActivityPub beside its own protocols. Posts (Notes, and Articles when +# titled), comments, likes and dislikes, follows, events. The official image on the shared MySQL (database friendica), +# installed by its autoinstall; its worker is the image's own daemon (cron.sh) in a sidecar sharing its files, or +# nothing federates. Its cache, locks and sessions live in the shared Redis (db 7), as upstream's compose has them, so +# both containers share them. It trusts Caddy's CA through the bundle mounted as its system store. Its API (Mastodon's +# and its own) takes HTTP Basic authentication, so a user's token is "nick:password". +FRIENDICA_IMAGE=${FRIENDICA_IMAGE:-docker.io/library/friendica:2026.05-apache} +FRIENDICA_PASSWORD=Friendica-Pasture-1 +. "$here/peers/shared.sh" + +friendica_env() { + echo -e "MYSQL_HOST=mysql\nMYSQL_USER=pasture\nMYSQL_PASSWORD=pasture\nMYSQL_DATABASE=friendica" + echo -e "FRIENDICA_ADMIN_MAIL=admin@friendica.test\nFRIENDICA_URL=https://friendica.test\nFRIENDICA_TZ=UTC\nFRIENDICA_LANG=en" + echo -e "FRIENDICA_SITENAME=Pasture Friendica\nREDIS_HOST=redis\nREDIS_DB=7" + # no check that an email's or a URL's domain resolves (friendica.test does only once Caddy names it) + echo "FRIENDICA_NO_VALIDATION=1" +} + +friendica_console() { podman exec -u www-data pasture-friendica php /var/www/html/bin/console.php "$@"; } + +friendica_up() { + shared_mysql_up + shared_redis_up + mysql_db friendica + mkdir -p "$here/.state/friendica" + friendica_env > "$here/.state/friendica/env" + podman volume exists pasture-friendica-html || podman volume create --label pasture=1 pasture-friendica-html >/dev/null + podman run -d --replace --name pasture-friendica --network $net --env-file "$here/.state/friendica/env" \ + -v pasture-friendica-html:/var/www/html -v "$ca/bundle.pem:/etc/ssl/certs/ca-certificates.crt:z,ro" "$FRIENDICA_IMAGE" >/dev/null + for _ in $(seq 1 150); do + site friendica.test -s -o /dev/null -w '%{http_code}' https://friendica.test:6443/nodeinfo/2.0 2>/dev/null | grep -q 200 && break + sleep 3 + done + podman run -d --replace --name pasture-friendica-cron --network $net --env-file "$here/.state/friendica/env" \ + --volumes-from pasture-friendica --entrypoint /cron.sh "$FRIENDICA_IMAGE" >/dev/null + friendica_settle + echo "friendica: https://friendica.test:6443" +} + +# a named system account, open registrations, a log, and fruser +friendica_settle() { + # its autoinstall leaves the system user (uid 0) nameless, so the system account that signs every fetch is never made + # and every lookup fails ("Could not find owner for uid 0"); a contact a failed attempt left at its address is dropped + podman exec pasture-mysql mysql -upasture -ppasture friendica -e "update user set nickname = 'friendica', username = 'System Account' + where uid = 0 and nickname = ''; delete from contact where uid = 0 and self = 0 and url = 'https://friendica.test/friendica'" 2>/dev/null + # the worker daemon runs in the sidecar, where the web container cannot see its pid, so a queued job never wakes it + # and everything waits for its five-minute cron; declared running, the web container signals it through worker-ipc + podman exec pasture-mysql mysql -upasture -ppasture friendica -e "replace into \`key-value\` (k, v, updated_at) + values ('worker_daemon_mode', '1', unix_timestamp())" 2>/dev/null + friendica_console config config register_policy 2 >/dev/null 2>&1 || true + # the image names the log (at notice) but leaves the file for Friendica to make, which it cannot, and logs nothing + # until debugging is on + podman exec pasture-friendica sh -c 'touch /var/www/html/friendica.log && chown www-data /var/www/html/friendica.log' + friendica_console config system debugging 1 >/dev/null 2>&1 || true + friendica_user fruser + echo "fruser:$FRIENDICA_PASSWORD" > "$here/.state/friendica.token" +} + +# friendica_user : an account with the pasture's password that takes followers without asking. Friendica makes +# them locked (its normal page type approves every contact by hand); saved through its API with locked=0 (a number: +# "false" and "true" both read as 0), it becomes a "soapbox" page, which approves each follower and follows nobody back. +# The "automatic friend" type would follow every follower back as a friend. +friendica_user() { + friendica_console user add "$1" "$1" "$1@friendica.test" en "" >/dev/null 2>&1 || true + friendica_console user password "$1" "$FRIENDICA_PASSWORD" >/dev/null 2>&1 || true + podman exec pasture-friendica curl -s -o /dev/null -X PATCH -u "$1:$FRIENDICA_PASSWORD" -H "Host: friendica.test" \ + -H "X-Forwarded-Proto: https" -d locked=0 http://localhost/api/v1/accounts/update_credentials + # Friendica caches its own users' actors (apcontact) only once something reads them. A Follow that arrives first makes + # it fetch the user from itself, signed as that user, and checking that signature builds the actor, which checks the + # signature again: the request recurses for about five minutes and holds the sender's Follow past any timeout. + # Reading the user's outbox once, unsigned, caches it (its own search answers from the contact and caches nothing). + podman exec pasture-friendica curl -s -o /dev/null -H "Host: friendica.test" -H "X-Forwarded-Proto: https" \ + -H 'Accept: application/activity+json' "http://localhost/outbox/$1" +} diff --git a/tools/pasture/peers/shared.sh b/tools/pasture/peers/shared.sh index 24a19bc..3b29c1a 100644 --- a/tools/pasture/peers/shared.sh +++ b/tools/pasture/peers/shared.sh @@ -1,4 +1,5 @@ -# Services several peers share: one Postgres (each peer gets its own database) and one Redis (each its own db number). +# Services several peers share: one Postgres and one MySQL (each peer gets its own database), and one Redis (each its own +# db number). shared_postgres_up() { podman container exists pasture-postgres && return 0 podman run -d --replace --name pasture-postgres --network $net --network-alias postgres \ @@ -20,3 +21,18 @@ pg_db() { podman exec pasture-postgres psql -U pasture -d "$name" -qc "create extension if not exists \"$ext\";" >/dev/null done } + +shared_mysql_up() { + podman container exists pasture-mysql && return 0 + podman run -d --replace --name pasture-mysql --network $net --network-alias mysql \ + -e MYSQL_ROOT_PASSWORD=pasture -e MYSQL_USER=pasture -e MYSQL_PASSWORD=pasture docker.io/library/mysql:8.4 >/dev/null + # its first start runs a temporary server without networking to set itself up: only TCP answers when it is ready + for _ in $(seq 1 90); do podman exec pasture-mysql mysqladmin ping -h127.0.0.1 --protocol=tcp -uroot -ppasture --silent >/dev/null 2>&1 && return 0; sleep 2; done + echo "mysql did not start" >&2; return 1 +} + +# mysql_db : a database of the shared MySQL for one peer, the pasture user owning it +mysql_db() { + podman exec pasture-mysql mysql -uroot -ppasture -e \ + "create database if not exists \`$1\` character set utf8mb4 collate utf8mb4_unicode_ci; grant all on \`$1\`.* to 'pasture'@'%';" 2>/dev/null +} diff --git a/tools/pasture/peers/wordpress.sh b/tools/pasture/peers/wordpress.sh index 7b1dbbc..b1537b9 100644 --- a/tools/pasture/peers/wordpress.sh +++ b/tools/pasture/peers/wordpress.sh @@ -10,21 +10,6 @@ WORDPRESS_ACTIVITYPUB=9.3.1 WORDPRESS_PASSWORD=Wordpress-Pasture-1 . "$here/peers/shared.sh" -shared_mysql_up() { - podman container exists pasture-mysql && return 0 - podman run -d --replace --name pasture-mysql --network $net --network-alias mysql \ - -e MYSQL_ROOT_PASSWORD=pasture -e MYSQL_USER=pasture -e MYSQL_PASSWORD=pasture docker.io/library/mysql:8.4 >/dev/null - # its first start runs a temporary server without networking to set itself up: only TCP answers when it is ready - for _ in $(seq 1 90); do podman exec pasture-mysql mysqladmin ping -h127.0.0.1 --protocol=tcp -uroot -ppasture --silent >/dev/null 2>&1 && return 0; sleep 2; done - echo "mysql did not start" >&2; return 1 -} - -# mysql_db : a database of the shared MySQL for one peer, the pasture user owning it -mysql_db() { - podman exec pasture-mysql mysql -uroot -ppasture -e \ - "create database if not exists \`$1\` character set utf8mb4 collate utf8mb4_unicode_ci; grant all on \`$1\`.* to 'pasture'@'%';" 2>/dev/null -} - # wp : wp-cli against the site, sharing its files wp() { podman run --rm --network $net --volumes-from pasture-wordpress --user 33:33 -e HOME=/tmp \ diff --git a/tools/pasture/scenarios/friendica.sh b/tools/pasture/scenarios/friendica.sh new file mode 100644 index 0000000..a8ccc8d --- /dev/null +++ b/tools/pasture/scenarios/friendica.sh @@ -0,0 +1,103 @@ +# Friendica 2026.05: follows both ways; posts both ways, a titled one as an Article; comments both ways; likes both +# ways, Friendica's dislike as a downvote, a boost; edits, deletes and unfollows both ways; statistics. Friendica keeps every +# activity (a like, a dislike, a boost) as an item of its own with a verb, so what it holds is read from its MySQL +# (database friendica) by `thr-parent`, never fetched. Its API takes HTTP Basic authentication. An edit made through its +# Mastodon API never federates (Statuses::put leaves `edited` alone, and only a changed `edited` notifies), so fruser +# edits through the web editor, signed in with a cookie. +FR=https://friendica.test:6443 +frc() { curl -sk --resolve friendica.test:6443:127.0.0.1 -u "$(cat "$here/.state/friendica.token" 2>/dev/null)" -H 'Accept: application/json' "$@"; } +fr_sql() { podman exec pasture-mysql mysql -upasture -ppasture friendica -Nse "$1" 2>/dev/null; } +# fr_count : the live activities of a verb on a post Friendica holds, by the verb's last part ("/like", +# "/dislike", "#Announce": a boost that arrives keeps ActivityStreams' name, where Friendica's own are "/share") +fr_count() { fr_sql "select count(*) from \`post-view\` where \`thr-parent\` = '$1' and verb like '%$2' and deleted = 0"; } +# fr_web_edit : fruser edits a post through the web editor (the API's status id is the uri-id) +fr_web_edit() { + local jar="$here/.state/friendica.cookies" credentials item + credentials=$(cat "$here/.state/friendica.token") + curl -sk --resolve friendica.test:6443:127.0.0.1 -c "$jar" -o /dev/null -X POST "$FR/login" -d auth-params=login -d username=fruser \ + --data-urlencode "password=${credentials#*:}" + item=$(fr_sql "select id from \`post-user\` where \`uri-id\` = $1 and uid = (select uid from user where nickname = 'fruser')") + curl -sk --resolve friendica.test:6443:127.0.0.1 -b "$jar" -o /dev/null -X POST "$FR/item" -d "post_id=$item" --data-urlencode "body=$2" +} +p_home_id() { curl -s -H "$PH" "$P/api/v1/timelines/home?limit=40" | j "print(next((o['id'] for o in ((s.get('reblog') or s) for s in d) if '$1' in (o['content'] or '') or '$1' in ((o.get('privapub') or {}).get('title') or '')), ''))"; } + +echo "friendica" +[ -s "$here/.state/friendica.token" ] && ok "Friendica credentials for fruser" || { ko "Friendica credentials"; return 1; } +PT=$(privapub_token alice_friendica) +PH="Authorization: Bearer $PT" +[ -n "$PT" ] && ok "PrivaPub token for alice_friendica" || { ko "PrivaPub token for alice_friendica"; return 1; } + +echo " discovery and follows" +alice_on_fr=$(frc "$FR/api/v2/search?q=@alice_friendica@privapub.test&resolve=true&type=accounts" | j "print(d['accounts'][0]['id'])") +[ -n "$alice_on_fr" ] && ok "Friendica resolves @alice_friendica@privapub.test" || ko "Friendica cannot resolve alice_friendica" +fr_on_p=$(curl -s -H "$PH" "$P/api/v2/search?q=fruser@friendica.test&resolve=true&type=accounts" | j "print(d['accounts'][0]['id'])") +[ -n "$fr_on_p" ] && ok "PrivaPub resolves @fruser@friendica.test" || ko "PrivaPub cannot resolve fruser" +# a run cut short, or a Friendica made anew, leaves follows behind on one side only: unfollow first, so both follows +# below are new requests +frc -o /dev/null -X POST "$FR/api/v1/accounts/$alice_on_fr/unfollow" +curl -s -o /dev/null -X POST -H "$PH" "$P/api/v1/accounts/$fr_on_p/unfollow" +sleep 3 +frc -o /dev/null -X POST "$FR/api/v1/accounts/$alice_on_fr/follow" +until_true 45 '[ "$(frc "$FR/api/v1/accounts/relationships?id[]=$alice_on_fr" | j "print(d[0][\"following\"])")" = "True" ]' \ + && ok "fruser follows alice_friendica (Accept arrived)" || ko "Friendica's follow not accepted" +curl -s -o /dev/null -X POST -H "$PH" "$P/api/v1/accounts/$fr_on_p/follow" +until_true 45 '[ "$(curl -s -H "$PH" "$P/api/v1/accounts/relationships?id[]=$fr_on_p" | j "print(d[0][\"following\"])")" = "True" ]' \ + && ok "alice_friendica follows fruser (Accept arrived)" || ko "PrivaPub's follow not accepted" + +echo " posts" +p_post=$(curl -s -X POST -H "$PH" "$P/api/v1/statuses" -d 'status=Hello Friendica from PrivaPub&visibility=public') +p_post_id=$(echo "$p_post" | j "print(d['id'])"); p_post_uri=$(echo "$p_post" | j "print(d['uri'])") +until_true 45 '[ "$(fr_sql "select count(*) from \`post-view\` where uri = '"'$p_post_uri'"' and deleted = 0")" -ge 1 ]' \ + && ok "alice_friendica's post reaches Friendica" || ko "alice_friendica's post missing on Friendica" +fr_post=$(frc -X POST "$FR/api/v1/statuses" -d 'status=Hello PrivaPub from Friendica&visibility=public' | j "print(d['id'])") +until_true 45 '[ -n "$(p_home_id "Hello PrivaPub from Friendica")" ]' && ok "fruser's post reaches alice_friendica's home" || ko "fruser's post never arrived" +fr_on_p_post=$(p_home_id "Hello PrivaPub from Friendica") +fr_post_uri=$(curl -s -H "$PH" "$P/api/v1/statuses/$fr_on_p_post" | j "print(d['uri'])") +frc -o /dev/null -X POST "$FR/api/v1/statuses" -d 'status=An article with paragraphs.&friendica[title]=A Friendica article&visibility=public' +until_true 45 '[ -n "$(p_home_id "A Friendica article")" ]' && ok "fruser's titled post arrives with its title" || ko "the titled post never arrived, or lost its title" + +echo " comments" +frc -o /dev/null -X POST "$FR/api/v1/statuses" -d "status=@alice_friendica@privapub.test a Friendica comment&in_reply_to_id=$(frc "$FR/api/v2/search?q=$(python3 -c "import urllib.parse,sys; print(urllib.parse.quote(sys.argv[1]))" "$p_post_uri")&resolve=true&type=statuses" | j "print(d['statuses'][0]['id'])")&visibility=public" +until_true 45 '[ "$(curl -s -H "$PH" "$P/api/v1/statuses/$p_post_id/context" | j "print(any(\"a Friendica comment\" in s[\"content\"] for s in d[\"descendants\"]))")" = "True" ]' \ + && ok "fruser's comment threads under alice_friendica's post" || ko "Friendica's comment missing on PrivaPub" +curl -s -o /dev/null -X POST -H "$PH" "$P/api/v1/statuses" -d "status=@fruser@friendica.test a PrivaPub comment&in_reply_to_id=$fr_on_p_post&visibility=public" +until_true 45 '[ "$(fr_sql "select count(*) from \`post-view\` where \`thr-parent\` = '"'$fr_post_uri'"' and body like '"'%a PrivaPub comment%'"'")" -ge 1 ]' \ + && ok "alice_friendica's comment threads under fruser's post on Friendica" || ko "alice_friendica's comment missing on Friendica" + +echo " likes, dislikes and boosts" +curl -s -o /dev/null -X POST -H "$PH" "$P/api/v1/statuses/$fr_on_p_post/favourite" +until_true 45 '[ "$(fr_count "$fr_post_uri" /like)" = "1" ]' && ok "alice_friendica's like counts on Friendica" || ko "like not counted on Friendica" +curl -s -o /dev/null -X POST -H "$PH" "$P/api/v1/statuses/$fr_on_p_post/reblog" +until_true 45 '[ "$(fr_count "$fr_post_uri" "#Announce")" = "1" ]' && ok "alice_friendica's boost counts on Friendica" || ko "boost not counted on Friendica" +p_on_fr=$(frc "$FR/api/v2/search?q=$(python3 -c "import urllib.parse,sys; print(urllib.parse.quote(sys.argv[1]))" "$p_post_uri")&resolve=true&type=statuses" | j "print(d['statuses'][0]['id'])") +frc -o /dev/null -X POST "$FR/api/v1/statuses/$p_on_fr/favourite" +until_true 45 '[ "$(curl -s -H "$PH" "$P/api/v1/statuses/$p_post_id" | j "print(d[\"favourites_count\"])")" = "1" ]' && ok "fruser's like counts on PrivaPub" || ko "Friendica's like not counted" +frc -o /dev/null -X POST "$FR/api/v1/statuses/$p_on_fr/unfavourite" +frc -o /dev/null -X POST "$FR/api/friendica/activity/dislike" -d "id=$p_on_fr" +until_true 45 '[ "$(curl -s -H "$PH" "$P/api/v1/statuses/$p_post_id" | j "print(d[\"favourites_count\"], ((d.get(\"privapub\") or {}).get(\"votes\") or {}).get(\"down\"))")" = "0 1" ]' \ + && ok "fruser's dislike counts as a downvote on PrivaPub" || ko "Friendica's dislike not counted" + +echo " edits and deletes" +curl -s -o /dev/null -X PUT -H "$PH" "$P/api/v1/statuses/$p_post_id" -d 'status=Hello Friendica from PrivaPub, edited' +until_true 45 '[ "$(fr_sql "select count(*) from \`post-view\` where uri = '"'$p_post_uri'"' and body like '"'%edited%'"'")" = "1" ]' \ + && ok "alice_friendica's edit reaches Friendica" || ko "PrivaPub's edit not applied on Friendica" +fr_web_edit "$fr_post" 'Hello PrivaPub from Friendica, edited' +until_true 45 '[ "$(curl -s -H "$PH" "$P/api/v1/statuses/$fr_on_p_post" | j "print(\"edited\" in d[\"content\"])")" = "True" ]' \ + && ok "fruser's edit reaches PrivaPub" || ko "Friendica's edit not applied" +curl -s -o /dev/null -X DELETE -H "$PH" "$P/api/v1/statuses/$p_post_id" +until_true 45 '[ "$(fr_sql "select count(*) from \`post-view\` where uri = '"'$p_post_uri'"' and deleted = 0")" = "0" ]' \ + && ok "alice_friendica's delete reaches Friendica" || ko "delete not applied on Friendica" +frc -o /dev/null -X DELETE "$FR/api/v1/statuses/$fr_post" +until_true 45 '[ "$(curl -s -o /dev/null -w "%{http_code}" -H "$PH" "$P/api/v1/statuses/$fr_on_p_post")" = "404" ]' \ + && ok "fruser's delete reaches PrivaPub" || ko "Friendica's delete not applied" + +echo " unfollows" +curl -s -o /dev/null -X POST -H "$PH" "$P/api/v1/accounts/$fr_on_p/unfollow" +until_true 45 '[ "$(frc "$FR/api/v1/accounts/relationships?id[]=$alice_on_fr" | j "print(d[0][\"followed_by\"])")" = "False" ]' \ + && ok "alice_friendica's unfollow reaches Friendica" || ko "PrivaPub's unfollow not applied on Friendica" +frc -o /dev/null -X POST "$FR/api/v1/accounts/$alice_on_fr/unfollow" +until_true 45 '[ "$(curl -s -H "$PH" "$P/api/v1/accounts/relationships?id[]=$fr_on_p" | j "print(d[0][\"followed_by\"])")" = "False" ]' \ + && ok "fruser's unfollow reaches PrivaPub" || ko "Friendica's unfollow not applied" + +echo " statistics" +stats_check friendica.test friendica diff --git a/tools/pasture/town/check.py b/tools/pasture/town/check.py index 9360e16..a240c22 100644 --- a/tools/pasture/town/check.py +++ b/tools/pasture/town/check.py @@ -31,6 +31,11 @@ DROPS_ORPHAN_REPLIES = {"misskey", "sharkey"} # Pixelfed is a photo platform: it keeps a post only when it carries a picture (a Note without one is dropped as it # arrives), and a reply only to a post it holds NEEDS_PICTURE = {"pixelfed"} +# Friendica shows a reply to an account only inside a thread that account holds, under a parent it holds, and counts on +# the thread's owner to relay the replies in it: a non-public reply never reaches an account that cannot see one of the +# posts above it, and a Friendica account's non-public reply in a thread another server owns reaches none of the +# others there (nobody relays a followers-only reply), nor anything under it +THREAD_BOUND = {"friendica"} class World: @@ -332,7 +337,7 @@ class Sweep: if not row or not row.exists or row.deleted: continue here = [k for k in self.w.planner.accounts if self.w.platform(k) == p and k != o["author"]] - inside = sorted(k for k in here if k in recipients)[:2] + inside = sorted(k for k in here if k in recipients and self.in_thread(p, o, k))[:2] outside = sorted(k for k in here if k not in recipients)[:2] for k in inside: by_platform[p].append((ref, o, k, True)) @@ -351,6 +356,19 @@ class Sweep: feature = f"{'see' if should else 'hide'}.{o['visibility']}" self.add(feature, o["origin"], p, seen is should, ref, should, seen, how=k) + def in_thread(self, p, o, k): + """Whether a reply can reach account k on platform p, where a platform shows replies only in threads (THREAD_BOUND).""" + if p not in THREAD_BOUND or not o.get("parent"): + return True + chain, node = [], o + while node is not None: + chain.append(node) + node = self.w.objects.get(node.get("parent") or "") + root, shown = chain[-1], ("public", "unlisted", "community") + if any(a["visibility"] not in shown and k != a["author"] and k not in self.w.recipients(a) for a in chain[1:]): + return False + return not any(a["origin"] == p and root["origin"] != p and a["visibility"] not in shown and k != a["author"] for a in chain[:-1]) + # -- the graph as each side reports it def graph(self): sessions = {f"{s.account.platform}/{s.account.username}": s for s in state.sessions()} diff --git a/tools/pasture/town/core/podman.py b/tools/pasture/town/core/podman.py index 96f7733..197a631 100644 --- a/tools/pasture/town/core/podman.py +++ b/tools/pasture/town/core/podman.py @@ -51,6 +51,19 @@ def psql_value(db, sql, *params): return next(iter(rows[0].values())) +def mysql_json(db, sql): + """The JSON value one MySQL query returns (built with json_object/json_arrayagg, so text keeps its newlines), from the + shared MySQL. The mysql client binds nothing: values go in through `mysql_quote`.""" + out = run("exec", "pasture-mysql", "mysql", "-upasture", "-ppasture", "--default-character-set=utf8mb4", "-N", "-B", "--raw", db, + "-e", sql) + text = "\n".join(line for line in out.splitlines() if not line.startswith("mysql: [Warning]")).strip() + return json.loads(text) if text and text != "NULL" else None + + +def mysql_quote(value): + return "'" + str(value).replace("\\", "\\\\").replace("'", "''") + "'" + + def mongo(js, db="PrivaPub"): """The JSON value of a mongosh expression, e.g. `db.Post.find({...}).toArray()`.""" script = f"print(EJSON.stringify(({js}), {{relaxed: true}}))" diff --git a/tools/pasture/town/dialects/__init__.py b/tools/pasture/town/dialects/__init__.py index 853bcc4..96fe292 100644 --- a/tools/pasture/town/dialects/__init__.py +++ b/tools/pasture/town/dialects/__init__.py @@ -1,5 +1,6 @@ """The platform registry: name -> driver class and its site.""" from dialects.akkoma import Akkoma +from dialects.friendica import Friendica from dialects.gts import Gts from dialects.hollo import Hollo from dialects.iceshrimp import Iceshrimp @@ -22,6 +23,7 @@ DRIVERS = { "iceshrimp": (Iceshrimp, "iceshrimp.test"), "pleroma": (Pleroma, "pleroma.test"), "pixelfed": (Pixelfed, "pixelfed.test"), + "friendica": (Friendica, "friendica.test"), } _made = {} diff --git a/tools/pasture/town/dialects/friendica.py b/tools/pasture/town/dialects/friendica.py new file mode 100644 index 0000000..7b43e17 --- /dev/null +++ b/tools/pasture/town/dialects/friendica.py @@ -0,0 +1,113 @@ +"""Friendica 2026.05: accounts made by its console as peers/friendica.sh makes fruser ("soapbox" pages, which take +followers without asking or following back, unless locked; each one's outbox read once so that a first Follow does not make Friendica recurse), its +Mastodon API with HTTP Basic credentials (a session's token is "nick:password"), objects read from its MySQL +(`friendica`, the `post-view` view by uri). An edit through its Mastodon API never federates, so edits go through its +web editor, signed in with a cookie. Its API says "private" for a DM too: Friendica keeps both as private posts.""" +import base64 + +from core import podman +from dialects.base import Session, Stored, Unsupported +from dialects.mastodon_api import MastodonApi + +CONTAINER = "pasture-friendica" +DB = "friendica" +# post-view.private, as its Mastodon API names it +VIS = {0: "public", 1: "followers", 2: "unlisted"} + + +def basic(token): + return "Basic " + base64.b64encode(token.encode()).decode() + + +class Friendica(MastodonApi): + platform = "friendica" + caps = frozenset({"post", "reply", "cw", "media", "like", "boost", "bookmark", "follow", "block", "mute", "dm", "delete", + "edit", "profile"}) + + def __init__(self, host): + super().__init__(host) + self._cookies = {} # nick -> session cookie of its web editor + + def api(self, s, method, path, ok=None, **kw): + headers = kw.pop("headers", {}) + if s is not None: + headers["Authorization"] = basic(s.token) + return self.http.request(method, self.base + path, headers=headers, ok=ok, template=path.split("?")[0], **kw) + + def session_from_token(self, account, token): + me = self.api_json(None, "GET", "/api/v1/accounts/verify_credentials", headers={"Authorization": basic(token)}) + return Session(account, token, me["id"], self.actor_uri(account.username) or me.get("url")) + + def flag(self, value): + # its API reads `locked` as a number: "true" is 0, unlocked + return 1 if value else 0 + + def _console(self, *args): + return podman.run("exec", "-u", "www-data", CONTAINER, "php", "/var/www/html/bin/console.php", *args, check=False) + + def provision(self, accounts): + existing = set(podman.mysql_json(DB, "select json_arrayagg(nickname) from user") or []) + for a in accounts: + if a.username not in existing: + self._console("user", "add", a.username, a.username, f"{a.username}@{self.host}", "en", "") + self._console("user", "password", a.username, a.password) + # the API makes an unlocked account a soapbox page (it takes followers without asking and follows nobody + # back) and a locked one a normal page, which holds each request + self.http.request("PATCH", self.base + "/api/v1/accounts/update_credentials", ok={200}, + headers={"Authorization": basic(f"{a.username}:{a.password}")}, form={"locked": self.flag(a.locked)}) + self.http.request("GET", self.base + f"/outbox/{a.username}", headers={"Accept": "application/activity+json"}) + return [self.session_from_token(a, f"{a.username}:{a.password}") for a in accounts] + + def post(self, s, spec): + if spec.poll: + raise Unsupported(self.platform, "post a poll") + return super().post(s, spec) + + def bookmark(self, s, uri): + row = self._rows([uri]).get(uri) + if row and row.parent_uri: + raise Unsupported(self.platform, "bookmark a reply") # "Only starting posts can be bookmarked" + super().bookmark(s, uri) + + def edit(self, s, uri, spec): + sid = self._own(s, uri) + text = spec.text + for acct in spec.mentions: + if f"@{acct}" not in text: + text = f"@{acct} {text}" + item = podman.mysql_json(DB, f"select id from `post-user` where `uri-id` = {int(sid)} and uid = " + f"(select uid from user where nickname = {podman.mysql_quote(s.account.username)})") + form = {"post_id": item, "body": text} + if spec.cw: + form["summary"] = spec.cw + self._web(s, "POST", "/item", form) + + def _web(self, s, method, path, form): + nick, password = s.token.split(":", 1) + if nick not in self._cookies: + r = self.http.request("POST", self.base + "/login", form={"auth-params": "login", "username": nick, "password": password}) + self._cookies[nick] = "; ".join(v.split(";")[0] for k, v in r.headers if k.lower() == "set-cookie") + return self.http.request(method, self.base + path, headers={"Cookie": self._cookies[nick]}, form=form) + + def _rows(self, uris): + if not uris: + return {} + # (its uri columns are binary, which json_object would give as base64) + listed = ", ".join(podman.mysql_quote(u) for u in uris) + rows = podman.mysql_json(DB, f""" + select json_arrayagg(json_object( + 'uri', convert(p.uri using utf8mb4), 'local_id', p.`uri-id`, 'private', p.private, 'deleted', p.deleted, 'gravity', p.gravity, + 'text', p.body, 'cw', p.`content-warning`, 'title', p.title, 'edited', p.edited > p.created, + 'parent', case when p.gravity = 6 then convert(p.`thr-parent` using utf8mb4) end, + 'likes', (select count(*) from `post-view` a where a.`thr-parent-id` = p.`uri-id` and a.gravity = 3 + and a.deleted = 0 and a.verb like '%/like'), + 'boosts', (select count(*) from `post-view` a where a.`thr-parent-id` = p.`uri-id` and a.gravity = 3 + and a.deleted = 0 and (a.verb like '%/share' or a.verb like '%#Announce')), + 'replies', (select count(*) from `post-view` a where a.`thr-parent-id` = p.`uri-id` and a.gravity = 6 and a.deleted = 0))) + from `post-view` p where p.uri in ({listed}) and p.gravity in (0, 6)""") or [] + out = {} + for r in rows: + out[r["uri"]] = Stored(True, bool(r["deleted"]), str(r["local_id"]), VIS.get(r["private"], str(r["private"])), r["text"], + r["cw"] or None, bool(r["edited"]), r["parent"], r["likes"], r["boosts"], r["replies"], None, {}, r) + return out + diff --git a/tools/pasture/town/dialects/mastodon_api.py b/tools/pasture/town/dialects/mastodon_api.py index 908bbad..e63aea6 100644 --- a/tools/pasture/town/dialects/mastodon_api.py +++ b/tools/pasture/town/dialects/mastodon_api.py @@ -37,8 +37,12 @@ class MastodonApi(Driver): return Session(account, token, me["id"], actor) # -- profiles + def flag(self, value): + """A boolean as the server's forms read one.""" + return value + def update_profile(self, s, account): - form = {"display_name": account.name, "note": account.bio, "locked": account.locked, "bot": account.bot} + form = {"display_name": account.name, "note": account.bio, "locked": self.flag(account.locked), "bot": self.flag(account.bot)} for i, (k, v) in enumerate(account.fields[:4]): form[f"fields_attributes[{i}][name]"] = k form[f"fields_attributes[{i}][value]"] = v diff --git a/tools/pasture/town/gen.py b/tools/pasture/town/gen.py index 728efb8..667c3f9 100644 --- a/tools/pasture/town/gen.py +++ b/tools/pasture/town/gen.py @@ -20,10 +20,12 @@ from core.rng import Rng GENERATOR_VERSION = 1 HOSTS = {"privapub": "privapub.test", "gts": "gts.test", "mastodon": "mastodon.test", "misskey": "misskey.test", "sharkey": "sharkey.test", "akkoma": "akkoma.test", "lemmy": "lemmy.test", "hollo": "hollo.test", - "iceshrimp": "iceshrimp.test", "pleroma": "pleroma.test", "pixelfed": "pixelfed.test"} + "iceshrimp": "iceshrimp.test", "pleroma": "pleroma.test", "pixelfed": "pixelfed.test", + "friendica": "friendica.test"} SHORT = {"privapub": "pp", "gts": "gt", "mastodon": "ms", "misskey": "mk", "sharkey": "sk", "akkoma": "ak", "lemmy": "lm", - "hollo": "ho", "iceshrimp": "is", "pleroma": "pl", "pixelfed": "pf"} -MICRO = ("privapub", "gts", "mastodon", "misskey", "sharkey", "akkoma", "hollo", "iceshrimp", "pleroma", "pixelfed") + "hollo": "ho", "iceshrimp": "is", "pleroma": "pl", "pixelfed": "pf", + "friendica": "fr"} +MICRO = ("privapub", "gts", "mastodon", "misskey", "sharkey", "akkoma", "hollo", "iceshrimp", "pleroma", "pixelfed", "friendica") CAPS = { "privapub": {"post", "reply", "cw", "media", "poll", "like", "boost", "bookmark", "follow", "block", "mute", "report", "dm", "delete", "edit", "vote", "react", "quote", "profile", "circle", "community", "located"}, @@ -47,6 +49,9 @@ CAPS = { # a photo platform: every post carries a picture (the driver adds one), and it has no polls or DMs through its API "pixelfed": {"post", "reply", "cw", "media", "like", "boost", "bookmark", "follow", "block", "mute", "delete", "edit", "profile"}, + # no polls through its API; a DM is a private post addressed to its recipients + "friendica": {"post", "reply", "cw", "media", "like", "boost", "bookmark", "follow", "block", "mute", "dm", "delete", + "edit", "profile"}, } FIRST = ["ada", "bo", "cleo", "dario", "elif", "femi", "gaia", "hiro", "ines", "jun", "kai", "lia", "milo", "nadia", diff --git a/tools/pasture/town/seed.py b/tools/pasture/town/seed.py index a22e34f..01dc81f 100644 --- a/tools/pasture/town/seed.py +++ b/tools/pasture/town/seed.py @@ -217,10 +217,16 @@ class Seeder: raise TimeoutError(f"{follower}'s follow request never reached {s['actor']}") def v_accept(self, s): - # a follow that came back accepted needs no answer: the target was unlocked after all, or a run before this one - # on the same accounts already accepted it - if any(f.get("type") == "relation" and f["verb"] == "follow" and f["actor"] == s["args"]["target"] - and f["target"] == s["actor"] and f["state"] == "accepted" for f in self.ledger.facts): + # a follow its target already counts needs no answer (the target was unlocked after all, or a run before this one + # on the same accounts accepted it). The target's own view decides: Friendica reports a follow of someone who + # already follows its account as made at once, while the target still holds the request + platform = s["actor"].split("/")[0] + try: + counted = driver(platform).relationship(self.session(s["actor"]), self.acct(s["args"]["target"])).get("followed_by") + except Unsupported: + counted = any(f.get("type") == "relation" and f["verb"] == "follow" and f["actor"] == s["args"]["target"] + and f["target"] == s["actor"] and f["state"] == "accepted" for f in self.ledger.facts) + if counted: self.ledger.add(type="relation", n=s["n"], verb="accept", actor=s["actor"], target=s["args"]["target"], state="accepted", already=True) return diff --git a/tools/pasture/town/selftest.py b/tools/pasture/town/selftest.py index efe525d..8aa4c0c 100644 --- a/tools/pasture/town/selftest.py +++ b/tools/pasture/town/selftest.py @@ -97,10 +97,13 @@ def microblog(peer, r): r.check(rrow and rrow.parent_uri == made.uri, f"the reply names its parent (got {rrow and rrow.parent_uri})") r.check(wait(lambda: (d.stored([made.uri])[made.uri].replies or 0) >= 1), "the parent counts the reply") - poll = d.post(a, PostSpec(text=f"selftest poll on {peer} (st3)", poll={"options": ["tea", "coffee"], "expires_in": 86400})) - d.vote(b, poll.uri, [1]) - r.check(wait(lambda: (d.stored([poll.uri])[poll.uri].votes or [None, None])[1] == 1), - f"a vote counts on the second option (got {d.stored([poll.uri])[poll.uri].votes})") + if "poll" in d.caps: + poll = d.post(a, PostSpec(text=f"selftest poll on {peer} (st3)", poll={"options": ["tea", "coffee"], "expires_in": 86400})) + d.vote(b, poll.uri, [1]) + r.check(wait(lambda: (d.stored([poll.uri])[poll.uri].votes or [None, None])[1] == 1), + f"a vote counts on the second option (got {d.stored([poll.uri])[poll.uri].votes})") + else: + r.skip(f"{peer} posts no polls") private = d.post(a, PostSpec(text=f"selftest followers-only on {peer} (st4)", visibility="followers")) prow = d.stored([private.uri])[private.uri] diff --git a/tools/pasture/town/specs/friendica-pair.json b/tools/pasture/town/specs/friendica-pair.json new file mode 100644 index 0000000..03ae006 --- /dev/null +++ b/tools/pasture/town/specs/friendica-pair.json @@ -0,0 +1,94 @@ +{ + "schema": "pasture-town/1", + "name": "friendica-pair", + "seed": 20261045, + "peers": { + "privapub": { + "roots": 1, + "personas": [ + 2, + 2 + ], + "circles": 1, + "communities": 0 + }, + "friendica": { + "accounts": 3 + } + }, + "profiles": { + "locked": 0.2, + "bot": 0.0, + "fields": [ + 0, + 2 + ], + "avatar": 0.9, + "header": 0.3, + "bioWords": [ + 5, + 12 + ], + "langs": { + "en": 80, + "it": 20 + } + }, + "graph": { + "follows": [ + 2, + 4 + ], + "mutual": 0.6, + "pending": 0.0, + "rejected": 0.0 + }, + "circleMembers": 2, + "content": { + "posts": 24, + "mix": { + "text": 40, + "cw": 10, + "tags": 10, + "mention": 10, + "image": 20, + "poll": 10 + }, + "visibility": { + "public": 50, + "unlisted": 10, + "followers": 20, + "direct": 10, + "circle": 10 + }, + "replyRounds": 2, + "replies": 0.5, + "deeperReplies": 0.4 + }, + "interactions": { + "likes": [ + 0, + 3 + ], + "boosts": [ + 0, + 1 + ], + "react": 0.4, + "vote": 0.8, + "bookmark": 0.1 + }, + "mutations": { + "edit": 0.15, + "delete": 0.05, + "blocks": 0, + "mutes": 1, + "reports": 0 + }, + "time": { + "roundSettleSeconds": 15, + "graphSettleSeconds": 20, + "settleSeconds": 30, + "deadlineSeconds": 120 + } +}