diff --git a/CLAUDE.md b/CLAUDE.md index 83efd6a..9277530 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -529,6 +529,17 @@ tools/pasture/run.sh down # removes e `/inbox` when there is none, and keeps serving a thread its moderator removed. `scenarios/piefed.sh`, 29 checks: communities both ways, threads with titles, comments, votes up and down both ways, a community poll and alice's vote, private messages both ways, a moderator's lock, unlock and removal, the unfollow, statistics. +- **Mbin (1.10.1):** its own image (FrankenPHP serving plain HTTP behind Caddy, `SERVER_NAME=:80`) and a messenger + worker, on the shared Postgres and Redis (db 12) with a RabbitMQ of its own (its transports carry AMQP options; it + runs as its own user on its own volume, or it cannot read the `.erlang.cookie` it wrote as root). The pasture's bundle + is mounted over the system one; its API's rate limits (two threads every six minutes) are raised by a copy of its + `rate_limiter.yaml`. Its admin mbuser is made by its console; `peers/mbin_token.py` gets mbuser's OAuth token through + the authorization-code flow (login form, consent), since a client-credentials client acts as a bot that may not vote. + Mbin names what it makes during a request after the request's host, so every call says `Host: mbin.test`, never + the workstation's port. Its API never starts a conversation with an account elsewhere. `scenarios/mbin.sh`, 24 + checks and one known gap (G-0008, direct messages): magazines both ways, threads with titles, a Note to a magazine + as a microblog post, comments both ways, favourites and upvotes both ways, a moderator's lock, unlock and removal, the + unfollow, statistics. - **Hollo (0.9.19):** Fedify's microblog server on the shared Postgres, set up through its web form (which checks `Origin` against `Host`, so the request names `hollo.test` without the port). It needs a 44-character `SECRET_KEY`, a media directory and a `themeColor`; statuses and votes go as JSON. Town only, no scenario. diff --git a/FEDERATION.md b/FEDERATION.md index 64a24f6..44764bb 100644 --- a/FEDERATION.md +++ b/FEDERATION.md @@ -30,6 +30,7 @@ and every run starting clean, with signed fetches required (as privapub.thepra.d - **Gancio 1.28.2** - **Funkwhale 2.0.11** - **PieFed 1.7.17** +- **Mbin 1.10.1** - in the town only (a seeded community checked server by server): **Hollo 0.9.19**, **Iceshrimp.NET 2026.1.2-beta**, **Pleroma 2.10.2** @@ -45,8 +46,8 @@ Checked both ways, where the peer has the feature: - communities and circles; - blocks and unfollows. -`docs/INTEROP.md` has each peer's evidence and what is still expected to fail. Mbin and the others not yet in the pasture -are covered by unit tests written in their documents' shape. +`docs/INTEROP.md` has each peer's evidence and what is still expected to fail. The platforms not yet in the pasture are +covered by unit tests written in their documents' shape. ## Supported FEPs @@ -114,9 +115,13 @@ A group is either a **community** or a **circle**. from its own origin, never taken from the announce. They are taken from a group someone here follows, and also when they relay a vote on one of our posts in a thread of that group (Lemmy sends a vote to the community alone). A post keeps the group its `audience` names, however it arrived. +- A post in a remote group followed here that the group's own server delivers itself, as its author's `Create` with the + group as its `audience` (Mbin sends a magazine's threads that way, with no announce), is kept as if announced: the + group's server speaks for posts in it. The same from another server is not. - A remote community's **moderation** reaches the posts it holds. A removal (`Announce{Delete}`) is believed at once from a community on the post's own server, which speaks for it (PieFed keeps serving a thread its moderator removed), - and from a community elsewhere once the post's origin answers it gone. A lock (`Announce{Lock}`, or `commentsEnabled: false` on the post) refuses replies, + and from a community elsewhere once the post's origin answers it gone. A lock sent as it is (`Lock`, Mbin) is taken + from the post's own server only. A lock (`Announce{Lock}`, or `commentsEnabled: false` on the post) refuses replies, ours included, until `Undo{Lock}`. A ban of a persona (`Announce{Block}` with the community as `target`) shows as `blocked_by` on the community and refuses the persona's posts and replies there until the `Undo`. diff --git a/PrivaPub.Tests/Federation/CommunityEditsTests.cs b/PrivaPub.Tests/Federation/CommunityEditsTests.cs index 4b56b05..ca1ce41 100644 --- a/PrivaPub.Tests/Federation/CommunityEditsTests.cs +++ b/PrivaPub.Tests/Federation/CommunityEditsTests.cs @@ -147,6 +147,60 @@ namespace PrivaPub.Tests.Federation Assert.False(await DB.Default.Find().Match(p => p.ID == post.ID).ExecuteAnyAsync(token)); } + // Mbin sends a magazine's thread to its subscribers as the author's Create, and a moderator's lock as it is, with no + // announce: the magazine's server speaks for them, another server does not + [Fact] + public async Task A_thread_and_its_lock_that_a_followed_groups_own_server_sends_without_announcing_them_are_taken() + { + var token = TestContext.Current.CancellationToken; + var (root, alice) = await _harness.Persona("alice"); + var magazine = new RemoteActor(_harness.Peer, "books", type: "Group"); + var poster = new RemoteActor(_harness.Peer, "poster"); + var stranger = new RemoteActor(_harness.Peer, "stranger", _harness.Peer.B); + await _harness.Follows.Follow(root, new FollowForm { AvatarId = alice.Id, Target = magazine.Id }, token); + await DB.Default.Update().Match(f => f.AvatarId == alice.Id).Modify(f => f.State, FollowState.Accepted).ExecuteAsync(token); + JsonObject Page(RemoteActor author) => new() + { + ["id"] = $"{Origin(author)}/m/books/t/{Guid.NewGuid():N}", ["type"] = "Page", ["name"] = "a thread", + ["attributedTo"] = author.Id, ["content"] = "

in the magazine

", + ["to"] = new JsonArray(magazine.Id, Addressing.Public), ["cc"] = new JsonArray(author.Id + "/followers"), + ["audience"] = magazine.Id, ["published"] = DateTime.UtcNow.ToString("O") + }; + JsonObject Create(RemoteActor author, JsonObject page) => new() + { + ["id"] = $"{Origin(author)}/f/object/{Guid.NewGuid():N}", ["type"] = "Create", ["actor"] = author.Id, + ["to"] = page["to"]!.DeepClone(), ["cc"] = page["cc"]!.DeepClone(), ["object"] = page + }; + JsonObject Lock(RemoteActor moderator, string uri) => new() + { + ["id"] = $"{Origin(moderator)}/f/object/{Guid.NewGuid():N}", ["type"] = "Lock", ["actor"] = moderator.Id, ["object"] = uri + }; + var page = Page(poster); + var elsewhere = Page(stranger); + var uri = page["id"]!.GetValue(); + + await _harness.Deliver(poster, "/human-centipede", Create(poster, page)); + await _harness.Deliver(stranger, "/human-centipede", Create(stranger, elsewhere)); + var kept = await DB.Default.Find().Match(p => p.ObjectURI == uri).ExecuteSingleAsync(token); + await _harness.Deliver(stranger, "/human-centipede", Lock(stranger, uri)); + var lockedByStranger = (await DB.Default.Find().OneAsync(kept.ID, token)).LockedAt; + var moderator = new RemoteActor(_harness.Peer, "moderator"); + var locking = Lock(moderator, uri); + await _harness.Deliver(moderator, "/human-centipede", locking); + var locked = (await DB.Default.Find().OneAsync(kept.ID, token)).LockedAt; + await _harness.Deliver(moderator, "/human-centipede", new JsonObject + { + ["id"] = $"{Origin(moderator)}/f/object/{Guid.NewGuid():N}", ["type"] = "Undo", ["actor"] = moderator.Id, ["object"] = locking.DeepClone() + }); + + Assert.Equal(magazine.Id, kept.AudienceURI); + Assert.True(await DB.Default.Find().Match(e => e.PostId == kept.ID && e.AvatarId == alice.Id).ExecuteAnyAsync(token)); + Assert.False(await DB.Default.Find().Match(p => p.ObjectURI == elsewhere["id"]!.GetValue()).ExecuteAnyAsync(token)); + Assert.Null(lockedByStranger); + Assert.NotNull(locked); + Assert.Null((await DB.Default.Find().OneAsync(kept.ID, token)).LockedAt); + } + [Fact] public async Task A_fetched_object_does_not_wear_the_signature_of_the_activity_that_caused_the_fetch() { diff --git a/PrivaPub.Tests/Support/Harness.cs b/PrivaPub.Tests/Support/Harness.cs index be48288..2d6ab22 100644 --- a/PrivaPub.Tests/Support/Harness.cs +++ b/PrivaPub.Tests/Support/Harness.cs @@ -70,7 +70,8 @@ namespace PrivaPub.Tests.Support new DeleteHandler(Db, Local, Remote, Delivery, Groups, Quotes), new UpdateHandler(Db, Local, Remote, Groups, Records, Quotes, Delivery), new FlagHandler(Db, Local), - new BlockHandler(Db, Local) + new BlockHandler(Db, Local), + new LockHandler(Db) }; ((AnnounceHandler)Handlers.First(h => h is AnnounceHandler)).Relays = Handlers; Processor = new InboxProcessor(Remote, Handlers, NullLogger.Instance, Ledger, Local); diff --git a/PrivaPub/Federation/Inbox/Handlers/CreateHandler.cs b/PrivaPub/Federation/Inbox/Handlers/CreateHandler.cs index abb1f88..bea2399 100644 --- a/PrivaPub/Federation/Inbox/Handlers/CreateHandler.cs +++ b/PrivaPub/Federation/Inbox/Handlers/CreateHandler.cs @@ -174,7 +174,13 @@ namespace PrivaPub.Federation.Inbox.Handlers // a reply in the thread of someone followed here, as Mastodon keeps them: what the thread's server forwards var repliesToFollowed = !followed && parent is { IsFederatedCopy: true } && visibility is PostVisibility.Public or PostVisibility.Unlisted && await _dbEntities.Followings.Match(f => f.TargetActorURI == parent.ActorURI && f.State == FollowState.Accepted).ExecuteAnyAsync(token); - if (visibility == PostVisibility.Direct ? persons.Count == 0 : group == default && persons.Count == 0 && !repliesToLocal && !followed && !repliesToFollowed) + // a post in a remote group followed here that the group's own server sends itself rather than announcing it (Mbin + // delivers a magazine's threads to its subscribers as their author's Create): the group's server speaks for it + var inFollowedGroup = !followed && visibility is PostVisibility.Public or PostVisibility.Unlisted && note.Audience != default + && Origin.Same(note.Audience, author.ActorURI) && Origin.Same(note.Id, note.Audience) + && await _dbEntities.Followings.Match(f => f.TargetActorURI == note.Audience && f.State == FollowState.Accepted).ExecuteAnyAsync(token); + if (visibility == PostVisibility.Direct ? persons.Count == 0 + : group == default && persons.Count == 0 && !repliesToLocal && !followed && !repliesToFollowed && !inFollowedGroup) { Arrival.Drop("not-addressed"); return; diff --git a/PrivaPub/Federation/Inbox/Handlers/LockHandler.cs b/PrivaPub/Federation/Inbox/Handlers/LockHandler.cs new file mode 100644 index 0000000..aee82a5 --- /dev/null +++ b/PrivaPub/Federation/Inbox/Handlers/LockHandler.cs @@ -0,0 +1,50 @@ +using MongoDB.Entities; + +using PrivaPub.Federation.Objects; +using PrivaPub.Models.User; +using PrivaPub.StaticServices; + +using System.Text.Json.Nodes; + +using static PrivaPub.Federation.Objects.ActivityJson; + +using PostEntity = PrivaPub.Models.Post.Post; + +namespace PrivaPub.Federation.Inbox.Handlers +{ + // A moderator's lock of a thread sent as it is, not inside its community's announce (Mbin sends it to the magazine's + // subscribers itself): taken only from the post's own server, which speaks for it. Replies are refused until the + // Undo{Lock}, as for a lock a community announces. + public class LockHandler : IActivityHandler + { + readonly DbEntities _dbEntities; + + public LockHandler(DbEntities dbEntities) + { + _dbEntities = dbEntities; + } + + public string Type => "Lock"; + + public async Task Handle(JsonNode activity, ForeignAvatar actor, CancellationToken token) + { + if (await Set(_dbEntities, Id(activity["object"]), actor, locked: true, token)) + Arrival.Accept("locked"); + else + Arrival.Drop("unknown-object"); + } + + // whether the post is ours to lock or unlock for that actor, and was + public static async Task Set(DbEntities dbEntities, string objectUri, ForeignAvatar actor, bool locked, CancellationToken token) + { + if (objectUri == default || !Origin.Same(objectUri, actor.ActorURI)) + return false; + var post = await dbEntities.Posts.Match(p => p.ObjectURI == objectUri && p.IsFederatedCopy).ExecuteFirstAsync(token); + if (post == default) + return false; + Arrival.About(visibility: post.Visibility, created: post.CreationDate); + await DB.Default.Update().MatchID(post.ID).Modify(p => p.LockedAt, locked ? DateTime.UtcNow : null).ExecuteAsync(token); + return true; + } + } +} diff --git a/PrivaPub/Federation/Inbox/Handlers/UndoHandler.cs b/PrivaPub/Federation/Inbox/Handlers/UndoHandler.cs index 01e3b83..051d789 100644 --- a/PrivaPub/Federation/Inbox/Handlers/UndoHandler.cs +++ b/PrivaPub/Federation/Inbox/Handlers/UndoHandler.cs @@ -49,6 +49,8 @@ namespace PrivaPub.Federation.Inbox.Handlers undone |= await UndoDislike(inner, innerId, actor, token); if (innerType is null or "Block") undone |= await BlockHandler.Undo(inner, innerId, actor, _localActors, token); + if (innerType == "Lock") + undone |= await LockHandler.Set(_dbEntities, Id(inner["object"]), actor, locked: false, token); if (innerType is null or "Like" or "EmojiReact") undone |= await _reactions.Withdraw(actor, innerId, inner is JsonObject ? Id(inner["object"]) : default, inner is JsonObject ? Value(inner, "_misskey_reaction") ?? Value(inner, "content") : default, token); diff --git a/PrivaPub/Middleware/SocialPubConfigurations.cs b/PrivaPub/Middleware/SocialPubConfigurations.cs index 1f81b9d..b10cb3b 100644 --- a/PrivaPub/Middleware/SocialPubConfigurations.cs +++ b/PrivaPub/Middleware/SocialPubConfigurations.cs @@ -93,6 +93,7 @@ namespace PrivaPub.Middleware .AddSingleton() .AddSingleton() .AddSingleton() + .AddSingleton() .AddSingleton() .AddSingleton() .AddSingleton() diff --git a/docs/INTEROP.md b/docs/INTEROP.md index f8dbcb8..224d4bf 100644 --- a/docs/INTEROP.md +++ b/docs/INTEROP.md @@ -580,6 +580,22 @@ What it showed: - Private messages are `ChatMessage`. - The magazine outbox is empty. - Mbin also auto-ingests Mastodon posts by hashtag and Announces them. + - **A magazine's threads go to its subscribers as their author's `Create`**, `to` and `audience` naming the + magazine, never announced by it. PrivaPub dropped them as addressed to nobody here until 2026-10-05; a post whose + group is followed here and lives on the post's server is now kept (FEDERATION.md, Groups). + - **A moderator's lock is a bare `Lock`** (and `Undo{Lock}`) from the moderator, not inside an announce; taken since + 2026-10-05 from the post's own server. A removal ("trash") is a `Delete` from the moderator, believed once Mbin + answers the thread gone. + - **Votes:** an upvote is an `Announce`, a favourite a `Like`. Downvotes stay on Mbin (no `Dislike`), and taking an + upvote back sends nothing (its vote listener announces only the upvote), so a boost counted here stays. + - **Private messages only as `ChatMessage`:** a direct `Note` is dropped ("PM: not implemented", G-0008), and Mbin's + actors say nothing that would let a sender choose; its API never starts a conversation with a remote account. + - Mbin names what it makes during a request after the request's host, port included. + - **Pasture evidence (2026-10-05, Mbin 1.10.1, `tools/pasture/scenarios/mbin.sh`):** 24 checks pass and one gap is + expected (G-0008): magazines both ways; an Mbin thread in our community arrives titled and ours reaches Mbin as a + thread; a magazine's thread reaches alice's home and her Note becomes a microblog post in it; comments both ways; + Mbin's favourite counts as a like and its upvote as a boost, alice's like and boost count there; a moderator's lock, + unlock and removal; the unfollow; statistics. - **Gaps:** - **P1:** Lemmy's P1 set, W2, and tolerating `source` as a string. - **P2:** galleries; post `Move`; `repliesEnabled`; `nsfl`; flairs; publish `likes`/`shares` totals. diff --git a/docs/ROADMAP.md b/docs/ROADMAP.md index 8eb71d2..6ecc337 100644 --- a/docs/ROADMAP.md +++ b/docs/ROADMAP.md @@ -69,9 +69,11 @@ Written 2026-10-01 from the original 2023 code, the decePubClient UI, a federati Funkwhale serve it; Funkwhale's answers named after our follow and its deletions of several uploads. - GoToSocial's interaction policies both ways; personas join and leave remote events; third-party replies to a persona's posts passed on to its followers (owner decisions 2026-10-05). - - wave 2, under way: PieFed in the pasture with a scenario (2026-10-05). What it showed and was fixed: the instance - actor answers at the server's root, where PieFed looks for the inbox it announces to; a community's removal of a - post on its own server is believed at once. + - wave 2, under way: PieFed and Mbin in the pasture with scenarios (2026-10-05). What they showed and was fixed: the + instance actor answers at the server's root, where PieFed looks for the inbox it announces to; a community's removal + of a post on its own server is believed at once; a followed group's post its own server sends without announcing + it is kept, and a bare `Lock` from the post's server is taken (Mbin). Open: direct messages to Mbin, which takes + them only as `ChatMessage` (G-0008, waits for the owner). - [ ] P7 Threads, communities, moderation, the social graph - [ ] P8 Signatures, discovery, the long tail - [x] P9 Reading at volume (owner decision 2026-10-04, back from "Cut"): lists, server-side filters, scheduled posts, diff --git a/tools/pasture/Caddyfile b/tools/pasture/Caddyfile index 736407e..aff603b 100644 --- a/tools/pasture/Caddyfile +++ b/tools/pasture/Caddyfile @@ -97,3 +97,8 @@ piefed.test { tls internal reverse_proxy pasture-piefed:5000 } + +mbin.test { + tls internal + reverse_proxy pasture-mbin:80 +} diff --git a/tools/pasture/peers/mbin.sh b/tools/pasture/peers/mbin.sh new file mode 100644 index 0000000..282ee38 --- /dev/null +++ b/tools/pasture/peers/mbin.sh @@ -0,0 +1,104 @@ +# Mbin 1.10.1: the threadiverse in Symfony (magazines, threads, comments, microblog posts, votes up and down), from its +# own image: FrankenPHP serving plain HTTP behind Caddy, and a messenger worker for its queues, on the shared Postgres +# (database mbin) and Redis (db 12), with a RabbitMQ of its own (its transports carry AMQP options). Symfony's HTTP +# client trusts the system bundle, so the pasture's is mounted over it. Its admin is mbuser, made by its console; its +# API takes an OAuth2 token, which mbin_settle gets through the authorization-code flow as mbuser (a client-credentials +# client acts as a bot, which may not vote). +MBIN_IMAGE=${MBIN_IMAGE:-ghcr.io/mbinorg/mbin:v1.10.1} +MBIN_RABBITMQ_IMAGE=${MBIN_RABBITMQ_IMAGE:-docker.io/library/rabbitmq:4-alpine} +MBIN_PASSWORD=Mbin-Pasture-Pass-1 +. "$here/peers/shared.sh" + +mbin_env() { + local dir="$here/.state/mbin" + [ -s "$dir/secret" ] || head -c 32 /dev/urandom | od -An -tx1 | tr -d ' \n' > "$dir/secret" + [ -s "$dir/mercure" ] || head -c 32 /dev/urandom | od -An -tx1 | tr -d ' \n' > "$dir/mercure" + [ -s "$dir/oauth-key" ] || head -c 16 /dev/urandom | od -An -tx1 | tr -d ' \n' > "$dir/oauth-key" + cat </dev/null + openssl rsa -in "$here/.state/mbin/oauth2/private.pem" -passin "pass:$MBIN_PASSWORD" -pubout -out "$here/.state/mbin/oauth2/public.pem" 2>/dev/null + chmod 644 "$here/.state/mbin/oauth2/"*.pem + fi + mbin_env > "$here/.state/mbin/env" + # its API's own limits (two threads every six minutes) would throttle a scripted run: the same file, every limit raised + podman run --rm --entrypoint cat "$MBIN_IMAGE" config/packages/rate_limiter.yaml \ + | sed -E 's/^( +limit:) [0-9]+$/\1 100000/' > "$here/.state/mbin/rate_limiter.yaml" + # (as its own user on a volume it owns: started as root, it writes an .erlang.cookie it then cannot read) + podman volume exists pasture-mbin-rabbitmq || podman volume create --label pasture=1 pasture-mbin-rabbitmq >/dev/null + podman run -d --replace --name pasture-mbin-rabbitmq --network $net --label pasture=1 --user rabbitmq -v pasture-mbin-rabbitmq:/var/lib/rabbitmq:U \ + "$MBIN_RABBITMQ_IMAGE" >/dev/null + for _ in $(seq 1 60); do podman exec pasture-mbin-rabbitmq rabbitmq-diagnostics -q ping >/dev/null 2>&1 && break; sleep 2; done + podman volume exists pasture-mbin-media || podman volume create --label pasture=1 pasture-mbin-media >/dev/null + local common=(--network $net --label pasture=1 --env-file "$here/.state/mbin/env" -v pasture-mbin-media:/app/public/media + -v "$here/.state/mbin/oauth2:/oauth2:z,ro" -v "$ca/bundle.pem:/etc/ssl/certs/ca-certificates.crt:z,ro" + -v "$here/.state/mbin/rate_limiter.yaml:/app/config/packages/rate_limiter.yaml:z,ro") + podman run -d --replace --name pasture-mbin "${common[@]}" "$MBIN_IMAGE" >/dev/null + # the web container runs the migrations as it starts; the worker waits for them + for _ in $(seq 1 120); do + podman logs pasture-mbin 2>&1 | grep -q "PHP app ready" && break + sleep 2 + done + podman run -d --replace --name pasture-mbin-worker "${common[@]}" "$MBIN_IMAGE" \ + php bin/console messenger:consume scheduler_default old async outbox deliver inbox resolve receive failed --time-limit=86400 >/dev/null + for _ in $(seq 1 60); do + site mbin.test -s -o /dev/null -w '%{http_code}' https://mbin.test:6443/api/instance 2>/dev/null | grep -q 200 && break + sleep 2 + done + mbin_settle + echo "mbin: https://mbin.test:6443" +} + +mbin_console() { podman exec pasture-mbin php bin/console "$@"; } + +# mbuser (admin, verified), the instance's keys, and mbuser's OAuth token from the authorization-code flow +mbin_settle() { + mbin_console mbin:ap:keys:update >/dev/null 2>&1 || true + mbin_console mbin:user:create mbuser mbuser@mbin.test "$MBIN_PASSWORD" >/dev/null 2>&1 || true + mbin_console mbin:user:admin mbuser >/dev/null 2>&1 || true + mbin_console mbin:user:verify mbuser >/dev/null 2>&1 || true + python3 "$here/peers/mbin_token.py" "$MBIN_PASSWORD" "$here/.state/mbin/client.json" > "$here/.state/mbin.token" 2>"$here/.state/mbin/token.log" || true +} diff --git a/tools/pasture/peers/mbin_token.py b/tools/pasture/peers/mbin_token.py new file mode 100644 index 0000000..46fc5d6 --- /dev/null +++ b/tools/pasture/peers/mbin_token.py @@ -0,0 +1,101 @@ +"""mbuser's OAuth2 access token on the pasture's Mbin, through the authorization-code flow a person would follow: an +OAuth client made through the API, mbuser signed in through the login form, consent given, the code exchanged. +Prints the token. Usage: mbin_token.py """ +import http.client +import json +import os +import re +import socket +import ssl +import sys +import urllib.parse + +HOST = "mbin.test" +CA = os.path.join(os.path.dirname(os.path.abspath(__file__)), "..", ".ca", "root.crt") +REDIRECT = "https://pasture.invalid/callback" +SCOPES = "read write delete subscribe block vote report user moderate" +cookies = {} + + +def request(method, path, body=None, headers=None): + """One request to Mbin through Caddy on 127.0.0.1:6443, named mbin.test (SNI and Host); the CA is the pasture's own""" + context = ssl.create_default_context(cafile=CA) + conn = http.client.HTTPSConnection(HOST, 6443, context=context, timeout=60) + conn.sock = context.wrap_socket(socket.create_connection(("127.0.0.1", 6443), timeout=60), server_hostname=HOST) + # (Mbin names what it makes after the request's host: mbin.test, without the workstation's port) + headers = dict(headers or {}, Host=HOST) + if cookies: + headers["Cookie"] = "; ".join(f"{k}={v}" for k, v in cookies.items()) + conn.request(method, path, body=body, headers=headers) + response = conn.getresponse() + data = response.read().decode("utf-8", "replace") + for name, value in response.getheaders(): + if name.lower() == "set-cookie": + key, _, rest = value.partition("=") + cookies[key.strip()] = rest.split(";")[0] + return response.status, dict((k.lower(), v) for k, v in response.getheaders()), data + + +def follow(path): + """GETs path and follows redirects on Mbin, returning the last answer and where a redirect away from Mbin pointed""" + for _ in range(10): + status, headers, data = request("GET", path) + location = headers.get("location") + if status not in (301, 302, 303, 307, 308) or not location: + return status, data, None + if location.startswith(REDIRECT): + return status, data, location + parsed = urllib.parse.urlsplit(location) + path = parsed.path + ("?" + parsed.query if parsed.query else "") + raise SystemExit("too many redirects") + + +def csrf(html): + match = re.search(r'name="_csrf_token"\s+value="([^"]+)"', html) + if not match: + raise SystemExit("no CSRF token in the page") + return match.group(1) + + +def main(password, saved): + # one client for every run (Mbin limits how many are made) + client = json.load(open(saved)) if os.path.exists(saved) else None + if client is None: + status, _, data = request("POST", "/api/client", json.dumps({ + "name": "pasture", "contactEmail": "pasture@mbin.test", "description": "the pasture's scenarios", "public": False, + "redirectUris": [REDIRECT], "grants": ["authorization_code", "refresh_token"], "scopes": SCOPES.split() + }), {"Content-Type": "application/json"}) + if status >= 300: + raise SystemExit(f"client: {status} {data[:300]}") + client = json.loads(data) + json.dump({"identifier": client["identifier"], "secret": client["secret"]}, open(saved, "w")) + + status, _, page = request("GET", "/login") + form = urllib.parse.urlencode({"email": "mbuser", "password": password, "_csrf_token": csrf(page)}) + status, headers, _ = request("POST", "/login", form, {"Content-Type": "application/x-www-form-urlencoded"}) + if status not in (302, 303) or "/login" in headers.get("location", ""): + raise SystemExit(f"login refused: {status} {headers.get('location')}") + + query = urllib.parse.urlencode({"response_type": "code", "client_id": client["identifier"], "redirect_uri": REDIRECT, + "scope": SCOPES, "state": "pasture"}) + status, page, done = follow("/authorize?" + query) + if done is None: + consent = "/consent?" + query + form = urllib.parse.urlencode({"consent": "yes", "_csrf_token": csrf(page)}) + status, headers, _ = request("POST", consent, form, {"Content-Type": "application/x-www-form-urlencoded"}) + location = headers.get("location", "") + parsed = urllib.parse.urlsplit(location) + status, page, done = follow(parsed.path + "?" + parsed.query) + if done is None: + raise SystemExit(f"no code: {status} {page[:300]}") + code = urllib.parse.parse_qs(urllib.parse.urlsplit(done).query)["code"][0] + + form = urllib.parse.urlencode({"grant_type": "authorization_code", "client_id": client["identifier"], + "client_secret": client["secret"], "redirect_uri": REDIRECT, "code": code}) + status, _, data = request("POST", "/token", form, {"Content-Type": "application/x-www-form-urlencoded"}) + if status >= 300: + raise SystemExit(f"token: {status} {data[:300]}") + print(json.loads(data)["access_token"]) + + +main(sys.argv[1], sys.argv[2]) diff --git a/tools/pasture/scenarios/mbin.sh b/tools/pasture/scenarios/mbin.sh new file mode 100644 index 0000000..c14ad07 --- /dev/null +++ b/tools/pasture/scenarios/mbin.sh @@ -0,0 +1,113 @@ +# Mbin 1.10.1: the threadiverse in Symfony. Magazines both ways (FEP-1b12), threads with titles, a Note addressed to a +# magazine as one of its microblog posts, comments both ways, favourites (Like) and upvotes (Announce) both ways (Mbin +# sends no downvote), a direct message (G-0008), a moderator's lock and removal, the unfollow, statistics. Mbin is driven +# through its API as mbuser, with the token peers/mbin_token.py got through the authorization-code flow. +MB=https://mbin.test:6443 +MT=$(cat "$here/.state/mbin.token" 2>/dev/null) +# mb [json]: an Mbin API call as mbuser. Mbin names what it makes during a request after the request's +# host, so the call says mbin.test, without the workstation's port +mb() { + local method=$1 path=$2 body=${3:-} + if [ -n "$body" ]; then + site mbin.test -s -X "$method" "$MB/api/$path" -H 'Host: mbin.test' -H "Authorization: Bearer $MT" -H 'Content-Type: application/json' -d "$body" + else + site mbin.test -s -X "$method" "$MB/api/$path" -H 'Host: mbin.test' -H "Authorization: Bearer $MT" + fi +} +mb_sql() { podman exec pasture-postgres psql -U pasture -d mbin -tAc "$1" 2>/dev/null; } +# what Mbin finds for a handle or a URL (search/v2 resolves it): the first of its kind, as JSON +mb_resolve() { mb GET "search/v2?q=$(python3 -c 'import sys, urllib.parse; print(urllib.parse.quote(sys.argv[1]))' "$1")" | j "print(json.dumps(next((r['$2'] for r in d.get('apResults', []) if r.get('$2')), None)))"; } +p_home_has() { curl -s -H "$PH" "$P/api/v1/timelines/home?limit=40" | j "print(next(((s.get('reblog') or s)['id'] for s in d if '$1' in ((s.get('reblog') or s)['content'] or '') or '$1' in (((s.get('reblog') or s).get('privapub') or {}).get('title') or '')), ''))"; } + +echo "mbin" +[ -n "$MT" ] && [ "$(mb GET users/me | j "print(d['username'])")" = "mbuser" ] && ok "Mbin token for mbuser" || { ko "Mbin token"; return 1; } +PT=$(privapub_token alice_mbin) +PH="Authorization: Bearer $PT" +[ -n "$PT" ] && ok "PrivaPub token for alice_mbin" || { ko "PrivaPub token for alice_mbin"; return 1; } +jwt=$(privapub_root) +alice_id=$(curl -s -H "$PH" "$P/api/v1/accounts/verify_credentials" | j "print(d['id'])") +run=$(date +%s) +dogs="mbdogs$run" +books="books$run" + +echo " magazines" +dogs_gid=$(curl -s -X POST $P/clientapi/group/insert -H 'Content-Type: application/json' -H "Authorization: Bearer $jwt" \ + -d "{\"avatarId\":\"$alice_id\",\"userName\":\"$dogs\",\"name\":\"Pasture dogs\",\"description\":\"a PrivaPub community\",\"isCommunity\":true}" | j "print(d['id'])") +dogs_on_mb=$(mb_resolve "!$dogs@privapub.test" magazine | j "print(d['magazineId'])") +[ -n "$dogs_on_mb" ] && ok "Mbin resolves a PrivaPub community as a magazine" || ko "Mbin cannot resolve the PrivaPub community" +dogs_on_p=$(curl -s -H "$PH" "$P/api/v1/accounts/lookup?acct=$dogs" | j "print(d['id'])") +mb PUT "magazine/$dogs_on_mb/subscribe" >/dev/null +until_true 45 '[ "$(curl -s -H "$PH" "$P/api/v1/accounts/$dogs_on_p" | j "print(d[\"followers_count\"])")" = "1" ]' \ + && ok "mbuser follows the PrivaPub community" || ko "Mbin's follow of the PrivaPub community never arrived" +podman exec pasture-mbin php bin/console mbin:magazine:create "$books" --owner=mbuser >/dev/null 2>&1 +books_on_mb=$(mb GET "magazine/name/$books" | j "print(d['magazineId'])") +books_on_p=$(curl -s -H "$PH" "$P/api/v2/search?q=@$books@mbin.test&resolve=true&type=accounts" | j "print(d['accounts'][0]['id'])") +[ -n "$books_on_p" ] && ok "PrivaPub resolves an Mbin magazine" || ko "PrivaPub cannot resolve the Mbin magazine" +curl -s -o /dev/null -X POST -H "$PH" "$P/api/v1/accounts/$books_on_p/follow" +until_true 45 '[ "$(curl -s -H "$PH" "$P/api/v1/accounts/relationships?id[]=$books_on_p" | j "print(d[0][\"following\"])")" = "True" ]' \ + && ok "alice follows the Mbin magazine (Accept arrived)" || ko "the magazine's Accept never arrived" + +echo " threads" +mb POST "magazine/$dogs_on_mb/article" "{\"title\":\"An Mbin thread $run\",\"body\":\"posted from Mbin into PrivaPub\",\"lang\":\"en\"}" >/dev/null +until_true 45 '[ "$(curl -s -H "$PH" "$P/api/v1/accounts/$dogs_on_p/statuses" | j "print(any((s.get(\"privapub\") or {}).get(\"title\")==\"An Mbin thread $run\" for s in d))")" = "True" ]' \ + && ok "an Mbin thread in the PrivaPub community arrives with its title" || ko "Mbin's thread missing or untitled on PrivaPub" +curl -s -o /dev/null -X POST $P/clientapi/post/insert -H 'Content-Type: application/json' -H "Authorization: Bearer $jwt" \ + -d "{\"avatarId\":\"$alice_id\",\"title\":\"A PrivaPub community thread $run\",\"text\":\"posted into our own community for Mbin\",\"groupId\":\"$dogs_gid\"}" +until_true 45 '[ "$(mb GET "magazine/$dogs_on_mb/entries?sort=newest" | j "print(any(e[\"title\"]==\"A PrivaPub community thread $run\" for e in d.get(\"items\", [])))")" = "True" ]' \ + && ok "a titled post in the PrivaPub community reaches Mbin as a thread" || ko "PrivaPub community post missing on Mbin" +books_thread=$(mb POST "magazine/$books_on_mb/article" "{\"title\":\"Books only $run\",\"body\":\"an Mbin magazine thread\",\"lang\":\"en\"}" | j "print(d['entryId'])") +until_true 45 '[ -n "$(p_home_has "Books only $run")" ]' && ok "the magazine's announce brings its thread to alice's home" || ko "Mbin magazine thread missing from alice's home" +books_thread_on_p=$(p_home_has "Books only $run") +p_note=$(curl -s -X POST -H "$PH" $P/api/v1/statuses --data-urlencode "status=@$books@mbin.test a PrivaPub note for the magazine $run" -d 'visibility=public') +p_note_id=$(echo "$p_note" | j "print(d['id'])") +mb_post_of() { mb GET "magazine/$books_on_mb/posts?sort=newest" | j "print(next((p['postId'] for p in d.get('items', []) if '$1' in (p.get('body') or '')), ''))"; } +until_true 45 '[ -n "$(mb_post_of "a PrivaPub note for the magazine $run")" ]' && ok "alice's note lands in the magazine as a microblog post" || ko "PrivaPub note missing from the Mbin magazine" +p_note_on_mb=$(mb_post_of "a PrivaPub note for the magazine $run") + +echo " comments" +mb POST "posts/$p_note_on_mb/comments" "{\"body\":\"an Mbin comment on PrivaPub $run\",\"lang\":\"en\"}" >/dev/null +until_true 45 '[ "$(curl -s -H "$PH" "$P/api/v1/statuses/$p_note_id/context" | j "print(any(\"an Mbin comment on PrivaPub\" in s[\"content\"] for s in d[\"descendants\"]))")" = "True" ]' \ + && ok "Mbin's comment threads under alice's note" || ko "Mbin's comment missing on PrivaPub" +curl -s -o /dev/null -X POST -H "$PH" $P/api/v1/statuses -d "status=a PrivaPub comment on Mbin $run&in_reply_to_id=$books_thread_on_p&visibility=public" +until_true 45 '[ "$(mb GET "entry/$books_thread/comments" | j "print(any(\"a PrivaPub comment on Mbin\" in (c.get(\"body\") or \"\") for c in d.get(\"items\", [])))")" = "True" ]' \ + && ok "alice's reply becomes a comment on the Mbin thread" || ko "PrivaPub reply missing as an Mbin comment" + +echo " votes" +p_counts() { curl -s -H "$PH" "$P/api/v1/statuses/$p_note_id" | j "print(d['favourites_count'], d['reblogs_count'], ((d.get('privapub') or {}).get('votes') or {}).get('down', 0))"; } +mb PUT "post/$p_note_on_mb/favourite" >/dev/null +until_true 45 '[ "$(p_counts | cut -d" " -f1)" = "1" ]' && ok "Mbin's favourite counts as a like on PrivaPub" || ko "Mbin's favourite not counted ($(p_counts))" +mb PUT "post/$p_note_on_mb/vote/1" >/dev/null +until_true 45 '[ "$(p_counts | cut -d" " -f2)" = "1" ]' && ok "Mbin's upvote arrives as a boost" || ko "Mbin's upvote not counted ($(p_counts))" +# (Mbin keeps downvotes to itself, and sends nothing when an upvote is taken back: its vote listener only announces an +# upvote, VoteHandleSubscriber::onVote) +curl -s -o /dev/null -X POST -H "$PH" "$P/api/v1/statuses/$books_thread_on_p/favourite" +until_true 45 '[ "$(mb GET "entry/$books_thread" | j "print(d[\"favourites\"])")" = "1" ]' \ + && ok "alice's like is a favourite on Mbin" || ko "alice's like not counted on Mbin ($(mb GET "entry/$books_thread" | j "print(d[\"favourites\"], d[\"uv\"])"))" +curl -s -o /dev/null -X POST -H "$PH" "$P/api/v1/statuses/$books_thread_on_p/reblog" +until_true 45 '[ "$(mb GET "entry/$books_thread" | j "print(d[\"uv\"])")" = "1" ]' \ + && ok "alice's boost is an upvote on Mbin" || ko "alice's boost not counted on Mbin ($(mb GET "entry/$books_thread" | j "print(d[\"favourites\"], d[\"uv\"])"))" + +echo " private messages" +# Mbin takes a private message only as a ChatMessage, and drops alice's direct Note ("PM: not implemented", G-0008); its +# API never starts a conversation with an account elsewhere, so nothing comes the other way through it +curl -s -o /dev/null -X POST -H "$PH" $P/api/v1/statuses -d "status=@mbuser@mbin.test a secret from PrivaPub $run&visibility=direct" +until_true 30 '[ "$(mb_sql "select count(*) from message where body like '"'%a secret from PrivaPub $run%'"'")" = "1" ]' \ + && ok "alice's DM arrives as an Mbin private message" || xf "alice's DM never reaches Mbin, which takes private messages only as ChatMessage (G-0008)" + +echo " moderation" +p_locked() { curl -s -H "$PH" "$P/api/v1/statuses/$books_thread_on_p" | j "print((d.get('privapub') or {}).get('locked'))"; } +mb PUT "moderate/entry/$books_thread/lock" >/dev/null +until_true 45 '[ "$(p_locked)" = "True" ]' && ok "a moderator's lock reaches PrivaPub" || ko "a moderator's lock did not reach PrivaPub" +mb PUT "moderate/entry/$books_thread/lock" >/dev/null +until_true 45 '[ "$(p_locked)" = "False" ]' && ok "a moderator's unlock reaches PrivaPub" || ko "a moderator's unlock did not reach PrivaPub" +mb PUT "moderate/entry/$books_thread/trash" >/dev/null +until_true 45 '[ "$(curl -s -o /dev/null -w "%{http_code}" -H "$PH" "$P/api/v1/statuses/$books_thread_on_p")" = "404" ]' \ + && ok "a moderator's removal reaches PrivaPub" || ko "a moderator's removal did not reach PrivaPub" + +echo " unfollow" +curl -s -o /dev/null -X POST -H "$PH" "$P/api/v1/accounts/$books_on_p/unfollow" +until_true 45 '[ "$(mb_sql "select count(*) from magazine_subscription s join \"user\" u on u.id = s.user_id where u.ap_profile_id like '"'%alice_mbin%'"' and s.magazine_id = $books_on_mb")" = "0" ]' \ + && ok "alice's unfollow reaches the Mbin magazine" || ko "the Mbin magazine still counts alice" + +echo " statistics" +stats_check mbin.test mbin diff --git a/tools/pasture/town/gaps.json b/tools/pasture/town/gaps.json index a395ea6..5a2a841 100644 --- a/tools/pasture/town/gaps.json +++ b/tools/pasture/town/gaps.json @@ -107,5 +107,19 @@ "opened": "2026-10-05", "status": "open", "note": "A delivered DM is handled as one. Resolving its address (a recipient pasting its link; the town's driver does it to reply) makes Pixelfed's instance actor fetch it, which PrivaPub allows since a recipient lives there, and the copy is stored with scope private." + }, + { + "id": "G-0008", + "title": "A direct message to an Mbin account never arrives: Mbin takes a private message only as a ChatMessage", + "match": { + "feature": "deliver\\.direct", + "observer": "mbin" + }, + "kind": "server", + "phase": "P3", + "code": "mbin 1.10.1 src/Service/ActivityPub/ActivityPubContent.php getVisibility(): a Note neither public nor to the author's followers throws 'PM: not implemented'; only CreateHandler's ChatMessage branch makes a private message", + "opened": "2026-10-05", + "status": "open", + "note": "Mbin's actors advertise nothing that says so, and PrivaPub never decides by a server's software name, so choosing ChatMessage for a recipient waits for the owner. Mbin's API also never starts a conversation with an account elsewhere, so nothing comes the other way through it." } ]