Owner decision 2026-10-04: fix the mismatches and every other mismatch of the same kind.
- One counting rule (Domain/Privacy/Counted), Mastodon's. It is used for a persona's statuses_count, its outbox
totalItems, NodeInfo localPosts and the instance status_count, which used to count four different things. It
counts every post that is neither deleted nor a DM, boosts included, and circle and located posts too (owner
decision). A group's count includes its remote members' posts.
- Users. Personas of banned or deleted roots no longer count, and are not found in search. NodeInfo now gives
activeMonth and activeHalfyear, and the v2 instance gives active_month instead of a constant 0.
- replies_count counts only public and unlisted replies, so it no longer tells anyone that a private reply exists.
Migration _012 recounts it.
- A remote account that deletes itself takes everything out of every count (GoneActors): its likes, downvotes,
reactions and poll votes go and their counters come back, as do its boosts', replies' and quotes' counts, and its
notifications. Lookups, account lists, search and favourited_by no longer show it. Migration _012 applies this to
accounts already gone.
- Deleting a post also deletes its pins and the local boosts of it.
- /stalking gives the same total as following_count. Members are still never listed, and hide_collections is now
always true, since the setting never did anything.
- Joining a community by invitation is following it, so /flock and /groupies agree; leaving unfollows.
- Search. Anyone may search, as on Mastodon; resolve and offset need a sign-in, offset pages, and deleted accounts
are never found.
- notifications/unread_count counts what the list shows, and the owner's follower and following lists page with
Link.
- The instance API advertises what is enforced:
- max_characters, now enforced with a 422;
- max_pinned_statuses = MaxPins;
- the media types and limits MediaService and MediaOptions accept;
- PollService's limits;
- the configured languages;
- no streaming URL until streaming exists.
domain_count counts the servers we have exchanged with; which ones stays unpublished (peers is empty).
- Routes Mastodon answers now answer instead of 404:
- directory, tags/{name}, timelines/link and identity_proofs;
- instance/languages, translation_languages, domain_blocks and privacy_policy;
- the v1 and v2 notification policy, and notification requests.
Also, from phase 3: a recovered password ends /clientapi sessions through a per-root SessionStamp claim instead of
comparing the JWT's whole-second nbf with the change time. That comparison let a token issued in the same second
survive, which made a test flaky.
671 tests pass.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ELjqpznMFMNrJoJUj6K5p2
The v1.19.0 deploy's tests failed where build.yml's passed. A test stored a public remote post with no author id, and
when the public timeline test ran after it, MastodonMapper.Statuses looked that null up in its accounts and answered
500. The mapper now skips such a post, and the test stores a real author.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ELjqpznMFMNrJoJUj6K5p2
Owner decision 2026-10-04: SecureMode on once the pasture passes with it.
- Both clean pasture passes were run over all six peers:
- normally: 246 passed, 0 failed;
- with Federation__SecureMode=true: every federation check passed. The only failures were four checks expecting an
unsigned GET to get 404 or 410 where SecureMode answers 401. Those checks now go through `unserved` and
`gone_unsigned` (lib/interop.sh), which expect 401 when SecureMode is on.
- Circle posts now reach their member on GoToSocial and Mastodon, and survive Mastodon's signed refetch, as does a
followers-only post. The GoToSocial expected failure is gone.
- appsettings.Production.json turns SecureMode on.
- The deploy now checks that an unsigned GET of @thepra answers 401 and that a browser is redirected. It reads
@thepra's discoverability through the Mastodon API, since the actor is no longer readable unsigned.
- docs/INTEROP.md (Mastodon, GoToSocial), CLAUDE.md and ROADMAP updated.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ELjqpznMFMNrJoJUj6K5p2
Owner decision 2026-10-04: fix the account privacy findings.
- Sign-in. Every failure answers "That username and password do not match." after the same work: an unknown login
is hashed against a decoy, and the comparison is constant-time. "Banned" is told only to someone who gave the right
password. This covers /clientapi/user/login, /invitation/login and /oauth/login.
- Recovery.
- Every request answers the same sentence and queues a SendRecovery job, whether or not the account exists or has an
email. The lookup, the code and SMTP move to RecoveryJob, so neither the answer nor its timing says anything.
- Codes are kept only as a SHA-256 hash, for one hour. Migration _011 drops the plaintext ones, which never expired.
- A recovered password ends every session of the root. RootSessions sets CredentialsChangedAt, which JwtEvents
checks against the JWT's issue time, now stamped as nbf, and revokes each persona's OAuth tokens and authorizations.
- Deleting a root (RootRemoval: the admin route, or the restored self-delete at /clientapi/user/delete, which asks for
the password).
- Its sessions end.
- Each persona and each group it owns sends Delete{Actor} to its followers, its members and the accounts it follows.
- The personas' posts are emptied.
- /peasants/{name} answers 410 with a Tombstone (formerType Person or Group), as do its inbox and WebFinger, through
LocalActorService.Gone. The names stay reserved.
- The root keeps only a unique `deleted-{id}` name; the second deletion on an instance used to collide on
"Deleted user".
Also, from phase 2's pasture: GoToSocial files a circle post like a DM and shows it only to accounts it mentions. Each
member's copy, and a member's refetch, now also mentions that member silently. The GoToSocial scenario checks circle
posts in conversations, like DMs, and they pass there now, as on Mastodon.
657 tests pass.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ELjqpznMFMNrJoJUj6K5p2
Circles (owner decision 2026-10-04: fix them for compatibility):
- Mastodon 4.7 and GoToSocial drop a post that names none of their accounts, and a circle post named only the circle
and its /flock. OutboxPublisher.Publish now sends each member a copy that also names that member in `cc`, on the
activity and on the object, and names no other member. The Create, every Update (edit, poll, quote approval, policy,
through the new PublishUpdate) and the Delete (StatusService.Remove now uses Publish) all go that way.
- UpdateOf renders with the post's group, so an Update keeps a circle post's `audience` and a community post's `Page`
and title.
- A reply to a circle post stays in the circle, whichever client wrote it.
- A circle post can no longer quote a post that needs permission: asking would show the circle post to its author.
Posts that are not public, on refetch (SignedFetchAuthorizer.MayRead):
- Followers-only, direct and circle posts are served to a signed request from someone they were for, or from the
instance actor of a server where one of them lives. That is a follower or an addressed account, an addressed
account, or a member. Everyone else still gets 404.
- Once deleted they answer those readers 410. Mastodon deletes its copy when a refetch answers 404.
- A circle refetch names the requesting member, or the members on the requesting server, as the delivered copy did.
- /grunts/create-{id} serves the same.
- /peasants/{name}/whispers/{id}, a DM's `context`, was never routed. It is now the conversation's posts, for its
participants only.
SecureMode lets browsers through to the redirect to the public page, instead of answering them 401.
653 tests pass.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ELjqpznMFMNrJoJUj6K5p2
Owner decisions (2026-10-04, recorded in docs/ROADMAP.md): production runs everything that is built, and nothing waits
on a person running a command.
- Geolocation updates itself. GeoUpdater, a hosted service, checks daily whether each DB-IP Lite database was built this
month. If not, it fetches this month's, or last month's early in the month. It installs a file only once it opens as
the right kind of database, then swaps it in atomically, and the locator reloads at once. Lookups now run under the
lock, so a reload can no longer dispose a reader mid-lookup. The systemd timer, its script and their setup.sh lines
are gone: the root step they needed never happened, and none is needed now. /stargazing names the database in use.
- The admin CLI runs after the app is built, with every service and nothing started.
- `create-root <login> [--admin]` takes the password on stdin; it is how the first login is made while sign-up is
closed.
- `smoke <persona>` keeps the root `deploy-smoke` and an undiscoverable persona, and gives the root a new password
on every run.
- The deploy signs in as @thepra. It runs the CLI, gets a token through the real OAuth flow (tools/smoke/oauth.sh,
moved out of the pasture's privapub_token, which now uses it), checks the signed-in API and that @thepra is
undiscoverable, then revokes the token. PRIVAPUB_SMOKE_TOKEN is gone.
- The deploy also fails when:
- NodeInfo and the instance API disagree about registrations;
- /stargazing does not say the crawler is on;
- the geolocation databases are missing or more than 40 days old.
- The crawler is on in production, seeded with ten large servers of different kinds. FEDERATION.md now describes it
and how to opt out.
- One registrations switch (Registrations:Mode, default Invitations; Open in tests and the pasture). It is read by
open sign-up (403 when closed), NodeInfo `openRegistrations`, and v1 and v2 of the instance API, so they can no longer
disagree. Before, NodeInfo said open and the instance API said closed. Group invitations always work, so
invites_enabled is true.
- A persona edit through /clientapi no longer resets what the Mastodon API set (discoverable, locked, quote policy…):
the theme is merged into the settings instead of replacing them.
650 tests pass. The deploy's smoke step was rehearsed against the pasture's PrivaPub.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ELjqpznMFMNrJoJUj6K5p2
Sharkey 2025.4.7 is the Misskey peer under another name, image, database, Redis db and
home. Its scenario is Misskey's plus edits both ways and the FEP-e232 quote tag: 40 checks
pass.
Akkoma 3.20.1 publishes no image, so tools/pasture/images/akkoma installs its OTP
release, pinned by checksum, and appends Caddy's CA to the CA bundles the release ships.
Its scenario has 44 checks, which pass three runs in a row:
- follows, posts, CW, followers-only posts and the published time;
- replies, likes, boosts and their undos;
- EmojiReact both ways and withdrawn;
- DMs, polls, quotes, media, edits with history and deletes, both ways;
- unfollow, block, unblock and statistics.
The two bugs, both fixed:
- Followers-only posts arrived as DMs on Pleroma and Akkoma. They call a post private
only if an address in `to` contains "/followers" or its cc is not empty. Ours is
/groupies, and a post mentioning nobody had an empty cc. The followers collection is now
named in cc as well, which tells nobody anything new.
- Akkoma's open polls showed as ended and refused votes. Akkoma carries an open poll's
end in `closed` and sends no `endTime`. A `closed` in the future is now read as the end.
Neither of these is a PrivaPub bug:
- Akkoma's Linkify never takes @user@host.test for a mention, so its DM addresses alice
with to[].
- Its API never reports a remote blocker as blocked_by, so the block is read from its
database.
Also in this commit:
- The rate limits are configuration (RateLimits: AccountsPerMinute, InboxBurst,
InboxPerTenSeconds), with the old values as defaults. The pasture raises the accounts
limit, which back-to-back runs from one address had hit.
- A new Lemmy never sends what it queued for a server before it started that server's
send worker, so the scenario waits for the worker before its first follow.
All six peers in one clean pass: GoToSocial 54 (+1 expected), Mastodon 49 (+2), Misskey
35, Sharkey 40, Akkoma 44, and Lemmy 20 (+3) once the worker wait was added. 642 tests
pass.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ELjqpznMFMNrJoJUj6K5p2
peers/lemmy.sh runs the Lemmy 1.0.0-beta.2 backend on the shared Postgres. It trusts
Caddy's CA through SSL_CERT_FILE and reaches the pasture through
DANGER_FEDERATION_ALLOW_LOCAL_IP. Lemmy 0.19 cannot join: its rustls trusts only its
bundled roots. LEMMY_LOG sets RUST_LOG.
scenarios/lemmy.sh drives Lemmy through its v4 API. 20 checks pass, three runs in a row:
- communities both ways;
- a titled thread each way, and alice's mention of a Lemmy community becoming a thread
there;
- the Lemmy community's announce reaching alice's home;
- comments both ways and alice's like as an upvote;
- private messages both ways;
- statistics.
Two expected failures: votes, which Lemmy sends only through the community as
Announce{Like|Dislike}, and a moderator's removal. Both belong to P7.
What it showed, in docs/INTEROP.md:
- Lemmy 1.0 keeps its user's thread in a remote community pending until the community
announces it back. It clears the flag before answering that echo 400 ("Object is not
remote"). Leaving the author's server out of the Announce, as tried here, left every
such thread pending, so GroupDistributor now says why the echo stays.
- Every bare Announce{object} is answered 400, as Lemmy answers its own compatibility
Announce(Page).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ELjqpznMFMNrJoJUj6K5p2
peers/misskey.sh runs Misskey on the shared Postgres and Redis. Its config is generated with
the pasture's private networks allowed and a setup password, it trusts Caddy's CA through
NODE_EXTRA_CA_CERTS, and the first account it makes is the scenario's user. A new Misskey
federates with nobody, so the setup also turns federation on.
scenarios/misskey.sh adds 35 checks, all passing, as listed in docs/INTEROP.md. They cover
posts, CW, MFM source, replies, unicode reactions both ways and their withdrawal, likes as
reactions, legacy quotes both ways, polls, specified notes, media, deletes, unfollow,
block and statistics. MISSKEY_NAME and MISSKEY_IMAGE are there so Sharkey can reuse the peer.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ELjqpznMFMNrJoJUj6K5p2
The GoToSocial scenario gains 17 checks:
- alice's reply threads under gtsuser's post;
- gtsuser's edit arrives with edited_at and two history entries;
- unlike and unboost both ways;
- images with alt text both ways, theirs through our proxy;
- gtsuser follows a PrivaPub community and its announce brings the post;
- gtsuser's request to join a circle waits for the owner and is approved, and the
circle post is never served unsigned;
- a locked persona holds gtsuser's follow, rejects it, then authorizes it;
- the deploy's Mastodon smoke check passes, signed in.
54 checks passed, three runs in a row. The circle post reaching gtsuser is an expected
failure: GoToSocial keeps no post addressed only to a collection it does not know.
It found a bug. An edit made within the second the post was published carries GoToSocial's
whole-second updated == published, and IsEdit wanted strictly newer, so the edit was taken
as a refresh and lost. A first edit now also counts when it is no older and the text,
warning or title actually changed. A bare refresh still never makes a revision.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ELjqpznMFMNrJoJUj6K5p2
peers/mastodon.sh runs Mastodon's web and sidekiq containers on a shared Postgres and Redis
(peers/shared.sh). They trust Caddy's CA through SSL_CERT_FILE and reach private addresses
through ALLOWED_PRIVATE_ADDRESSES. Its users and tokens come from tootctl and rails runner.
scenarios/mastodon.sh adds 49 checks, as listed in docs/INTEROP.md: follows, posts,
replies, likes and boosts with undos, DMs, polls, FEP-044f quotes both ways, media through
the proxy, edits, deletes, locked follows, a circle request, reports, blocks and statistics.
Two are expected failures:
- inbound Block (P7);
- circle posts. Mastodon 4.7 loses the recipient of deliveries to its numeric
/ap/users/<id>/inbox and then drops a post that names no local account. The fix on our
side changes what a circle reveals, so it waits for the owner.
GoToSocial and Mastodon together: 86 passed, 0 failed.
The pasture now copies Caddy's root certificate reliably, readable by the peers, and
rebuilds the bundle each time. The CA directory is mounted shared (:z), because a private
:Z label locks out every container but the last. pfetch reaches PrivaPub's own https URIs
through Caddy. PRIVAPUB_ENV passes settings to PrivaPub, which scenarios/crawler.sh uses to
check the opt-in crawler against Mastodon: it visits, describes and reads the peers list.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ELjqpznMFMNrJoJUj6K5p2
Off by default (Statistics:Crawler:Enabled), as the owner decided. When it is on:
- CrawlPlan runs hourly, from StatisticsSchedule. It inserts the configured seeds and
queues up to HostsPerHour servers whose last visit is older than RevisitDays, are not
paused by the breaker and are not domain-blocked, spread across the hour.
- CrawlInstance visits one server at a time as
"PrivaPub-Stargazer/<ref> (+<base>/stargazing)":
- it reads robots.txt (RFC 9309: its own group first, then PrivaPub, then *; longest
rule wins; a 4xx allows everything; a 5xx or no answer keeps it out);
- it describes servers that only crawling ever found, through
InstanceDescriber.Describe with robots.txt as the path filter;
- it reads /api/v1/instance/peers through the new GetStringArray, which keeps what it
read from the first 1 MB instead of refusing a large list;
- it adds the names a server could ever be reached at as "crawled": DNS only,
punycode, no addresses, ports or hidden services, and the reserved test names only on
a test network. Never more than MaxNewHostsPerCrawl per visit or MaxHosts in all, and
never over a touched server.
- It reads nothing but robots.txt, NodeInfo, the instance API and the peers list.
IFederationHttp.GetText serves robots.txt, and HttpScope.Crawl carries the
User-Agent.
- /stargazing explains all this and how to keep the crawler out, says whether it is on,
and credits DB-IP. GET /clientapi/admin/statistics/crawler shows the frontier.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ELjqpznMFMNrJoJUj6K5p2
- Touches: the ledger marks a server as touched when it sends us a verified activity, when
we exchange activities with it, or when we read its actors, keys, objects or WebFinger.
It upserts RemoteInstance.Seen, FirstSeenAt and LastSeenAt at most hourly per server, and
queues one DescribeInstance a week with the same dedupe key ObjectRecords uses. Suspended
servers and pages behind link previews are never described. Migration _010 marks the
servers already known as touched, with their dates.
- InstanceDescriber.Describe(host, crawled, allowed) reads:
- NodeInfo 2.2/2.1/2.0, now with its published user counts, posts, comments,
description, languages and schema version;
- for software with a Mastodon API, /api/v2/instance falling back to v1: title,
languages, registration mode, character limit, API version, source URL.
It never keeps a contact as a field; the raw document is kept for the admin only. It
locates the server from the address our connection reached (DB-IP Lite city and ASN, the
CDN named when fronted) and writes a RemoteInstanceSnapshot per ISO week, unreachable
weeks included. A crawled server is upserted as crawled only on insert, so it never
downgrades a touched one, and robots.txt can deny any path.
- PublicGeo.Project is the only public form of a location: a CDN-fronted server shows its
CDN only, a server reporting at least ten users shows its city, coordinates and network,
any other only its country.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ELjqpznMFMNrJoJUj6K5p2
88 integration tests drive the Mastodon client API through the whole server
(PrivaPubHost), with remote actors on an in-process Peer and deliveries read
from the job queue. Helpers live in Support/Host/MastodonHelpers.cs.
Coverage:
- Accounts: verify_credentials (no root id or login name); update_credentials
with indexed and array fields_attributes (form and JSON), source[*],
quote_policy, locked/bot, avatar and header uploads resized and stripped of
EXIF and XMP; lookup (local, @domain, remote; a circle, the instance actor
and a circle's id answer 404); search with and without resolve (only a
signed-in persona resolves, the Peer is untouched otherwise), by post and
actor address, hashtags, undiscoverable personas; account statuses with
pinned, exclude_replies, exclude_reblogs, only_media, tagged and Link paging
both ways; followers-only posts for followers (local and remote authors);
community accounts; followers/following only to their owner; follow (open,
locked, remote Follow delivery), unfollow and Undo; follow requests from
local and remote followers answered with the original Follow;
remove_from_followers; blocks with Reject and Block/Undo deliveries; mutes
with duration and the notifications choice, never federated; domain blocks;
relationships with junk ids; a banned login's tokens; reports forwarded as a
Flag from the instance actor only; account stub routes.
- Statuses: each visibility's to/cc as delivered; CW as summary; replies to
local and remote posts (mention, inReplyTo, the author's inbox); polls and
votes (local, and remote votes only to the author without published); media
attached only by its owner; quotes, the quotes list and revocation; edit
history, source and the Update delivery; delete for redraft, the 410
Tombstone and the Delete delivery; favourite/reblog counts with Like,
Announce and their Undos; favourited_by, reblogged_by; bookmarks; pins;
interaction_policy matching canQuote in the note and in the Update;
strangers get 404 for followers-only and direct posts; a located post is
unreachable by id for anyone else on every route; statuses?id[];
Idempotency-Key; scopes; deleting a reblog.
- Timelines: home paging with max_id, since_id and min_id; public local and
remote; tag (anonymous); list stub; favourites; conversations and read;
markers; notifications with types[], exclude_types[], account_id, paging,
get, dismiss, clear and unread_count.
- Instance: v1 and v2 (4.2.0 (compatible; PrivaPub)), peers, activity, rules,
extended_description, apps and every stub route.
- Media: v1 and v2 uploads, owner-only GET and PUT, 422 for unsupported or
unreadable files, video and audio made with ffmpeg lavfi sources and checked
with ffprobe; every remote media address goes through the proxy; the proxy
refuses unsigned URLs, streams ranges as 206 without caching, caches whole
downloads and serves them with ranges, and streams anything over
Media:MaxProxiedBytes (a SmallProxyHost) without caching.
- Provenance of local, delivered (signature) and fetched (instance actor,
no signature, the trigger as activity) posts, visibility of provenance,
instance descriptions; reading any of them makes no outbound request.
Pleroma reactions with EmojiReact and Undo deliveries, and local reaction
notifications.
Bugs fixed:
- remove_from_followers deleted the Follower row but never told a remote
follower. It now sends Reject{Follow} with the stored Follow id, through
RelationshipService.RemoveFollower, which Block now shares.
- VisibilityPolicy.CanSee refused followers-only posts to accepted followers,
so a post in their home timeline answered 404 to GET, context, favourite and
reply. Followers of the author (local or remote) may now see them.
- Account statuses of a remote account hid followers-only posts from
personas that follow it.
- exclude_replies dropped the author's own threads; like Mastodon it now
drops only replies to other accounts.
- A community account's statuses were always empty: they are now the posts
addressed to the community.
- Pinning someone else's visible post answered 404; it answers 422 like
Mastodon.
- GET /api/v1/notifications/:id answered 200 with null when the notification's
post was gone; it answers 404.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ELjqpznMFMNrJoJUj6K5p2
Owner decision (2026-10-03, "A remote account deletes itself"): its posts are kept but
hidden everywhere.
- Post.AuthorGone (additive bool). DeleteHandler's actor-delete branch sets it on every
post whose ActorURI is the actor (one update-many), besides dropping its follows and
timeline rows as before. RemotePosts.Build sets it on a post stored later for an
account already marked Deleted.
- One rule in VisibilityPolicy: IsShown (not deleted, author not gone), IsPublic and
CanSee exclude AuthorGone, plus Shown(post) for loaded posts.
- Lookups by id answer 404 through CanSee (statuses/:id and every sub-route, context,
bookmarks, favourites, polls, reactions, search); provenance, account statuses,
home/public/tag timelines, notifications, conversations, reblogged_by, the clientapi
home and post/DM lists, a community's outbox and our Announces filter on IsShown or
IsPublic; the Mastodon mapper never renders a hidden post or a boost of one.
Tests (30 new):
- AuthorGoneTests: the rule, the handler (posts kept, boosts included, follows and rows
gone), a post fetched after the delete, and 20 Mastodon/ActivityPub lookups over HTTP
seen before and hidden after.
- InboxGapTests: actor Update refresh (name, sanitised summary, key rotation in place
and to a new key id) even with an older `updated`; Undo{Follow} by activity id and by
object; Reject of our QuoteRequest (and a stranger's ignored); group-wrapped
Announce{Like} and Announce{Undo{Like}}; a locked persona's pending follow,
FollowRequest notification, and Decide accepting and rejecting with the original Follow.
- JobHandlerTests: AncestorsJobHandler up to its depth limit; PollRefreshJob and
PollCloseJob (local and remote polls); InstanceDescriber from a peer's NodeInfo and
the weekly dedupe through ObjectRecords; LinkPreviews for public posts only;
DeliveryJobHandler outcomes (2xx, 404/410, 429/503 with Retry-After in seconds and as
a date, 5xx) and a signature and Digest the peer can verify; MediaJanitor.Sweep;
OAuthPruner.Prune.
- MigrationTests: _003, _004, _006 and _007 on seeded rows.
- PublicPagesTests: /@user and /@user/{id} (visibility, junk ids, exact CSP,
Referrer-Policy and nosniff), circle 404, community page, the instance actor,
ActivityPub redirects, and markup escaped in posts, titles and bios.
Production changes besides the rule:
- LinkPreviews.Handle re-checks that a post is still shown and public (the rule
Wanted applies) before fetching anything; before, only enqueueing checked it.
- The legacy /clientapi post and DM lists no longer return soft-deleted posts.
- MediaJanitor.Sweep and OAuthPruner.Prune are the loop bodies, now public and tested.
- InstanceDescriber.Address: a protected virtual identity seam so a test can point
the https NodeInfo addresses at a plain-http peer; production behaviour unchanged.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ELjqpznMFMNrJoJUj6K5p2
Tests through the real routes of PeasantsController, WellKnownController and
UsersController, on the whole server under test (34 new tests):
- FederationGetTests: the actor document (activity+json, SPKI key at
#main-key owned by the actor, sharedInbox, published = PublishedOn's day,
no creation date, no root); ld+json; browsers sent to /@name; Vary: Accept;
/users 301; the outbox's totalItems and ?page=true&max_id paging across
the 20-item boundary, boosts as Announces, and no followers-only, direct,
located, federated-copy or deleted post; /groupies and /stalking naming
nobody; /trophies (public pins, newest first) and /tattoos; /scribbles
(public and unlisted 200, browsers redirected, followers-only, direct and
located 404, deleted 410 Tombstone); a circle post only for a signed member
or its instance actor; a circle's /groupies, /flock and /wardens only for
members; /grunts create- and announce- ids; /parrot-licences 200, revoked
410, wrong author 404; secure mode's 401 for every unsigned GET but the
instance actor's.
- WellKnownTests: WebFinger by acct:, @-prefixed, bare, upper-case and actor
URI; other domains, unknown names, a root's login name and no resource;
the instance actor, a community, a circle (answered: current behaviour);
NodeInfo links, 2.0 and 2.1 naming no root, unknown versions 404; usage
counting only public, unlisted, non-boost local posts (Exclusive).
- InboxRouteTests: all three inboxes accept a signed delivery and refuse
junk (400), unsigned (401), a bad Digest, a two-hour-old Date, a signature
for another host and a swapped body (401); an unknown persona's /mouth is
404; ld+json with the ActivityStreams profile is accepted; a signer whose
actor answers 503 gets 503 with Retry-After; the 301st unsigned POST from
one address is 429 while a signed server from it is not.
- PersonaSeparationHttpTests: with a sibling persona and its community on
the same login, every GET under /api (filled with the persona's ids) plus
search, lookup and relationships, and a crawl of everything federation
publishes about the persona (actor, outbox pages, collections, scribbles,
grunts, WebFinger, NodeInfo, /@ pages), never name the sibling, its
community or the login.
Fixed:
- A circle's /groupies told anyone how many followers (members) it has,
while its /flock and /wardens were already for members only; it now
answers 404 to anyone but a signed member or a member's instance actor.
- WebFinger answered 404 to a bare user@domain or @user@domain resource,
which Mastodon, GoToSocial and Pleroma all accept; it now treats them as
acct: (noted in docs/INTEROP.md).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ELjqpznMFMNrJoJUj6K5p2
96 integration tests through PrivaPubHost, the real pipeline end to end:
- OAuth: the token's subject is the persona, and neither the token response,
verify_credentials nor the stored token entries name the root. A wrong password shows
an error and sets no login cookie; a login without the antiforgery token is a 400; the
return address never leaves the site; deny answers access_denied with no code; another
root's persona re-renders the choice with no code; a banned root is sent back to the
login and a code issued before the ban buys no token; force_login asks again; a code
works once and its reuse revokes the token it bought; password and refresh_token
grants are refused; a client_credentials token gets 401 on user routes; a read-only
token gets 403 with a Mastodon error on POST /api/v1/statuses; follow covers
read:follows; revoke works; the login and authorize pages send their CSP and no-store;
the 11th /oauth/login from one address in a minute is a 429.
- Accounts: sign-up, duplicates in any case, invalid models answer 400 with a message,
login and logout, recovery email, settings, password change, invitation sign-up and
login (refusing a persona named after the login), recovery without an email, through
an unreachable mail server, with a wrong and with a valid code, token refresh, the 11th
sign-up from one address, expired, garbage and foreign-key JWTs.
- Personas: a rootId in the body is ignored, the username regex and reserved names hold,
personas and groups share ReservedName, an update delivers Update{Person} to followers,
PublishedOn and the id's day fall within two weeks before creation, the list holds only
one's own personas, another root's persona cannot be updated.
- Groups: communities and circles are created, joining takes the code and the password,
members leave and owners cannot, a remote follow request becomes a member only on
approval, a circle never shows in lookup, account by id, v2 search or /@name, and its
/flock and /wardens answer 404 unsigned and to non-members, 200 to a member's signed GET.
- Moderation (Exclusive, it suspends localhost): ban, unban and remove; non-admins get
403; reports are listed without the reporter and resolved; domain blocks are inserted,
listed and deleted, bad domains refused, and a suspended server's delivery is answered
202 and kept nowhere; the data endpoints.
- AdminCommands: exit codes 0, 1 and 2 and the resulting policies.
Fixed:
- A banned or removed root kept using /clientapi with its JWT until it expired: only /api
re-checked the root. JwtEvents.TokenValidated now loads the root and fails the request
when it is banned or deleted, and takes the policy claims from the database, so a
demoted admin loses admin at once (and a promoted one gains it).
- The 401 and 403 bodies JwtEvents writes were PascalCase while every other /clientapi
answer is camelCase; they now use the web defaults.
- /clientapi/user/sniff/again (token refresh) answered an empty 200; it now answers a
fresh JwtUser, like login.
- Password recovery answered SMTP reply codes as HTTP statuses (421, 454, 554, and 550 for
an invalid address); a mail server failure is now 503 and an invalid address 400.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ELjqpznMFMNrJoJUj6K5p2
tools/pasture/run.sh up [peer...] | down | logs | ps and interop.sh [peer...] are now built
from parts:
- lib/pasture.sh: network, Caddy with its CA in a volume and copied to .ca/root.crt, Mongo,
PrivaPub;
- peers/<name>.sh: each peer's <name>_up;
- lib/interop.sh: ok, ko, and xf for checks expected to fail until a later phase;
privapub_token <persona>, which gives each peer its own persona under one root; and
stats_check;
- scenarios/<name>.sh: each peer's checks.
GoToSocial is pinned at 0.22.1, the version the 33 checks were proven on. Its scenario
gains four statistics checks:
- the admin statistics describe gts.test as gotosocial;
- they count inbound and outbound traffic;
- they name no account.
37/37 passed three runs in a row.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ELjqpznMFMNrJoJUj6K5p2
Under /clientapi/admin/statistics, admin only (the root JWT's IsAdmin policy, like the domain
blocks; /api tokens are persona tokens):
- GET overview?days: totals per channel, inbound and outbound outcomes, the delivery
success rate, delivery latency p50/p95 from the buckets, active and known servers,
distinct accounts, and the ledger's written/dropped/failed counts;
- GET hosts?days&sort=volume|failures|latency|host&software&page&limit: per server
traffic, refusals, failures, latency, accounts, software and delivery health;
- GET hosts/{host}?days: the server's description, its daily series and its last 100
events;
- GET events?host&channel&outcome&reason&before&limit, and GET server?days for the
ServerDay rows;
- POST rollups/{day} refolds a past day; POST hosts/{host}/describe describes a server
again.
Days not yet rolled up, today included, are folded live from the events, so the numbers are
current. Answers that may carry locations carry the DB-IP attribution.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ELjqpznMFMNrJoJUj6K5p2
- IConnectedAddresses remembers which address each host's last connection reached, set in
SafeHttpHandlerFactory's connect callback. That is once per pooled connection, with no
second DNS lookup, and the address is never stored.
- IGeoLocator / DbIpLocator reads the offline DB-IP Lite city and ASN databases (MaxMind
.mmdb, via MaxMind.Db). It maps memory, swaps to new files within ten minutes, rounds
coordinates to one decimal, never looks up a private address, and answers nothing
when the files are missing. CdnNetworks names the CDNs whose edge addresses say nothing
about where a server is.
- deploy/max/geo-update.sh fetches this or last month's databases, checks them and swaps
them in atomically. The privapub-geo timer runs it monthly as www-data, and setup.sh
installs the directory, the script, the units and a first download.
Describing servers will use these in M8.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ELjqpznMFMNrJoJUj6K5p2
A RollupDay job folds each finished day's InteractionEvents into one InstanceDay per server:
- Counters for the admin, keyed channel:activity:object:outcome:reason, plus signature
schemes, audiences, local kinds and features;
- PublicCounters, everything a public page may ever read:
- inbound and outbound activities from an allowlist, on public, unlisted or unaddressed
traffic only, with the outcome collapsed (accepted or dropped, delivered or failed);
- no reasons, no Flag or Block;
- features of public objects;
- health:ok or health:failed from reachability (a 4xx means the server answered);
- latency, wait and byte histograms, and the number of distinct accounts from that day's
hashes.
Re-running a day replaces it and keeps the live Reads counters. The day's salt is then
deleted, so its hashes can never be recomputed, and the next day is queued.
StatisticsSchedule plans today's rollup every hour and catches up any of the last seven
days that have events but no rollup. Waits round up into their bucket.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ELjqpznMFMNrJoJUj6K5p2
- A fetched ObjectRecord no longer wears the signature, key, signed headers and @context
of the activity that caused the fetch, and its ReceivedAt is the fetch's own time. Its
activity fields now name the trigger. Provenance answers signature null and
fetchedBy "instance-actor". Migration _008 clears the old fetched records.
- ObjectRecords store their ObjectFeatures extensions and context namespaces (migration
_009 fills older records in batches), so statistics can count them. Provenance reads the
stored lists.
- ForeignAvatar.Features records what an actor document uses (ActorFeatures): featured,
shared inbox, FEP-521a, FEP-8b32, identity proofs, Misskey fields, indexable,
undiscoverable, locked, key size, and more.
- RemoteEdits.Apply and RemoteDeletes.Remove are shared by the Update, Delete and Announce
handlers. A community-wrapped Update is now an edit only when newer, refreshes polls
and media otherwise, revises the ObjectRecord and re-resolves quotes. A
community-wrapped Delete now tombstones the object, removes its ObjectRecord, reblogs
and timeline rows, and lowers the reply count. Any deleted quoting post lowers the
quoted post's quote count.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ELjqpznMFMNrJoJUj6K5p2
- HttpScope (AsyncLocal) tags each outbound request with a purpose and a trigger:
- purpose is set by the caller: actor, key, object, webfinger, context, nodeinfo;
- trigger is set by the job kind, by "verify" during inbox verification, or defaults
to "request".
- FederationHttp records every JSON, media and stream fetch: status, time, bytes, hops,
and an outcome of ok, refused or failed, with a reason: disallowed, remembered,
bad-redirect, too-many-redirects, content-type, too-large, bad-json, private-address,
timeout, network, or the status. A fetch a reader caused (trigger "request") is only
counted per server per day.
- Link previews record a 'preview' event: card, no-card or failed.
- The media proxy counts cache hits.
- TrafficMeter counts the client API per endpoint group, method and status class. It
counts our served documents (actor, outbox, collection, object, activity, licence,
webfinger, nodeinfo) by kind, status and whether signed, per day and never per server,
and never names a circle's collections.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ELjqpznMFMNrJoJUj6K5p2
DeliveryJobHandler records each attempt as an 'out' event: the activity, object type and
audience read from the body (ActivityShape, shared with the inbox side), the receiving
server, the status, the time the POST took, the wait since it was queued, the bytes, the
attempt number and the signer's kind (none for circles or private traffic). The outcome is
ok, deferred (429 or 503 with Retry-After, or an open breaker: host-unavailable), retry
(5xx, 408, timeout, network), or dead: another 4xx, private-address, not-deliverable,
signer-gone, or a retry at the last attempt. Until now none of this outlived the job's
seven-day TTL, and the last error was overwritten on success.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ELjqpznMFMNrJoJUj6K5p2
Arrival carries a verdict that handlers set with one line before an existing return
(Arrival.Drop, Reject, Accept, About), so no handler signature changes. InboxProcessor
records it as an 'in' event, with the time taken, the wait since the inbox accepted it, the
attempt, the audience (from the post's visibility, or else the activity's addressing), the
local actor kind, the object's age for updates, deletes and reactions, and the FEP features
of a delivered object. It also records types nothing handles (unknown-type), actors that
cannot be loaded (deferred) and handler failures.
Drop reasons: fetch-failed, unparseable, misattributed, duplicate, deleted, not-addressed,
not-followed, not-public, not-visible, not-deleted, unknown-object, unknown-recipient,
cross-origin, unsupported. Accepted sub-reasons: stored, poll-vote, edit, refresh,
actor-refresh, actor-delete, removed, tombstone-only, auto-accepted, pending,
follow-answer, quote-answer, quote-granted, undone, reaction, reported. Rejected: blocked,
ignored, quote-refused.
A circle's traffic is private and kindless, and a stranger posting into one is just
not-addressed, so the event store cannot reveal a circle.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ELjqpznMFMNrJoJUj6K5p2
InboxReceiver records each answer once, in a finally, with a reason: too-large, not-json,
not-activity, missing-type-or-actor, no-signature, the signature check's own codes
(headers-unsigned, digest-mismatch, header-unreadable, date-skew, expired,
algorithm-unsupported), signature-invalid, actor-not-key-owner, key-unavailable,
id-cross-origin, undo-foreign, misattributed, unknown-recipient, and for 202s queued,
duplicate, suspended or self-delete-unknown-key. Each event carries the activity and object
type, the inbox, the signature scheme, the bytes and the time taken. A 404 for an unknown
/mouth and a rate-limited inbox (429, from OnRejected) are recorded too.
Until the signature verifies, the host is only claimed, so it is kept only if the server is
already known. A suspended server is recorded under its own name.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ELjqpznMFMNrJoJUj6K5p2
InteractionEvent records one interaction with a remote server: its channel (recv, in, out,
http, preview, crawl), activity and object type, outcome and reason, status, latency, wait,
bytes, attempt, audience, local actor kind, inbox, signature, features and the object's age.
IInteractionLedger.Record never blocks and never throws: events go into a bounded channel
of 10k, a full channel drops and counts, and a hosted service writes batches of up to 1000
every two seconds.
Privacy, as decided by the owner:
- no persona, root, group or activity id, inbox URL, actor URI or sender IP is stored;
- distinct accounts are counted with an HMAC keyed by a per-day salt (InteractionSalt,
upserted so restarts agree, never created for a past day);
- the local actor kind survives only on public and unlisted traffic;
- a host claimed by an unverified sender is kept only if it is already known.
Traffic caused by reading is only counted per day (InstanceDay.Reads, ServerDay). Indexes:
a 90-day TTL on events, unique day rows, and a TTL safety net on salts.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ELjqpznMFMNrJoJUj6K5p2
NeverFiveHundredTests walks every endpoint the server maps and fills each route parameter
with junk ('x', zeroed and random ObjectIds, a dot-dot segment, 5000 characters). It calls
each one anonymously, as a persona, and with a junk token, and asserts that nothing
answers 5xx or throws, and that every /api error carries a Mastodon error body.
It found two bugs:
- A junk 'Authorization: Bearer' on any non-/api route, anonymous ones included
(/build.json, /peasants/*, inboxes), broke the response. JwtEvents.AuthenticationFailed
wrote a body during authentication and the endpoint then wrote its own. The 401 body now
comes from the Challenge event, which runs only when an endpoint needs a user.
- /api answered 401, 404 and 429 with no body. UseMastodonErrorBodies gives any /api error
that leaves without a body Mastodon's {"error": ...}.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ELjqpznMFMNrJoJUj6K5p2
PrivaPubHost is a WebApplicationFactory<Program> on the fixture's database, configured only
through UseSetting (visible before Build, unlike ConfigureAppConfiguration). It drops the
background workers so tests run the jobs they queue (Jobs.Run, RunInbox), gives each client
its own address for the rate limiter, and has a SecureMode variant. Accounts signs up roots,
adds personas and gets Mastodon tokens through the real /oauth code flow. RemoteActor signs
HttpRequestMessages for the real /peasants routes; Peer records bodies and headers and
serves files with ranges and text pages.
Program registers the Guid serializer with TryRegisterSerializer, so a second host in one
process starts; the fixture runs the migrations in production's order before any test.
HostBootTests: the host shares the fixture database; the harness handles exactly the
activities the server registers; every job kind has one handler; every controller and page
model can be made; the service graph validates with ValidateOnBuild and ValidateScopes;
Swagger is 404 outside Development; a persona's token never names its root; a signed DM
through the real /mouth route is queued, processed and stored.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ELjqpznMFMNrJoJUj6K5p2
tools/ci/with-test-mongod.sh starts mongod (or podman's mongo:8) on a random localhost port
with a temporary data directory, runs the command, and removes both. build.yml and deploy.yml
test through it, so the ~85 integration tests stop being skipped in CI. MongoFixture refuses
production's port and data directory, and in CI anything but the wrapper's mongod; with
PRIVAPUB_TEST_REQUIRE_MONGOD=1 a missing mongod fails instead of skipping.
The deploy now passes the PRIVAPUB_SMOKE_TOKEN secret to the Mastodon smoke check, so its
signed-in half runs once the owner creates the persona and the secret.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ELjqpznMFMNrJoJUj6K5p2
- JobQueue takes an optional scope, so a test's worker leases and reaps only its own jobs.
- The dead-host delivery test runs alone (Exclusive), on its own jobs, and cleans up the
breaker rows it trips; the breaker has tests of its own on unique hosts.
- Index and migration tests run alone: they drop indexes and rewrite every post.
- DomainBlocks.Load replaces reflection and a database-wide block in tests.
- Harness.Outgoing sees only deliveries queued since the harness started: Peer ports are
reused within a run, which made the circle test flaky.
- Two pure-logic tests leave Mongo-gated classes, so CI runs them.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ELjqpznMFMNrJoJUj6K5p2
- Our public and unlisted posts state interactionPolicy.canQuote. The policy comes from the post, then the persona
(`source[quote_policy]`: public, followers or nobody; default public as the owner chose), and is always nobody for
followers-only posts and DMs.
- QuoteRequests are answered with Accept{result} naming a parrot-licence at /peasants/{name}/parrot-licences/{id}
(the route name the owner chose), or with Reject. Followers-only checks that the requester really follows.
- A licence is a QuoteAuthorization naming both posts; revoking it (POST /api/v1/statuses/:id/quotes/:quoting_id/revoke)
marks it 410, sends Delete{licence} to the quoter and the persona's followers, and revokes our own copy of the quote.
- A quote that arrives with one of our licences is accepted only if that licence is ours, unrevoked and names exactly
that quoting post. One persona quoting another gets a licence too.
- Mastodon API: quote_approval for our posts (automatic, followers, current_user), `quote_approval_policy` when posting,
PUT /api/v1/statuses/:id/interaction_policy, `source.quote_policy`.
Checked live: GoToSocial still accepts our posts with the policy stated, and leaves likes, replies and boosts open.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB
- The media proxy streams ranged requests from the origin (passing the range on, never caching), downloads and caches
whole files otherwise, and serves cached files with range support. A PeerTube video is never fetched whole for one
viewer, and clients still never contact the remote host.
- PeerTube's fragmented MP4 files inside an HLS entry are read as variants, so HLS-only instances play too.
- A remote Video or Audio post becomes one playable Mastodon attachment: the best MP4 up to 720p that carries both
sound and picture, with its poster and duration; the card is kept only when nothing is playable.
- nginx: /media/proxy/ with proxy_buffering off and a 600 s read timeout (applied on the box, with a backup).
Checked live: a GoToSocial image through the proxy answers 206 with exactly the asked range when streamed, 200 when
cached, and 206 with the right Content-Range from the cache.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB
- Received quotes: read from `quote`, `quoteUrl`, `quoteUri`, `_misskey_quote` or a FEP-e232 Link tag; the quoted post
is fetched once; a quoteAuthorization stamp is verified field by field on the quoted author's origin; a consent
quote without a stamp is pending; an older-key quote of a public post is shown; Delete of a stamp revokes. Counts
and a `quote` notification follow the accepted state. The `quote-inline` fallback survives sanitising and is removed
from content when the real quote is shown.
- Personas quote through `quoted_status_id`: posts that state a quote policy get a QuoteRequest and stay pending until
an Accept brings a stamp we can verify, then an Update adds quoteAuthorization; posts that state none are quoted
the older way, without `quote`; another persona's posts cannot be quoted yet (we issue no stamps). Quoting posts
are delivered to the quoted author too.
- Mastodon API: Status.quote (with the quoted status one level deep), quotes_count, quote_approval from the remote
policy, GET /api/v1/statuses/:id/quotes, `quote` notifications, and api_versions.mastodon = 7.
Checked live: GoToSocial's author-only quote policy is respected.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB
- A public post that links to a page gets a preview card: from the post's own data first (FEP-8967 Link preview, the
object's image, title and summary); otherwise the server reads the page once, 0-60 s after the post arrives, never
when someone reads it. OpenGraph and Twitter tags give title, description and image; one LinkPreview per address
is cached for 7 days and shared by the whole server, so a fetch never points at a persona.
- Lemmy link posts, which carry no title or description, get them filled in.
- The page fetch uses the guarded client (public addresses, three redirects, HTML only, first 512 KB).
- Federation:FetchLinkPreviews switches page fetching off.
- Local public posts get cards too.
Checked live: a link to a GoToSocial profile page becomes a card with its title, description and proxied image.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB
- Incoming reactions in all three shapes: Misskey and Sharkey Likes carrying an emoji (`content` or
`_misskey_reaction`), Pleroma, Akkoma and Iceshrimp.NET EmojiReacts, and their Undo. Unicode reactions are one
grapheme; custom ones need their Emoji tag and keep its image; `:name@host:` is read as `name`. A Like whose
content is a heart stays a favourite.
- Reactions are kept per account and emoji on our posts and on remote ones we hold, and the author of a local post
gets a `pleroma:emoji_reaction` notification with the emoji.
- Personas react through Pleroma's API (PUT/DELETE /api/v1/pleroma/statuses/:id/reactions/:emoji, GET for the list),
which sends an EmojiReact (or its Undo) to the post's author.
- Statuses carry `emoji_reactions` (read by Phanpy) and `pleroma.emoji_reactions`, with counts and whether the viewer
reacted.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB
- Custom emoji (Emoji tags) on posts, display names, bios and profile fields, at most 64 per object, proxied, in
Status.emojis and Account.emojis.
- Remote profiles keep their header, profile fields, locked flag, published date, movedTo, indexable, memorial and
image descriptions (a locked GoToSocial account no longer shows as open).
- Polls: incoming Questions (Mastodon, Misskey, Pleroma, GoToSocial shapes) with counts, voters, end and closed;
our own polls from the Mastodon API go out as Questions; votes in are counted once per voter and never become
replies; personas vote on other servers' polls with one Note per choice; counts refresh with an Update at most every
three minutes; a poll closes on time and tells its voters and its author. GET /api/v1/polls/:id and POST
/api/v1/polls/:id/votes.
- An Update without a newer `updated` only refreshes poll, video, audio and event details and leaves no revision.
Checked live against GoToSocial: each side's poll reaches the other as a poll and each side's vote is counted.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB
- When a sender's key cannot be fetched because its server timed out or answered 5xx, the inbox answers 503 with
Retry-After: 300 instead of 401, so Mastodon 4.7 retries rather than switching to RFC 9421 signatures we do not
verify yet. The fetcher's failure cache now remembers whether a failure was temporary.
- Follow, Like, Block, Accept, Reject and Undo ids are deliberately not dereferenceable: serving them would publish
who follows, likes and blocks whom. They are always sent with their object embedded.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB
- Dislike and its Undo are kept as downvotes (Lemmy, PieFed, Mbin, Friendica) and shown with favourites as votes.
- ChatMessage (Lemmy 0.19, Mbin, PieFed to those two) arrives as a direct message.
- A deleted object's id is remembered for 90 days, so a Create that arrives after its Delete cannot bring the post
back; the post's ObjectRecord goes with it.
- A Join of one of our objects is answered with Ignore, as FEP-8a8e asks of a server without RSVP.
- A 503 with Retry-After is waited out like a 429 instead of counting as a failure of the host (GoToSocial throttling,
Mastodon's temporary key failures).
- Status.privapub carries what a Mastodon Status cannot: object type, title, excerpt, cover, the author's source,
link, video, audio and event details, and up/down votes, with every media URL proxied.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB
- ObjectShapes reads `url`, `icon` and `image` as a value, a Link or an array; Markdown `content` (PeerTube) is
rendered; a missing `mediaType` is inferred from the extension or the attachment type (Bridgy); alt text comes from
`name`, `summary` or their language maps; thumbnails come from attachment `icon`, object `icon[]`, `image` or a Loops
`preview`; durations are ISO 8601 or seconds; per-attachment `sensitive` is kept; the language falls back to
`@context` `@language` (Pleroma); titles and excerpts are plain text; hashtags normalise with NFKC like Mastodon.
- Typed details on Post: Link (Lemmy link posts, Mbin `source` URLs, Mastodon 4.7 Link attachments with an FEP-8967
publisher preview), Video (PeerTube files with their streams, HLS playlist, poster, captions, chapters, licence,
channel, live state, comment policy), Audio (Funkwhale), Event (Mobilizon, Gancio, Hubzilla: times, zone, floating
time, places and addresses, online link, capacity, status), the cover image, and the author's own source text.
- Mastodon API: a card built from those details without fetching anything, an event's "when · where" line, attachment
thumbnails and durations, and no media file offered as a preview image.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB
- ObjectRecord, one per stored remote object: the raw JSON (up to 256 KB, always hashed), delivered or fetched,
refetched from origin or not, the activity that brought it (or caused the fetch), shared or personal inbox, the
signature's key, algorithm and signed headers, received time, published and updated, the delivering activity's
@context, and up to ten later revisions from Update.
- Delivery details travel from InboxReceiver through the inbox job to the handlers as Arrival.Current.
- A host is described from its NodeInfo when we first hear from it, at most weekly (DescribeInstance job), never when
someone opens the details view.
- GET /api/privapub/v1/statuses/:id/provenance and /api/privapub/v1/instances/:host, with the extensions an object
used detected from its raw form (044f quotes, interaction policies, contexts, proofs, Misskey fields, MFM, FEP-8967
links, emoji, polls, language maps, url variants, Markdown content).
Checked live: a GoToSocial reply shows as delivered to the shared inbox, signed hs2019 with GoToSocial's fragment-less
key id, with its interaction policy detected, and gts.test is described as gotosocial 0.22.1.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB
- A remote `summary` is a content warning only on a Note or Question, or when `sensitive` is set. On Articles,
Events, Videos, Pages and Audio it is an excerpt (WordPress teasers, Mobilizon dates and places, Mbin titles) and
is now kept as `Post.Excerpt` instead of hiding the post. `Post.ObjectType` records the remote type, and the
Mastodon API shows a remote non-Note object's title above its body again.
- Persona usernames must match `^[a-z0-9_]+$`, as groups already did; a name outside it was unreachable from
Mastodon and Misskey.
- `postingRestrictedToMods` is defined in our JSON-LD context (Iceshrimp.NET drops undefined terms).
- `Vary: Accept` on actor and object URLs.
- Owner decision: blocks federate. A block sends `Block`, an unblock `Undo{Block}`; checked live against GoToSocial.
- Owner decision: a persona's and a group's `published`, and the day in new ids, is a random day up to two weeks
before creation, so personas made the same day no longer share a date. Migration _007 gives existing ones theirs.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB
docs/INTEROP.md collects five research passes from 2026-10-01: Mastodon 4.7 and GoToSocial 0.22, the Misskey and
Pleroma families, the threadiverse (Lemmy 0.19/1.0, PieFed, Mbin, NodeBB), media and long-form (PeerTube, Loops,
Pixelfed, WordPress, Ghost, events, audio, books, Threads, Flipboard, Bridgy Fed), and cross-cutting FEPs and
signatures. Each claim was checked against source code or live fetches, with dates and versions.
It is checked against our own code: what is already right, three cheap things that are wrong today (summary read as a
CW on every type, unchecked usernames, an undefined context term), what the rich client needs kept (a post kind with
typed payloads, raw capture and provenance for the details view), and the six privacy choices the owner has to make.
The roadmap's P5 becomes P5 to P8, ordered by that evidence.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB
tools/pasture/run.sh starts PrivaPub, GoToSocial and Mongo on one podman network behind Caddy's internal CA, and
interop.sh drives both through their own client APIs: follows (one to a locked account), posts, CW, replies, likes,
boosts, DMs, edits, deletes and unfollow. All 25 checks pass, three fresh runs in a row.
- Federation:AcceptAnyCertificate joins the two test-network switches; startup refuses all three in Production.
- WebFinger falls back to http only when AllowPlainHttp is on.
- A bootstrap logger, so a failure before the host is built is no longer silent.
- P4 is ticked in the roadmap, with what has not been run live (Lemmy, a Mastodon circle member).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB
Found by the first live run against GoToSocial:
- A remote post's id came from its published second plus random bytes, so a reply arriving in the same second as
the post it answers could sort under it, and a late arrival landed behind a client's since_id. A post published
within the last hour now gets an id for its arrival; backfill keeps its published time.
- NoteParser.Time returned default(DateTime) for a missing "updated", so every remote post was stored as edited in
year one. Migration _006 clears the stored ones.
- published now carries milliseconds, so peers that derive ids from it (GoToSocial, Mastodon) keep our posts in
order within a second.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB
Communities:
- a post addressed to a community (to, cc or audience) is accepted
according to its posting policy - followers, anyone, or moderators -
and GroupDistributor announces the whole activity with `audience` to
the community's followers, plus the object for new posts so Mastodon
shows them; updates and deletes of community content are announced too;
- top-level posts are Pages with a name (the title, or a headline from
the text); /flock counts members, /wardens lists moderators;
- a Mastodon client posts into a community by mentioning it, or into a
remote group, which sets `audience`;
- an Announce of an activity from a remote group a persona follows (Lemmy)
is followed through: the object is fetched from its own origin, kept with
its AudienceURI, and fanned out to the group's local followers; updates
are applied in place and deletes checked against the origin.
Circles stop being local-only: an undiscoverable Group actor whose follows
are all requests the owner approves; posts addressed to the circle and its
/flock and delivered to members' own inboxes, never announced, never
public; a remote member's post into the circle is accepted from members
only. SignedFetchAuthorizer serves circle posts and collections only to a
signed request from a member or a member server's instance actor - 404 for
anyone else. Circles never surface in search, lookups, mentions, account
ids or profile pages.
Federation:SecureMode requires a valid signature on every GET under
/peasants except the instance actor. Group forms take a posting policy.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB
A post given a latitude and longitude becomes LocalGeo: its position is
rounded to two decimals (about a kilometre) and stored as a 2dsphere
point, its radius clamped to 1-50 km, and it is local only - no Create,
no delivery, no outbox, never in the Mastodon API or on a profile page.
/clientapi/post/nearby takes the viewer's position for the query only
(it is neither stored nor logged), runs $geoNear and keeps posts within
each post's own radius, minus authors the viewer blocks or mutes. A
located post cannot be direct or in a group.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB
- /api/v1/media and /api/v2/media (and GET/PUT /api/v1/media/:id):
images go through libvips (NetVips, its native build bundled):
autorotated, every kind of metadata dropped (EXIF, GPS, XMP, IPTC,
comments), capped at 4096 px, with a 640 px preview and a blurhash
(own encoder, the reference algorithm); animated GIFs are re-encoded;
video and audio are remuxed by ffmpeg with -map_metadata -1, never
re-encoded, and a video gets a still preview. Files get random names
under /var/lib/privapub/media, outside the web root deploys replace, and
are served at /media/files with nosniff and a sandbox CSP.
- media_ids on create and edit (four at most, the persona's own, each used
once); notes carry them as Document attachments with alt text, blurhash,
focalPoint and size; inbound attachments were already kept.
- avatar and header uploads in update_credentials, cropped to 400x400 and
1500x500, federated with Update{Person}.
- Remote media reaches clients only through /media/proxy/{hmac}/{url},
fetched by the guarded client (no SVG, 40 MB cap) and cached outside the
served root, trimmed to 5 GB; foreign avatars and headers use it too, so
a client never contacts another server.
- MediaJanitor deletes uploads left unattached for a day.
- nginx accepts 100 MB bodies on the upload endpoints only (applied on Max).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB
- Blocks are per persona and never federate (a Block activity would tell
the other server who blocked whom): the blocked account is removed as a
follower, with a Reject{Follow} if it is remote, unfollowed, cleared from
home and notifications, and refused with Reject if it follows again.
- Mutes (optionally timed, optionally sparing notifications) and per-
persona domain blocks keep authors out of home timelines, notifications
and every status list the API returns; the boosts of a hidden author's
posts are hidden too.
- Bookmarks and pins (at most five, public or unlisted, own posts) with
their Mastodon endpoints and flags; pinned posts are the actor's
`featured` collection at /trophies, and `featuredTags` points at
/tattoos.
- Reports: /api/v1/reports stores the report and, when forwarding to a
remote account, sends Flag from the instance actor, so the reporting
persona is never named to the other server. An inbound Flag about a local
persona or its posts becomes a report; moderators list and resolve them
under /clientapi/moderator/reports.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB
v1.4.0 serves OAuth and the Mastodon client API; tools/smoke/mastodon-api.sh
passes against production. The signed-in side was verified locally with two
personas of one login; no real client has logged into production yet.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB
- PersonaSeparationTests: two personas of one login follow the same
account and post; nothing the API maps for one contains the other's id,
username or the root id.
- tools/smoke/mastodon-api.sh checks what a client meets first (instance
v1/v2, discovery, app registration, client credentials, an app token
refused by a user endpoint, the public timeline, revocation) and, given
a persona token, verify_credentials, home and notifications. deploy.yml
runs it after every deploy.
- OAuthPruner removes invalid tokens and authorizations older than two
weeks, every six hours.
- The consent page no longer sets form-action, which browsers apply to the
redirect back to the client after the form is posted.
CLAUDE.md gains the Mastodon API layout and invariants.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB
It asserted fewer than ten requests to the dead host by the time the live
one arrived, which a busy workstation can exceed without the per-host
limit being broken; the live delivery arriving at all is the point.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB
With a token for a persona, a Mastodon client can now:
- accounts: verify/update credentials (display name, note, fields, locked,
bot, discoverable, indexable, hide collections, posting defaults; the
change federates as Update{Person}), get, lookup, statuses (paged, by
visibility to the viewer), relationships, search, follow and unfollow,
follow requests (authorize and reject answer remote followers with
Accept or Reject), remove from followers. Followers and following lists
are shown to their owner only.
- statuses: post (plain text, mentions, hashtags, replies, content
warnings, the four visibilities, Idempotency-Key), get, edit (PUT),
delete returning the source for redrafting, context, history, source,
favourite, reblog and their undos, favourited_by and reblogged_by.
- timelines: home, public (local or remote), tag; favourites;
conversations; markers.
- notifications: list with types and exclude_types, get, dismiss, clear,
unread count.
- /api/v2/search, resolving a handle or a URL to an account or a post.
Media, polls, pins, bookmarks, mutes, blocks, lists, filters, trends and
push answer empty lists or a 422 saying they are not supported yet, so
clients degrade instead of failing. Public reads work without a token.
Persona settings (locked, bot, indexable, discoverable) now also shape
the ActivityPub actor.
Fixed on the way: three conditional expressions whose `default` was the
value type's, so a missing limit became 1, a missing flag became false and
an attachment without dimensions became 0x0.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB
Domain/Statuses/StatusService takes a persona, not a root, and is what
/clientapi and the Mastodon API share:
- Publish renders Markdown (clientapi) or plain text (Mastodon clients),
checks the persona may see the post it replies to, opens or reuses the
conversation for a direct post from its recipients and mentions, fans
out and hands the Create to the outbox;
- Edit keeps a revision and sends Update{Note}; Remove soft-deletes and
returns the post, so a client can delete and redraft;
- Favourite and Reblog work on anything the persona can see and send Like,
Announce and their Undo to the author (and, for boosts, to followers);
boosts are refused for anything but public and unlisted posts.
PostsService is now the clientapi wrapper that checks the root owns the
persona. A persona's own boost is a local row: the outbox renders it as
Announce, /grunts/announce-{id} resolves, and object endpoints, profile
pages and NodeInfo skip it. ContentFormat gains Plain.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB
OpenIddict 7.7 (MongoDB stores, keys kept in Mongo so tokens survive
restarts) serves /oauth/authorize, /oauth/token, /oauth/revoke and the
discovery documents, including /.well-known/oauth-authorization-server:
- authorization code (PKCE optional) and client credentials, Mastodon's
scopes including the granular ones, non-expiring reference tokens,
`created_at` in the token response, and the urn:ietf:wg:oauth:2.0:oob
page that shows the code;
- /oauth/login signs the private login into a fifteen-minute cookie that
only /oauth sees (rate limited, antiforgery-protected); /oauth/authorize
then asks which persona the application acts as. The token's subject
is that persona's id and nothing else; no root id reaches a token, an
authorization or a log line;
- the token exchange refuses a persona whose login is banned or deleted,
and every API request checks the same.
/api/v1/apps registers applications dynamically, /api/v1/apps/
verify_credentials, /api/v1/instance (v1 and v2, "4.2.0 (compatible;
PrivaPub)") and verify_credentials answer in Mastodon's shapes: snake_case
with explicit nulls, Rails-style parameters from query, form or JSON,
{"error": ...} on failure, Link paging. CORS exposes Link.
/api goes to OpenIddict validation and everything else to the existing
JWT; the JWT failure handler no longer sends the exception and stack trace
to the client.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB
Deleting a remote post now removes its timeline entries and the reblogs
of it and lowers its parent's reply count; deleting a remote actor also
drops the follows pointing at it and its entries in home timelines.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB
- Like: a Favourite per account and post (unique), the post's counter, a
Favourite notification for a local author; only on posts the liker
could see (public and unlisted, or followers-only and direct when they
were addressed).
- Announce: the original is always refetched from its own origin (or is
ours), and must be public or unlisted. A boost of a local post counts
and notifies; a boost by an account a persona follows is stored as a
reblog row and fanned out to its followers' home timelines (as a reblog,
where reblogs are wanted). A boost by a stranger of a stranger's post is
ignored. FEP-1b12 group announces of activities wait for P4.
- Undo of a Like or an Announce reverses each of them; an Undo naming only
an id is matched against follows, likes and boosts in turn.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB
Fanout writes a TimelineEntry for every persona a post should reach: the
author, local followers of a local author, local followers of a remote one,
the members of a direct conversation or a circle. Mastodon's home rules
apply when it is written: a reply shows only to followers of both sides
(or to the one replied to), a reblog only where reblogs are wanted. A
mention of a local persona becomes a Mention notification.
Inbound posts are now also kept when a persona follows their author.
RemotePosts holds what CreateHandler and backfill share: building a Post
from a note, and fetching a public parent the first time a reply to it
arrives, so its author is known (a reply to an unknown or non-public
parent stays out of home timelines). Each fetched ancestor queues a
FetchAncestors job for the next one, up to ten deep.
/clientapi/timeline/home and /clientapi/notifications (with
/notifications/read) page by max_id for the persona's own root only.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB
OutboxPublisher works out who a post goes to, Mastodon's way: followers'
shared inboxes for public, unlisted and followers-only; every mentioned
remote account's own inbox; the recipients only for direct; the parent's
author for a reply; a community's followers for a post in it; nobody for
a circle. Creates, updates and deletes all use it.
- Local posts take a visibility (public, unlisted, followers-only) and a
spoiler text next to the content-warning flag.
- /clientapi/post/update keeps the previous version as a revision,
re-renders, and sends Update{Note} with `updated` to the same to/cc.
- Deleting is now soft: the content, title, spoiler, media and revisions
are cleared, timeline entries removed, the parent's reply count goes
down, Delete goes to the stored audience, and the object answers 410
with a Tombstone instead of 404.
- Editing a persona's profile sends Update{Person} to its followers.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB
Following records what a local persona follows, local or remote, with
its state and the Follow activity's id. FollowService (behind
/clientapi/follow, /clientapi/unfollow and /clientapi/following):
- resolves @user, user@host or an actor URI;
- a local account is followed in-process: accepted unless it approves
followers by hand, a Follower row on the other side, a community gains
the persona as a member, and a Follow or FollowRequest notification;
- a remote account gets a Follow signed by the persona, at
/grunts/follow-{id}, and stays Requested until an Accept;
- unfollowing deletes the rows, or sends Undo{Follow} to a remote account.
Inbound Accept and Reject are matched to the Follow by its id (or, for an
embedded Follow without one, by its actor), and only from the account that
was followed. A remote Follow now notifies the persona too.
Notification is the per-persona record P1.2 builds on, deduplicated by
type, persona, sender and post. TimelineEntry and Favourite are created
with their indexes for the next commits.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB
The route table lists the themed names now in use, the repo map gains
Jobs, Ids, Moderation, Domain and Web, and the invariants add the job
queue, VisibilityPolicy and the rule for which remote posts are stored.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB
VisibilityPolicy.IsPublic is the single expression for "anyone may see
this" (Public or Unlisted, not deleted); the outbox, the object and
activity endpoints, the HTML pages and NodeInfo all use it instead of
spelling it out. CanSee answers for a persona: the author, a mentioned
local persona, a conversation member for Direct, a circle member for
Circle; followers-only waits for P1.2's follows.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB
An ObjectId carries its creation second and a per-process counter, so two
avatars made one after the other by the same login got ids a few counts
apart: a link between personas that every Mastodon client would have
seen. Avatar and Group now generate ids from the UTC day plus eight random
bytes (still valid ObjectIds), and a remote post's id is made from its
published time with a random tail, so posts page in the order they were
written rather than the order they arrived.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB
DomainBlock (domain, severity, reject-media, public and private comment)
covers the domain and its subdomains. Admins manage them under
/clientapi/admin/domainblocks/{list,insert,delete}; the set is kept in
memory, reloaded on every change and at most five minutes stale.
A suspended domain is refused by FederationHttp.IsAllowed, so nothing is
fetched from it and no job delivers to it, and the inbox drops its
activities with a 202 before fetching any key. Reject-media strips the
attachments of posts from that domain. Silence is recorded for the
timelines and notifications that arrive in P1.2.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB
/@user and /@user/{id} are Razor pages showing an avatar's or community's
public and unlisted posts: no scripts, a strict CSP, no-referrer, noindex,
and an alternate link to the ActivityPub document. A client asking them
for activity+json is redirected to the actor or note, and the actor and
note redirect browsers here.
NodeInfo answers 2.1 as well as 2.0 (repository, homepage, a link to
FEDERATION.md) and counts only public local posts.
FEDERATION.md (FEP-67ff) lists the protocols, FEPs, route names,
activities and the security rules a peer will notice.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB
Local text (Domain/Content/ContentRenderer): Markdig with raw HTML off and
autolinks, or plain text for API clients; then @user and @user@host are
resolved (locally or through WebFinger) into Mastodon's h-card markup and
#tags into hashtag links, skipping code, links and e-mail addresses. A
post stores the result, its mentions and its tags, and is delivered to
the mentioned actors and the parent's author as well as to followers.
Renderer:
- Mastodon's @context (toot, schema PropertyValue, discoverable,
indexable, blurhash, focalPoint, featured, alsoKnownAs, movedTo) and
FEP-2c59's webfinger;
- an actor's url is its HTML page, its fields are PropertyValue
attachments, published is cut to the day and indexable is false;
- a note's content is the stored rendering; a title becomes name and a
bold first line, a content warning without its own text is summarised
by the title or "Content warning"; Mention and Hashtag tags, contentMap,
updated, and to/cc by visibility.
Routes take the agreed names: /groupies, /stalking, /scribbles/{id},
/grunts/{id} (a Create resolves), /whispers/{id} for a DM's context. The
outbox is a collection with a first page, paged by max_id. A browser
asking for an actor or a note is redirected to /@user, and WebFinger's
profile-page points there.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB
Post gains what federation and the Mastodon API need: Visibility (Public,
Unlisted, FollowersOnly, Direct, Circle, LocalGeo), the author's account
id, to/cc, the Create's id, url, context, quote, InReplyToURI and the
parent's author, a separate SpoilerText next to the title, language,
mentions, hashtags, remote attachments (alt text, blurhash, focus, size),
reply/favourite/reblog counters, revisions, EditedAt and DeletedAt.
Direct messages are Posts with Visibility Direct and a ConversationId;
migration _005 copies DmPost rows across with their ids and fills the new
fields of existing posts. DmPost is left in place so a rollback still sees
the old messages.
Inbound:
- NoteParser reads Note, Article, Page, Question and media types: content,
then contentMap, then _misskey_content; summary as the spoiler and name
as the title; Mention and Hashtag tags; attachments; a PeerTube-style
list attribution prefers the person over the channel; quote URIs.
- Addressing classifies like Mastodon, finding followers-only by the
author's own followers URL (now stored on ForeignAvatar), not by a
"/followers" suffix.
- Create keeps a post when a local persona is addressed or mentioned, when
it replies to a local post (the parent's reply count goes up) or when a
community it follows is addressed; an unsolicited public post is not
stored. Update keeps the previous version as a revision.
The outbox and object endpoints serve only Public and Unlisted posts.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB
InboxService is split the way the roadmap lays out Federation/Inbox:
- InboxReceiver reads and verifies the request exactly as before, runs
the checks that need no fetch (the activity id's origin, a Follow of a
missing or local-only actor, an Undo of someone else's activity, an
embedded object attributed to someone else), queues a ProcessInbox job
and answers 202. The job's dedupe key is the activity id, so a peer that
delivers the same activity twice is processed once.
- InboxProcessor (two at a time, eight attempts) loads the verified actor
and hands the activity to the handler for its type.
- Handlers/{Follow,Undo,Create,Delete,Update}Handler are the old methods,
unchanged except that they no longer produce status codes; the JSON
helpers live in Objects/ActivityJson and the group membership helpers in
Inbox/ForeignMembers.
A slow fetch of an object or a remote actor now delays the job, not the
sender's HTTP request.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB
The single serial DeliveryWorker is replaced by Infrastructure/Jobs:
- Job rows are leased with one FindOneAndUpdate (oldest RunAt first, a
two-minute lease) and a reaper returns expired leases every 30 s;
- enqueueing wakes the workers, which otherwise poll every five seconds;
- delivery runs eight at a time with at most two per host, so a slow or
dead server holds two slots, not the queue;
- a failure waits n^4 + 15 + jitter seconds (Mastodon's curve) for up to
16 attempts; a 4xx other than 408/429 is final, a 429 honours
Retry-After;
- RemoteInstance is a per-host circuit breaker: ten consecutive failures
quarantine a host for an hour, doubling to a week, and its jobs wait
without spending attempts;
- a delivery is queued once per activity and inbox (unique DedupeKey), and
finished jobs expire after seven days (TTL on FinishedAt).
Migration _004 moves pending Delivery rows into jobs and marks them
abandoned, so a rollback to the old worker cannot send them twice.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB
The repo map gains Infrastructure/ and Objects/, the federation invariants
are the ones the code now enforces instead of a list of known holes, and
Commands, Data, Testing and Deploy cover the test project, the startup
order (warm maps, migrate, index), the admin CLI and the deploy checks.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB
- Sign-up, login, the invitation flows and password recovery allow ten
requests a minute per client address; the inboxes give each sending
origin (the keyId's) a bucket of 300 that refills at 300 a minute, and
answer 429 beyond it, which peers retry.
- deploy.yml dumps the PrivaPub database to /var/backups before it stops
the service (the last seven are kept), and after the swap checks that
Swagger answers 404 and that the shared inbox answers junk with 400 and
an unsigned activity with 401.
- ActivityPubClient and PostBoost, never used, are gone.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB
Infrastructure/Data/Indexes runs at every start, after the migrations:
unique on Post/DmPost ObjectURI, ForeignAvatar ActorURI, the Follower
triple, RootToAvatar, RootUser UserName and ReservedName; plain indexes on
the lookups the services actually make (PublicKeyId, author and group
post listings, ParticipantsKey, the delivery queue).
Migration _001 runs first and removes the duplicates the races could
already have left (keeping the newest actor row, the oldest post, the
accepted follower), then fills ReservedName from every avatar and group.
ReservedName is one username space for personas, groups and the instance:
a name is reserved by an insert the unique index arbitrates, before the
avatar or group is saved, so two simultaneous sign-ups cannot both get it.
A short list of names (admin, support, abuse, postmaster, ...) is never
available.
EntityMaps.Warm touches every entity's collection one at a time before
anything else runs. MongoDB.Entities maps the Entity base class on first
touch, and two types mapped at once throw "An item with the same key has
already been added" and stay broken for the life of the process; the
parallel test run hit it, and the delivery worker racing a request could
have too.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB
S14 and the privacy items of P0:
- signing up as "admin" no longer grants admin; `PrivaPub admin promote
<root>` (and `demote`) does, run on the box against the configured
database;
- Swagger is served in Development only;
- every service and controller answers "Something went wrong." where it
used to send ex.Message, and the SMTP warnings no longer log the
recipient's address;
- sign-up and login no longer log the IP, User-Agent and root id together;
- invitation sign-up takes the persona's own AvatarUserName (and optional
AvatarName) instead of naming the avatar after the private login, and
refuses a persona username equal to the login's. Invitation login uses
the named persona, creating it if it is new;
- recovery mail comes from "PrivaPub", not collAnon's support address name.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB
S8: a direct message joins a conversation only when its participants are
exactly that conversation's members, found through a new
DmGroup.ParticipantsKey (a hash of the sorted members). A remote context
no longer decides anything: it let anyone who knew a conversation's
context post into it, and joining by context while dropping a participant
would have shown a reply to someone it was not addressed to. A context is
kept only when it is on the author's origin. Sending a DM to the same
people again reuses their conversation instead of opening a new one.
S9: Group.Kind is Circle or Community. A circle is not a federated actor:
its actor, collections, WebFinger and inbox answer 404, a remote Follow is
refused, and posts in it are IsLocalOnly - never delivered, never in an
outbox, never served. Communities keep today's behaviour until P4.
Migration _003 makes every existing group a circle, marks their posts
local-only and backfills the conversation keys.
End-to-end inbox tests sign real deliveries from a fake peer: a context
injection, a forged activity id, a note attributed to someone else, a
cross-origin object, a bad signature, junk bodies and a Follow of a circle.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB
S5 of the roadmap. ContentSanitizer wraps HtmlSanitizer with Mastodon's
allowlist: the inline and list tags Mastodon keeps, href/rel/class and
the list attributes, microformat and mention/hashtag/ellipsis/invisible
classes, Mastodon's link schemes, every link rel=nofollow noopener
noreferrer, relative links unlinked, headings folded to a bold paragraph,
and the contents of script, style, svg, iframe and friends dropped rather
than kept as text.
Post and DmPost gain ContentHtml (what is shown) and ContentFormat
(Markdown for local, Html for remote). Inbound Create and Update, and a
remote actor's biography, are sanitized on the way in; local posts store
their Markdig rendering. Migration _002 does the same to what is already
stored, and migrations now run at startup.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB
S6 of the roadmap. An inbox POST is refused unless its signature covers
(request-target) and host, as well as the digest and a date or (created).
The Date or (created) may be at most an hour old and fifteen minutes ahead
(it was twelve hours either way), and an (expires) in the past is refused.
The request target is read raw from the server, so a percent-encoded path
verifies as the sender signed it.
Tests cover a Mastodon-shaped delivery and each way of tampering with it,
plus a round trip of our own outbound signature.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB
S1 and S2 of the roadmap. RemoteActorService:
- FetchObject accepts a document only when its id is the address it was
served from; a same-origin document naming another address is asked for
at that address once (how GoToSocial serves its key URIs), anything else
is dropped;
- GetActorByKeyId accepts a key only when the actor lists it, its owner is
the actor and it lives on the actor's origin, whether the keyId points at
the actor or at a key document;
- a refetch for a key or an actor happens at most once per five minutes,
so a bad signature cannot make us hammer a host;
- the cache row is written by one atomic upsert on ActorURI;
- every fetch is signed by the instance actor, never by the persona that
happened to receive the activity.
The inbox refuses an activity whose id is not on its actor's origin, and
an Undo of someone else's activity; a Create's object, an Update and a
Delete must be on the actor's origin too, and a cross-origin object is
refetched from its own origin before it is trusted.
Tests: a fake peer on two origins serves forged actors, foreign-owned keys,
cross-origin key documents, aliases and a GoToSocial-style key address
(integration, PRIVAPUB_TEST_MONGOD=1).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB
Infrastructure/Http adds the client the roadmap's S3 and S4 ask for:
- the connect callback resolves the name itself and refuses loopback,
private, link-local, CGNAT, documentation, multicast, ULA, NAT64, 6to4,
Teredo and IPv4-mapped/compatible forms, then connects to the vetted
address, so DNS rebinding cannot swap it afterwards;
- redirects are followed by hand, at most three, each one re-checked;
- bodies are capped at 1 MB after decompression, only JSON media types are
read, every request has a 15 s budget, and a refused URL is not asked
again for five minutes.
Actor and WebFinger fetches and inbox deliveries all use it. Test networks
can switch on Federation:AllowPrivateNetworks/AllowPlainHttp; startup
refuses both in Production.
PrivaPub.Tests (xUnit v3) starts with the address table and the fetcher's
limits against an in-process peer; build.yml and deploy.yml run it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB
Services/Federation and Controllers/ServerToServer become
Federation/{Actors,Signing,Inbox,Outbox,Rendering,Controllers}, the first step
of the roadmap's layout. No type, route or behaviour changes.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB
CLAUDE.md records what the project is (one private login owning unlinkable
personas, each its own actor), the deliberately odd route names (peasants,
mouth, anus, human-centipede, sniff/again, and the agreed ones still to come),
the federation and privacy invariants, the data and style conventions, and
how it deploys. docs/ROADMAP.md holds the intent reconstructed from the 2023
code, the gap audit, the owner's decisions of 2026-10-01 (Mastodon client
API, personas unlinkable to others, local-only range posts, communities and
circles), the libraries chosen, and phases P0-P5.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB
The tree had not compiled since its first commit: Group, DmGroup and
IGroupUsersService were referenced and never written, an IDE rename had
turned the user-settings DTO into the ViewAvatarServer enum, and the settings
were saved as an entity they no longer were.
Built now:
- Group (an ActivityPub Group actor with its own keys, members, invitation
code and optional password) and DmGroup (a conversation), with
/clientapi/group/{list,insert,update,join,leave,approve}.
- Posts and DMs: /clientapi/post/{list,insert,delete}, /clientapi/dm/{list,insert};
DM recipients are local usernames or user@host handles resolved by WebFinger.
- Invitation sign-up and login against the group's invitation code, checking
the password before any account is created.
- Federation: WebFinger, NodeInfo 2.0, actors at /peasants/{name} (Person,
Group, and an Application instance actor) with SPKI keys, draft-cavage
RSA-SHA256 HTTP signatures both ways, an inbox handling Follow (+Accept),
Undo, Create, Delete and Update, an outbox, notes at /posts/{id}, and a
persisted, retried, signed delivery queue. A post to a group is announced
by the group to its followers (FEP-1b12).
- The unused, broken typed ActivityPub models are replaced by a renderer;
NSign's HMAC setup, which could not federate, is gone.
Upgrade: net10.0, MongoDB.Entities 25.1 (instance DB API, Standard GUIDs),
Swashbuckle 10 / OpenApi 2, Serilog.AspNetCore 10, MailKit 4.18,
PasswordGenerator 3. The JWT keys are 64 bytes (IdentityModel 8 refuses
shorter for HS512). Production runs its own mongod on 127.0.0.1:27022, as
Sintopia's apps do, and deploys to privapub.thepra.dev from the build runner.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB
thepraandClaude Opus 5.5