Votes out. A persona's downvote is a Dislike (POST /api/v1/statuses/:id/downvote and /undownvote, the viewer's own as
privapub.votes.downvoted). One vote a post: a changed vote is sent as the new vote alone, as Lemmy sends one, since an
Undo of the old one beside it could arrive after it and take the new one away; Undo goes only when a vote is taken back.
A vote on a post in a remote community goes to the community, which counts it, and to the author only when on another
server: one copy a server, since PieFed drops a second copy of an activity it has just seen. Mbin keeps a favourite
apart from a vote, so there a favourite stays after a switch to a downvote.
Feeds followed (owner decision 2026-10-07: through the server). Following a feed (Lemmy's multi-community, PieFed's feed)
keeps a FeedSubscription for the persona and nothing else; the new Service privapub_feeds (LocalActorKind.Reader,
reserved by migration _017) follows every community of the feeds read here, reconciled when a persona follows or leaves
one, when a feed's list is read again (kept as it was when it cannot be read) and every six hours. Its Following rows
carry FollowerKind, so nobody's home gets what it brings in and its unanswered follows are sent again as its own. The
persona reads GET /api/v1/timelines/feed/:id (the feed's threads, ours included) and lists its feeds at GET /api/v1/feeds.
Checked in the pasture, every scenario: 873 pass. The 14 failures are Hubzilla's (identical on the previous commit:
Hubzilla no longer answers a follow in this pasture since its restore) and two activities Smithereen never sent;
followsync passes once the pasture's restore is older than the 14-day pause. Live: alice's downvotes count as downvotes
on Lemmy 1.0 and PieFed, replacing her upvote; Lemmy 1.0 and PieFed take privapub_feeds' follows and their threads reach
the feed timelines.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB
A root exports one of its personas (a job) as Mastodon's account archive, so other servers' importers read it: the
actor with its public key only, its own posts and boosts with their media, likes, bookmarks and Mastodon's CSV files,
plus PrivaPub's filters, followed hashtags, notification policy, pins, scheduled and located posts; nothing of its root,
its siblings, its keys or anyone's token. A ticket link downloads it, for a week.
An archive (PrivaPub's or Mastodon's) uploaded in pieces is imported into a persona in a job, the parts the root picks,
with progress and a stop. SafeArchive refuses links, escaping paths, duplicates, bombs and oversized items, and reads the
outbox one item at a time. Imported posts are delivered to no one, put in no home and notify nobody, yet show on the
profile, outbox, hashtags and search; back home a post keeps its id, from another actor it gets one of its date and
ImportedFromURI, so importing twice changes nothing. Relationships go through the existing services; located, scheduled
and likes only when asked; followers never.
tools/pasture/scenarios/persona-archive.sh imports mastouser's real Mastodon archive (156 posts, 24 pictures) into a
persona Mastodon follows: Mastodon receives none of it, and a second import changes nothing.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
Every audio and video upload was probed and remuxed inside the request, whatever its length; ffmpeg would read any
protocol and probe any format; `-map 0` kept the data tracks iPhones add, which mp4 refuses; nothing was ever
transcoded, so HEVC or MPEG-4 Part 2 reached browsers that can't play them, and the advertised video_matrix_limit
and frame rate limit were never applied; the output was read whole into memory, the video was saved before its
poster could fail, and the poster's frame leaked in /tmp. FLAC uploads were served as 404.
Now an upload sent to /api/v2/media is stored as sent in media-incoming (beside the media root, never served) and
answered with 202 and no url, while a ProcessMedia job, one at a time, does the work; GET /api/v1/media/:id answers 206
until it is ready, or 422 with why, and media still processing can't be posted. v1 processes before answering.
ffmpeg reads only that file (protocol whitelist, format forced from the probe) and drops data and subtitle tracks. A
video browsers play as it is (H.264, VP8, VP9, AV1 within Media:MaxVideoPixels and MaxFrameRate) is remuxed, anything
else transcoded to H.264 that fits, as Mastodon does; longer than Media:MaxSeconds is refused. Outputs move into place
only once everything succeeded, every temporary file goes, durations are kept, FLAC is served as audio/flac, and the
unit gets PrivateTmp. The instance API advertises the limits that are now applied.
No pasture scenario uploads audio or video through PrivaPub, so the sweep could not see this. MastodonMediaTests: v2
answers 202 then the job makes it playable (and an unreadable file 422 once processed), media still processing can't
be posted, MPEG-4 Part 2 becomes H.264, a video over the limit is made smaller, FLAC is served.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
Both speak Lemmy's shapes and joined the servers that get reports in that shape only once the pasture showed each keeps
one with its reason. By their source, PieFed dropped the instance actor's report (it makes a user only of a Person or a
Service) and Mbin kept it without the persona's words (it reads the reason from `summary` alone). With them in
`ServiceReportTakers`, each keeps the reports of a thread and of a comment from "Reports from privapub.test" with
alice's words, and none names her (piefed.sh and mbin.sh, 65 checks with the full sweep's 876). A report of a PieFed
account alone, which PieFed would take from the reporter, is not sent yet; INTEROP says so.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
Lemmy takes a report only from a person or a service, about one post or comment, addressed to its community, and it
answered PrivaPub's Flag (the instance actor's, an Application, with no `to` and the account and posts as its object)
400. A report of a post or comment in a community on a server whose NodeInfo names Lemmy now leaves from
`privapub_reports`, a Service with its own key that names nobody: one Flag per post, `to` the community (its own
audience, else its thread's), with the persona's words, or the category, in `summary` and `content`, sent to the
community's inbox. This is the second exception to "a server's software is for display" (owner decision 2026-10-06,
`ReportService.ServiceReportTakers`). Every other server keeps the instance actor's report. An account alone is not
reported to Lemmy, which takes no such report, and `forwarded` now says whether anything left.
The reporter is read unsigned in SecureMode and answers WebFinger like the instance actor. Nobody follows or mentions
it, the Mastodon API has no account for it, and a migration reserves its name. Checked live: Lemmy 1.0 and 0.19 keep the
reports of a thread and of a comment, with alice's words, from "Reports from privapub.test", and none names her (69
checks). A sweep of every scenario with this and the next commit: 876 checks pass; Ghost's Network feed listed alice's
post too late once, and Ghost passes alone.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
FEP-8fcf, received. When a delivery's Collection-Synchronization header digests the sender's followers on PrivaPub
otherwise than the personas following it, a job reads the list the header names (on the sender's origin, signed by the
instance actor): a follow the list leaves out ends, only when the list is the one the digest describes; a request it
lists is taken as accepted; a persona it lists that follows nothing there sends Undo{Follow}, as Mastodon does. Each
claiming delivery is compared once.
Checked live (scenarios/followsync.sh, now 15 checks): PrivaPub ends a follow Mastodon lost and undoes one only
Mastodon remembered.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
Once a follow holds, OutboxBackfill reads the account's latest public posts from its outbox's first page (twenty at
most, once a day) and keeps them as any fetched post: its profile shows them at once instead of only what it posts from
then on. Homes still get only what arrives afterwards, as on Mastodon. Announces and other servers' objects in the
outbox are left out. Checked live against Mastodon (scenarios/pins.sh, now 9 checks).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
Two owner decisions of 2026-10-05, both recorded in ROADMAP:
- After a verified Move the personas following the old account follow the new one, in the same lists, and a mute or
block of the old account carries over, as Mastodon does it.
- A direct message to one account on a server whose NodeInfo names Lemmy before 1.0 or Mbin goes as a ChatMessage,
the one place PrivaPub decides by a server's software (invariant 17). G-0008 is closed.
Mbin addresses its private messages to the recipient's profile page, so a Create addressed to a persona's /@name now
reaches the persona. Checked live: moves 8/8, Lemmy 0.19 30/30, Mbin 26/26 with messages both ways. The software
theory runs alone, since every test's peer shares 127.0.0.1.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
The context of a remote post showed only what PrivaPub happened to hold:
replies from servers nobody here follows were never seen, and only the
ancestors were ever fetched. Now a persona opening a public remote thread
queues FetchReplies for the post and its root, at most once an hour each.
The job reads the thread's own collection first (FEP-7888 `context`, which
Mastodon 4.5+ serves with every reply at any depth; posts or, as FEP-f228
allows, the activities that made them), and otherwise the post's `replies`
(PeerTube's `comments`) and the replies' own, two levels down. At most 5
pages and 100 posts a job, signed by the instance actor, never a persona;
each post is fetched from its own origin and stored through StoreContext,
so only public and unlisted ones are kept.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
POST /api/v1/statuses with scheduled_at no longer posts at once: the post is
kept as asked (at least five minutes ahead; 300 waiting, 25 a day, as
Mastodon allows), its media kept from the janitor, and a PublishScheduled job
publishes it at its time as the persona. scheduled_statuses lists, moves and
drops them; a moved post's old job finds it not due. Idempotency-Key holds
for scheduling too.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
PrivaPub now finds CDNs three ways, best first: the address ranges the
CDNs publish (Cloudflare, Fastly, Amazon CloudFront, Bunny, Gcore,
Imperva), downloaded daily by CdnUpdater and kept in CdnRangeSet; the
CDN's fingerprint in the responses it already gets from a server
(EdgeHintsHandler on the federation client); and the networks that carry
only a CDN. The fixed ASN list is gone; ASNs shared with plain hosting
(AWS, DataPacket) no longer hide a server. A server's Geo records the
CDN, its domain and how it was found, and weekly snapshots now keep the
city and coordinates too.
Servers through time (ServerPlaces): /instances/:host/history lists a
server's weekly snapshots, a CDN-fronted server's geo names the CDN's
domain and where the server was before it (before_cdn), and
/api/privapub/v1/cdns and /cdns/:domain group servers by CDN with week
by week who joined and who left. Owner decisions recorded in ROADMAP.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
- A community's announces resolve, as FEDERATION.md and ROADMAP always said Announce ids do. GroupDistributor keeps
each one it sends (GroupAnnouncement, unique by id). /grunts serves it while the post it carries is shown and 410
after, and also serves the announce-{postId} ids the group outbox lists, which now keep a stable `published`
instead of the time of the fetch.
- A persona's boost points at the boosted post: its `url` in the Mastodon API is the boosted post's page, and a browser
following the boost's id is redirected there instead of getting JSON.
- /tags/{tag}, where every Hashtag link we send points, is now a public page of this server's public posts with that
tag, with the same strict CSP and noindex as the profile pages.
- Grouped notifications (/api/v2/notifications, its unread count, a group, its accounts and dismiss). We advertise
api_versions.mastodon = 7 so clients show quotes, and clients that trust it call these; they answered 404. Likes and
boosts of one post group together, as do follows within an hour. The version string stays 4.2.0 until streaming
and Web Push exist.
- A remote account's follower, following and post counts are what its server publishes. AccountCountsJob reads its
collections' totalItems from its own origin, signed by the instance actor, at most daily and only after the account
was fetched, never when someone looks. They used to be 0.
- The other ids that do not resolve are documented as such: Update, Delete, EmojiReact, QuoteRequest and its answers,
Flag, Ignore and poll votes.
676 tests pass.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ELjqpznMFMNrJoJUj6K5p2
Owner decision 2026-10-04: fix the account privacy findings.
- Sign-in. Every failure answers "That username and password do not match." after the same work: an unknown login
is hashed against a decoy, and the comparison is constant-time. "Banned" is told only to someone who gave the right
password. This covers /clientapi/user/login, /invitation/login and /oauth/login.
- Recovery.
- Every request answers the same sentence and queues a SendRecovery job, whether or not the account exists or has an
email. The lookup, the code and SMTP move to RecoveryJob, so neither the answer nor its timing says anything.
- Codes are kept only as a SHA-256 hash, for one hour. Migration _011 drops the plaintext ones, which never expired.
- A recovered password ends every session of the root. RootSessions sets CredentialsChangedAt, which JwtEvents
checks against the JWT's issue time, now stamped as nbf, and revokes each persona's OAuth tokens and authorizations.
- Deleting a root (RootRemoval: the admin route, or the restored self-delete at /clientapi/user/delete, which asks for
the password).
- Its sessions end.
- Each persona and each group it owns sends Delete{Actor} to its followers, its members and the accounts it follows.
- The personas' posts are emptied.
- /peasants/{name} answers 410 with a Tombstone (formerType Person or Group), as do its inbox and WebFinger, through
LocalActorService.Gone. The names stay reserved.
- The root keeps only a unique `deleted-{id}` name; the second deletion on an instance used to collide on
"Deleted user".
Also, from phase 2's pasture: GoToSocial files a circle post like a DM and shows it only to accounts it mentions. Each
member's copy, and a member's refetch, now also mentions that member silently. The GoToSocial scenario checks circle
posts in conversations, like DMs, and they pass there now, as on Mastodon.
657 tests pass.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ELjqpznMFMNrJoJUj6K5p2
Off by default (Statistics:Crawler:Enabled), as the owner decided. When it is on:
- CrawlPlan runs hourly, from StatisticsSchedule. It inserts the configured seeds and
queues up to HostsPerHour servers whose last visit is older than RevisitDays, are not
paused by the breaker and are not domain-blocked, spread across the hour.
- CrawlInstance visits one server at a time as
"PrivaPub-Stargazer/<ref> (+<base>/stargazing)":
- it reads robots.txt (RFC 9309: its own group first, then PrivaPub, then *; longest
rule wins; a 4xx allows everything; a 5xx or no answer keeps it out);
- it describes servers that only crawling ever found, through
InstanceDescriber.Describe with robots.txt as the path filter;
- it reads /api/v1/instance/peers through the new GetStringArray, which keeps what it
read from the first 1 MB instead of refusing a large list;
- it adds the names a server could ever be reached at as "crawled": DNS only,
punycode, no addresses, ports or hidden services, and the reserved test names only on
a test network. Never more than MaxNewHostsPerCrawl per visit or MaxHosts in all, and
never over a touched server.
- It reads nothing but robots.txt, NodeInfo, the instance API and the peers list.
IFederationHttp.GetText serves robots.txt, and HttpScope.Crawl carries the
User-Agent.
- /stargazing explains all this and how to keep the crawler out, says whether it is on,
and credits DB-IP. GET /clientapi/admin/statistics/crawler shows the frontier.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ELjqpznMFMNrJoJUj6K5p2
- Touches: the ledger marks a server as touched when it sends us a verified activity, when
we exchange activities with it, or when we read its actors, keys, objects or WebFinger.
It upserts RemoteInstance.Seen, FirstSeenAt and LastSeenAt at most hourly per server, and
queues one DescribeInstance a week with the same dedupe key ObjectRecords uses. Suspended
servers and pages behind link previews are never described. Migration _010 marks the
servers already known as touched, with their dates.
- InstanceDescriber.Describe(host, crawled, allowed) reads:
- NodeInfo 2.2/2.1/2.0, now with its published user counts, posts, comments,
description, languages and schema version;
- for software with a Mastodon API, /api/v2/instance falling back to v1: title,
languages, registration mode, character limit, API version, source URL.
It never keeps a contact as a field; the raw document is kept for the admin only. It
locates the server from the address our connection reached (DB-IP Lite city and ASN, the
CDN named when fronted) and writes a RemoteInstanceSnapshot per ISO week, unreachable
weeks included. A crawled server is upserted as crawled only on insert, so it never
downgrades a touched one, and robots.txt can deny any path.
- PublicGeo.Project is the only public form of a location: a CDN-fronted server shows its
CDN only, a server reporting at least ten users shows its city, coordinates and network,
any other only its country.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ELjqpznMFMNrJoJUj6K5p2
A RollupDay job folds each finished day's InteractionEvents into one InstanceDay per server:
- Counters for the admin, keyed channel:activity:object:outcome:reason, plus signature
schemes, audiences, local kinds and features;
- PublicCounters, everything a public page may ever read:
- inbound and outbound activities from an allowlist, on public, unlisted or unaddressed
traffic only, with the outcome collapsed (accepted or dropped, delivered or failed);
- no reasons, no Flag or Block;
- features of public objects;
- health:ok or health:failed from reachability (a 4xx means the server answered);
- latency, wait and byte histograms, and the number of distinct accounts from that day's
hashes.
Re-running a day replaces it and keeps the live Reads counters. The day's salt is then
deleted, so its hashes can never be recomputed, and the next day is queued.
StatisticsSchedule plans today's rollup every hour and catches up any of the last seven
days that have events but no rollup. Waits round up into their bucket.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ELjqpznMFMNrJoJUj6K5p2
- A public post that links to a page gets a preview card: from the post's own data first (FEP-8967 Link preview, the
object's image, title and summary); otherwise the server reads the page once, 0-60 s after the post arrives, never
when someone reads it. OpenGraph and Twitter tags give title, description and image; one LinkPreview per address
is cached for 7 days and shared by the whole server, so a fetch never points at a persona.
- Lemmy link posts, which carry no title or description, get them filled in.
- The page fetch uses the guarded client (public addresses, three redirects, HTML only, first 512 KB).
- Federation:FetchLinkPreviews switches page fetching off.
- Local public posts get cards too.
Checked live: a link to a GoToSocial profile page becomes a card with its title, description and proxied image.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB
- Custom emoji (Emoji tags) on posts, display names, bios and profile fields, at most 64 per object, proxied, in
Status.emojis and Account.emojis.
- Remote profiles keep their header, profile fields, locked flag, published date, movedTo, indexable, memorial and
image descriptions (a locked GoToSocial account no longer shows as open).
- Polls: incoming Questions (Mastodon, Misskey, Pleroma, GoToSocial shapes) with counts, voters, end and closed;
our own polls from the Mastodon API go out as Questions; votes in are counted once per voter and never become
replies; personas vote on other servers' polls with one Note per choice; counts refresh with an Update at most every
three minutes; a poll closes on time and tells its voters and its author. GET /api/v1/polls/:id and POST
/api/v1/polls/:id/votes.
- An Update without a newer `updated` only refreshes poll, video, audio and event details and leaves no revision.
Checked live against GoToSocial: each side's poll reaches the other as a poll and each side's vote is counted.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB
- ObjectRecord, one per stored remote object: the raw JSON (up to 256 KB, always hashed), delivered or fetched,
refetched from origin or not, the activity that brought it (or caused the fetch), shared or personal inbox, the
signature's key, algorithm and signed headers, received time, published and updated, the delivering activity's
@context, and up to ten later revisions from Update.
- Delivery details travel from InboxReceiver through the inbox job to the handlers as Arrival.Current.
- A host is described from its NodeInfo when we first hear from it, at most weekly (DescribeInstance job), never when
someone opens the details view.
- GET /api/privapub/v1/statuses/:id/provenance and /api/privapub/v1/instances/:host, with the extensions an object
used detected from its raw form (044f quotes, interaction policies, contexts, proofs, Misskey fields, MFM, FEP-8967
links, emoji, polls, language maps, url variants, Markdown content).
Checked live: a GoToSocial reply shows as delivered to the shared inbox, signed hs2019 with GoToSocial's fragment-less
key id, with its interaction policy detected, and gts.test is described as gotosocial 0.22.1.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB
Fanout writes a TimelineEntry for every persona a post should reach: the
author, local followers of a local author, local followers of a remote one,
the members of a direct conversation or a circle. Mastodon's home rules
apply when it is written: a reply shows only to followers of both sides
(or to the one replied to), a reblog only where reblogs are wanted. A
mention of a local persona becomes a Mention notification.
Inbound posts are now also kept when a persona follows their author.
RemotePosts holds what CreateHandler and backfill share: building a Post
from a note, and fetching a public parent the first time a reply to it
arrives, so its author is known (a reply to an unknown or non-public
parent stays out of home timelines). Each fetched ancestor queues a
FetchAncestors job for the next one, up to ten deep.
/clientapi/timeline/home and /clientapi/notifications (with
/notifications/read) page by max_id for the persona's own root only.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB
The single serial DeliveryWorker is replaced by Infrastructure/Jobs:
- Job rows are leased with one FindOneAndUpdate (oldest RunAt first, a
two-minute lease) and a reaper returns expired leases every 30 s;
- enqueueing wakes the workers, which otherwise poll every five seconds;
- delivery runs eight at a time with at most two per host, so a slow or
dead server holds two slots, not the queue;
- a failure waits n^4 + 15 + jitter seconds (Mastodon's curve) for up to
16 attempts; a 4xx other than 408/429 is final, a 429 honours
Retry-After;
- RemoteInstance is a per-host circuit breaker: ten consecutive failures
quarantine a host for an hour, doubling to a week, and its jobs wait
without spending attempts;
- a delivery is queued once per activity and inbox (unique DedupeKey), and
finished jobs expire after seven days (TTL on FinishedAt).
Migration _004 moves pending Delivery rows into jobs and marks them
abandoned, so a rollback to the old worker cannot send them twice.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB
thepraandClaude Opus 5.5