SocialPub/docs/ROADMAP.md

PrivaPub roadmap

Written 2026-10-01 from the original 2023 code, the decePubClient UI, a federation gap audit of commit 075c222, research on .NET ActivityPub libraries, and the owner's decisions. Each phase ends in a tagged deploy plus verification; tick phases off here as they land.

Status

• P0 Security baseline: v1.1.0, deployed and verified 2026-10-01

• P1.1 Infrastructure and content model: v1.2.0, deployed and verified 2026-10-01

• P1.2 Social graph and timelines: v1.3.0, deployed and verified 2026-10-01

• P2 Mastodon client API: v1.4.0, deployed 2026-10-01; OAuth and the anonymous API verified on production, the signed-in API verified locally (no real-client login on production yet)

• P3 Social features: v1.5.0, deployed 2026-10-01; media, proxy, blocks, mutes, bookmarks, pins and reports verified by tests and locally (no upload on production yet)

• P4 Groups and privacy features: v1.6.0, deployed 2026-10-01; communities, circles and local-only located posts verified by tests. v1.6.1 adds the pasture (tools/pasture/): live interop with GoToSocial 0.22.1 passes all 25 checks, three runs in a row. Lemmy and a live Mastodon circle member are not run yet; the pasture has GoToSocial only

• P5 Lose nothing: v1.7.0 to v1.9.1, deployed 2026-10-01; parsing, typed details, provenance, downvotes, tombstones, federated blocks, all checked live against GoToSocial. Book reviews and forum threads keep their raw form only (typed in P7 and P8)

• P6 Emoji, polls, quotes, reactions, cards, players: v1.10.0 to v1.15.0, deployed 2026-10-01; polls, link cards, ranged video streaming and quote policies checked live against GoToSocial

• T/M Test sweep, the full pasture and the interaction ledger (see "Sweep and statistics" below). T1–T4: v1.15.1, deployed and verified 2026-10-03 (CI runs all tests on a throwaway mongod; nothing answers 500). M1–M6: v1.16.0, deployed and verified 2026-10-03 (the interaction ledger records every inbox answer, handler verdict, delivery attempt and outbound request; the GoToSocial pasture passes 33/33 with no names in the statistics). T5–T9, T11 and M7–M11: v1.17.0, 2026-10-03:

  • 639 tests, now over HTTP too;
  • daily rollups, every touched server described and located, the admin statistics API, the opt-in crawler and /stargazing;
  • the pasture as plugins: GoToSocial 37/37, Mastodon 49 plus 2 expected failures.

  T10 and T12–T14: v1.17.1, deployed and verified 2026-10-03. The pasture runs all six peers in one pass:

  • GoToSocial 54 plus 1 expected failure;
  • Mastodon 49 plus 2;
  • Misskey 35, Sharkey 40, Akkoma 44;
  • Lemmy 20 plus 3.

  The expected failures are circle posts on GoToSocial and Mastodon, inbound Block, and Lemmy's relayed votes and removals (P7). It found three bugs, all fixed:

  • quick GoToSocial edits were lost;
  • followers-only posts arrived as DMs on Pleroma and Akkoma;
  • Akkoma's open polls showed as ended and refused votes.

  Both open items were closed without a root step (owner decisions, 2026-10-04): the server fetches its geolocation databases itself, and the deploy makes and signs in as @thepra.

• Everything on in production (owner decisions 2026-10-04):

  • v1.18.0, deployed and verified 2026-10-04: geolocation that updates itself (DB-IP Lite 2026-10 loaded), the deploy signing in as @thepra (undiscoverable), the crawler on (1010 servers known within the hour), sign-up by invitation with one registrations switch;
  • v1.19.0, 2026-10-04: circle posts reach Mastodon and GoToSocial members (each copy names and mentions its member), followers-only, direct and circle posts served to signed refetches from those they were for, SecureMode on (all six pasture peers pass under it), and account privacy (sign-in and recovery say nothing, recovery codes hashed for an hour, a recovered password ends every session, a deleted root's personas and groups are deleted everywhere);
  • still to come: one answer everywhere (the mismatch sweep).

• P7 Threads, communities, moderation, the social graph

• P8 Signatures, discovery, the long tail

Intent

It is a self-hosted, Pleroma-like microblogging server for privacy-minded individuals and small communities. It federates with Mastodon, Pleroma/Akkoma, Misskey and GoToSocial (AvatarServer enum). Its distinguishing idea is one private login (RootUser) owning several public personas (Avatars), each a separate actor with its own keys. The rename SocialPub → PrivaPub happened in the commit that added the private avatar service.

Its privacy additions on top of the usual fediverse features:

• Location-ranged posts: Post.Location and RangeKm (5 km default), currently unused.
• Contacts a persona chooses to share: Avatar.SharedPersonalContacts, plus a root contact book (ContactItem).
• Private notes: about other accounts (RootUserNote) and on one's own avatar (PersonalNote).
• Invitation-gated groups and DM groups.
• Signup without email: email is only for password recovery.

The client (decePubClient) is a near copy of Pleroma-FE plus Pleroma's admin panel: a visibility picker, a subject line, Plain/HTML/Markdown content types, media with alt text, boosts and likes, threads, mutes and blocks, data import/export, and admin sections for users, reports, emoji, MRF policies and uploads. Today it shows mock data and calls nothing.

Leftovers of the collAnon template, not intent:

• the discussion and confrontation resources;
• collAnon's invitation flow;
• QR scanning;
• the "collAnon support" mail sender.

The finishing pass reinterpreted the owner's never-written Group and DmGroup as an ActivityPub Group actor with an invitation code and a direct-message conversation. The owner confirmed it on 2026-10-01, with groups split into communities and circles (see Owner decisions).

Status at HEAD:

Status	Features
Works	Accounts and JWT, several avatars, per-avatar actors and keys, posts, replies, CW, delete, DMs, groups and invitations, WebFinger, NodeInfo, signed inbox for Follow/Undo/Create/Delete/Update, delivery queue
Missing	Following remote accounts (no outgoing Follow), home/local/federated timelines, notifications, likes, boosts, visibility other than public, media, polls, edits by the author, profile Update federation, mutes, blocks, reports (Flag), Move, custom emoji, the location and contact features, admin and moderation, and a client that talks to the server

Persona separation is breached in three places today:

• invitation signup names the avatar after the root username;
• logs pair root IDs with IP addresses;
• NodeInfo counts avatars as users.

Where it stood

Works:

• WebFinger, including acct: for groups
• NodeInfo 2.0
• Person, Group and Application actors with SPKI keys
• Outbound cavage signatures (every fetch is signed, so authorized-fetch servers work)
• Inbound cavage verification, with key refetch and a key-owner check
• Shared inbox, durable delivery with backoff
• Follow with auto-Accept or manual approval for groups, and Undo
• Inbound public, group and DM Creates
• Delete and Update restricted to the author
• DMs both ways, with Mention tags

Critical (security):

• S1, actor/key cache poisoning: RemoteActorService.Upsert and GetActorByKeyId accept any document under its own id, with no origin, publicKey.id or owner checks. Any remote actor can be impersonated.
• S2: no check that an object's id host matches its actor's.
• S3, SSRF: only string checks. Names resolving to private addresses, redirects and rebinding all get through, and a request can trigger it before authentication.
• S4: unbounded fetch size, and 500s on unexpected content.
• S5: remote HTML is stored raw, with no format flag.
• S6: loose signature freshness.
• S8: DM conversation injection.
• S9: "private" groups federate as Public.
• S14: the "admin" username grants admin, and Swagger is open in production.

Blockers:

• O1: no outgoing Follow, so no home timeline.
• O2, O3: replies and mentions aren't delivered to their targets.
• I1: inbound replies and mentions are stored but invisible, and there are no notifications.
• G1: groups Announce the object URI instead of the activity, so Lemmy and other FEP-1b12 software see empty communities.
• K7: no media, timelines, likes or boosts for local users.

Important:

• Data: no indexes or unique constraints, upsert races (K1-K4).
• Inbox: processing runs synchronously with no idempotency store (I9, I10).
• Delivery: one serial worker, and a single poisoned row can stall the queue (L1-L3).
• Objects:
  • Edits and profile updates don't federate (O7).
  • No 410 Tombstones (A7).
  • Content warning without a title (O5).
  • Titles are lost on Mastodon (O6).
  • Visibility modes are missing (O4).
• Interactions: likes, boosts and reports are dropped (I2, I3).
• Media: inbound attachments are dropped, and there's no media proxy (I6, S13).
• Actor profile:
  • Actor url serves JSON to browsers (A1).
  • No attachment fields, Move or alsoKnownAs (A2, A3).
  • No locked or undiscoverable accounts (A4).
• Signatures: inbound RFC 9421 and Content-Digest (H1, H2).

Owner decisions (2026-10-01)

Question	Decision
Client interface	Mastodon client API, so Tusky, Elk, Phanpy, Ivory and the official apps work. Each avatar is its own Mastodon account: at OAuth authorize, the logged-in RootUser picks the avatar the token is for. PrivaPub-only features (avatars, groups, contacts, range posts) stay on /clientapi. Moving decePubClient onto the Mastodon API is out of scope.
Personas	Unlinkable to other users and servers. The admin can still see the link in the database. No root IDs next to IPs in logs, NodeInfo counts nothing that links personas, blocks, mutes and notifications are per avatar, and invitation signup no longer names the avatar after the root username.
Location-ranged posts	Local only, never federated. Shown to local users within the radius, with coordinates rounded on storage.
Groups	Per group, two kinds. A community federates per FEP-1b12, Lemmy-compatible: it Announces the activity, uses audience, and accepts posts from non-followers. A circle is invitation-only: posts are addressed to the members collection, and objects are served only to signed requests from members.

Owner decisions on what PrivaPub reveals (2026-10-01, from docs/INTEROP.md §6)

Question	Decision
Link previews	The server fetches the linked page itself. It does this only for public posts, after a short random delay, once per link for the whole server (a shared cache, so a fetch never points at one persona), and when the post arrives, never when someone reads it. Cards built from the post's own data are used first. The client never contacts the site.
Blocks	Federated. A persona's block is sent to the blocked account's server as Block, and an unblock as Undo{Block}. Their server enforces it too, and they can learn they were blocked. This replaces the earlier "blocks never federate" rule.
Website authorship (attributionDomains, fediverse:creator)	Off. It is never emitted, not even as a per-persona option, because it would publicly tie a persona to a website.
PeerTube views	Never sent. The remote video file is still downloaded through our proxy when it is played; that cannot be avoided without pre-downloading video.
Bluesky bridging (Bridgy Fed)	Allowed per persona, with a clear warning that the posts become far more widely copied. Leaving the bridge must work, through a federated Block sent to the bridge. Persona creation dates are moved back by a random number of days, so personas created the same day no longer share a date.
Reactions and votes	Public, with a one-time notice. The client tells each persona once, before its first reaction or vote, that these are public and visible as that persona.
Who may quote a persona (default)	Anyone, automatically, as on Mastodon, for public and unlisted posts only. Each persona can change its default (followers only, or nobody) and each post can be changed; a granted quote can be revoked. Followers-only posts and DMs can never be quoted.
Quote permission address	/peasants/{name}/parrot-licences/{id}

Owner decisions on statistics (2026-10-03)

Question	Decision
Fediverse statistics	Recorded now, published later as per-server aggregates only. The following are each kept for 90 days as an event that names the remote server, never a remote account or a local persona: every inbox answer, every processed activity, every delivery attempt and every outbound request. Each day folds into per-server counters kept indefinitely, plus weekly server snapshots. A future public page, /stargazing, shows per-server aggregates for educational use, never per account and never per persona.
Remote accounts in statistics	Never stored. Distinct accounts per server per day are counted with a keyed hash. Its key is made for that day, kept only until the day's rollup and then destroyed, so the hashes can be neither reversed nor linked across days.
Local side in statistics	Only the kind of local actor (person, group, application), and only on public and unlisted traffic. DMs, followers-only and circle traffic are one "private" class, never broken out per server in public. Circles are never named, whether as a kind or as a reason. Fetches of our own documents are counted per day, never per server.
Reading-driven traffic	Counted per day, never logged per event: the media proxy, lookups a client asks for, and the client API per endpoint group (admin only). Client app names are not recorded.
Describing servers	Every server we exchange activities with is described weekly, from its NodeInfo (including the user counts it publishes) and its Mastodon instance API, never its contact account. These requests are unsigned, because they are not ActivityPub documents. Never on read.
Server locations	City and network (ASN) from the offline DB-IP Lite databases (CC BY 4.0, attributed), downloaded monthly (by the server itself since 2026-10-04). The location comes from the address we connected to; an inbound sender's address is never recorded, and no address is stored. In public: city and network only for servers reporting at least 10 users and not behind a CDN; the country otherwise; only the CDN's name for CDN-fronted servers. The admin sees everything.
Crawler	Off by default (Statistics:Crawler:Enabled); on in production since 2026-10-04, see below. When on, it identifies as PrivaPub-Stargazer/<version> (+https://privapub.thepra.dev/stargazing), where /stargazing explains it and how to opt out. It honours robots.txt (an unreachable robots.txt means "keep out") and domain blocks. It visits one server a minute, each at most weekly, and at most 5000 servers. It reads only robots.txt, NodeInfo, the instance API and the peers list, never accounts, posts or directories. Crawled servers stay marked as crawled.
A remote account deletes itself	Its posts are kept but hidden everywhere (Post.AuthorGone): from timelines, profiles, search and lookups by id. Its follows and timeline rows go, as before.
Signed-in smoke check in production	An undiscoverable persona, the deploy checks verify_credentials, home and notifications with it. Superseded 2026-10-04: the deploy makes and keeps the persona itself, below.

Owner decisions on running everything in production (2026-10-04)

Question	Decision
What runs in production	Everything that is built is on and checked by the deploy, and nothing waits on a person running a command.
Geolocation	The server fetches DB-IP Lite itself (GeoUpdater): it checks daily, installs a new month's databases once they are published, refuses a file that does not open as the right kind of database, and keeps the old one when anything fails. No timer and no root step. The deploy fails if the databases are missing or more than 40 days old.
Crawler	On in production, seeded with a handful of large servers of different kinds (appsettings.Production.json); the deploy fails if /stargazing does not say it is on.
Sign-up	Closed: invitations only. A group invitation creates an account; open sign-up answers 403. NodeInfo, /api/v1/instance and /api/v2/instance read the same switch (Registrations:Mode), and the deploy fails if they disagree. The first login on a server is made with PrivaPub admin create-root.
Signed-in smoke check	@thepra, undiscoverable, made and kept by the deploy: PrivaPub admin smoke thepra creates or keeps the root deploy-smoke and the persona and gives the root a new password on every deploy; the deploy signs in through the real OAuth flow, checks the signed-in API and revokes its token. No secret is stored.
Signed fetches (SecureMode)	On in production once the pasture passes with it on (Phase 2 of the 2026-10-04 plan).
Circles on Mastodon and GoToSocial	Each member's copy names that member in cc; a member's refetch names the member, an instance actor's refetch the members on its server. Nothing new is revealed to anyone outside the circle.
What circles and located posts reveal	Unchanged: circles still answer WebFinger, and circle and located posts still count in a persona's post count, "a good balance for the fediverse to work".
Public /stargazing statistics	Later, as decided on 2026-10-03; the crawler and the admin API keep collecting meanwhile.

Libraries (researched; no maintained .NET ActivityPub library exists, so Letterbook and Iceshrimp.NET both wrote their own)

Area	Choice
AS2 / ActivityPub model	Own thin layer on System.Text.Json.Nodes: inbound helpers for single-or-array values, id-or-object, type arrays and the three Public forms; outbound builders with Mastodon's @context. No JSON-LD processing; fetch from the origin instead of verifying LD signatures.
HTTP signatures	Keep own draft-cavage (sign and verify). Add NSign 1.2.5 (NSign.AspNetCore) to verify inbound RFC 9421 and Content-Digest. Always answer a bad signature with 401, never 500.
HTML	HtmlSanitizer 9.x with Mastodon's allowlist (tags p br span a del pre code em strong b i u ul ol li blockquote, attributes href rel class, microformat classes, a scheme allowlist, forced rel=nofollow noopener noreferrer)
Markdown	Markdig 1.4: DisableHtml, autolinks, custom inline parsers for @user@domain and #tag emitting Mastodon's h-card and hashtag markup
Media	NetVips (+NetVips.Native) for images: autorotate, strip EXIF/GPS, thumbnails. Blurhash.Core, FFMpegCore for video. Own IMediaStore (local disk now). Not ImageSharp 4, whose license key is enforced at build.
OAuth	OpenIddict 7.7 + OpenIddict.MongoDb: authorization code with PKCE, client_credentials, oob redirect, dynamic apps from POST /api/v1/apps, reference tokens
Jobs	Own Mongo job collection for both inbox processing and delivery: FindOneAndUpdate leases, a Channel wake-up, per-host circuit breaker, Mastodon's backoff (n⁴+15+jitter, 16 tries). No MassTransit (commercial from v9, no Mongo transport).
Tests	New xUnit project with fixture JSON captured from Mastodon, GoToSocial, Misskey, Lemmy and Akkoma; interop checked with Fediverse Pasture, verify.funfedi.dev and activitypub.academy

FEPs, now: FEP-f1d5 (NodeInfo 2.0 and 2.1), FEP-67ff (FEDERATION.md), FEP-8fcf, FEP-5feb indexable, FEP-7628 Move, FEP-2c59, FEP-044f quotes (parse, plus interactionPolicy), FEP-1b12 groups.

FEPs, later: FEP-521a, FEP-8b32, FEP-e232, relays (FEP-ae0c).

FEPs, skipped: FEP-c390, FEP-ef61, FEP-fb2a, FEP-844e.

Route names (deliberate; part of the project's character)

The odd names are the owner's and stay. Every new actor-scoped route follows the same theme. The names marked new were agreed on 2026-10-01.

Thing	Route	Status
Actor	/peasants/{name}	kept (/users/{name} 301-redirects here)
Inbox	/peasants/{name}/mouth	kept
Outbox	/peasants/{name}/anus	kept
Shared inbox	/human-centipede (and /peasants/{name}/human-centipede)	kept
Token refresh (client API)	/clientapi/user/sniff/again	kept
Followers	/peasants/{name}/groupies	new, replaces /followers in P1
Following	/peasants/{name}/stalking	new, replaces /following in P1
Notes (objects)	/peasants/{name}/scribbles/{id}	new, replaces /posts/{id} in P1
Activities	/peasants/{name}/grunts/{id}	new, replaces /activities/{id} in P1
Replies, likes, shares collections	/scribbles/{id}/gossip, /drool, /echoes	new
Featured (pins)	/peasants/{name}/trophies	new
Featured tags	/peasants/{name}/tattoos	new
Group members	/peasants/{group}/flock	new
Group moderators	/peasants/{group}/wardens	new
DM conversation context	/peasants/{name}/whispers/{id}	new, replaces /conversations/{id}
Quote permission (FEP-044f stamp)	/peasants/{name}/parrot-licences/{id}	chosen by the owner, 2026-10-01

Routes outside the actor namespace stay conventional, because other software looks for them by name:

• /.well-known/* and /nodeinfo/*;
• the Mastodon API (/api/v1/*, /oauth/*);
• the human pages /@{name} and /@{name}/{id};
• /media/*.

Once a P1 deploy has federated a name, it is frozen.

Target structure (moved incrementally; entity class names and URI shapes never change)

PrivaPub/
  Infrastructure/  Http/ (IpRangeGuard, SafeHttpHandlerFactory, FederationHttp, BoundedReader), Jobs/ (JobQueue, JobWorker,
                   Backoff, HostCircuitBreaker), Ids/ (PrivacyIdGenerator, PostIds), Data/ (Indexes, Migrations/_NNN_*),
                   RateLimiting, ErrorHandling
  Federation/      Signing/ (DraftCavage ← Services/Federation/HttpSignatures.cs, SignedFetchAuthorizer, Rfc9421Verifier)
                   Actors/ (LocalActorService, Keys, RemoteActorService, WebFingerClient, ActorParser)
                   Objects/ (Origin, ObjectFetcher, NoteParser, Addressing, ContentSanitizer)
                   Inbox/ (InboxReceiver, InboxProcessor, Handlers/{Follow,Accept,Reject,Undo,Create,Update,Delete,Like,Announce,Flag,Move}Handler)
                   Outbox/ (AudienceResolver, OutboxPublisher, GroupDistributor)  Rendering/ (ApContext, Actor/Note/Activity/CollectionRenderer)
                   Controllers/ (Peasants, WellKnown, NodeInfo, Users)
  Domain/          AvatarContext, Statuses/StatusService (← Services/PostsService.cs), Timelines/FanoutService, Notifications/,
                   Relationships/, Groups/GroupService (← GroupUsersService.cs), Content/ (MarkdownRenderer, PlainTextRenderer,
                   MentionTagParsers), Privacy/VisibilityPolicy (single CanSee), Media/ (P3), Geo/ (P4)
  Api/             ClientApi/Controllers (← Controllers/ClientToServer, routes unchanged), Mastodon/{Controllers,Entities,Mappers,Auth,Infrastructure}
  Web/Pages/       Profile, Status (P1); OAuth/Login, ChooseAvatar, Oob (P2)

The first refactor commit is a pure move with namespaces only. Logic changes follow in separate commits.

Phases (each one ends in a tagged deploy plus verification)

P0 Security baseline

• S1, actor verification: RemoteActorService.
  • GetActor accepts a document only if its id equals the final URL, on the same origin.
  • GetActorByKeyId resolves the owner, then the strict actor, and requires publicKey.id == keyId and owner == id.
  • Upserts become atomic on a unique ActorURI.
  • Key refetch is limited to once per 5 minutes.
• S3 and S4, safe outbound HTTP:
  • A SocketsHttpHandler.ConnectCallback resolves DNS and rejects loopback, private, link-local, CGNAT and ULA addresses, plus NAT64, 6to4, Teredo and IPv4-mapped forms. It then connects to the vetted IP.
  • Redirects are manual (at most 3, each re-checked).
  • Responses are capped at 1 MB with a content-type check and a 15 s timeout, with a negative cache for failing URLs.
  • Exceptions map to ProblemDetails: 400 or 401, never 500.
• S2, origin checks: Origin.Same. An activity's or object's id must share the actor's origin, and cross-origin embedded objects are refetched.
• S5, sanitising: ContentSanitizer with HtmlSanitizer. Post.ContentHtml and ContentFormat are added, and migration _002 sanitises stored remote content.
• S6, signature freshness: (request-target) and host are required, plus date or (created) and digest. The window is −1 h to +15 min. (expires) is enforced, and the raw request target is used.
• S8, DM injection: a DM joins a conversation by context only if the author is already a member; otherwise it is matched on DmGroup.ParticipantsKey.
• S9, groups: a Group.Kind field is added. Existing groups become Circles (migration _003), and Circle posts stay local-only until P4.
• S14:
  • Delete the "admin" signup branch and add the CLI admin promote.
  • Serve Swagger only in Development.
  • Return generic error messages (no ex.Message).
• Privacy:
  • Drop the IP, User-Agent and root-id log lines in RootUserController.
  • Invitation signup takes its own AvatarUserName.
  • All fetches are signed by the instance actor.
• Indexes and races: Infrastructure/Data/Indexes.cs, with migration _001 deduplicating first.
  • Unique: ObjectURI, ForeignAvatar.ActorURI, the Follower triple, RootToAvatar, and ReservedName (one username space across avatars and groups).
  • Plain: PublicKeyId, (GroupUserId, _id), (GroupId, _id).
• Rate limiting on login, signup, invitation and inbox (per key host).
• Cleanup: delete Services/ActivityPubClient.cs and Models/Post/PostBoost.cs.
• New PrivaPub.Tests (xUnit v3): IpRangeGuard table, signature fixtures (a real Mastodon request plus tampering cases), key-poisoning, origin rules, XSS corpus, DM injection.
• CI: build.yml runs the tests. deploy.yml runs the tests and a mongodump before swapping.

P1 Interop foundations (two deploys)

P1.1, infrastructure and content model:

• Mongo job queue (Job, RemoteInstance):

  • Leases via UpdateAndGet, Channel wake-up, a reaper for expired leases.
  • Workers: delivery 8 (at most 2 per host), inbox 2.
  • Backoff n⁴+15+jitter, 16 tries. Non-transient errors go straight to Dead.
  • Per-host circuit breaker. Dedupe on activity.id and activityId|inbox.
  • TTL on finished jobs.
  • Migration _004 moves pending Delivery rows into jobs.

• Async inbox: InboxReceiver verifies and enqueues, then answers 202 (a duplicate also gets 202). InboxProcessor dispatches to the handlers split out of InboxService.

• Merge DmPost into Post (migration _005; ids kept; Visibility=Direct, ConversationId). Post gains:

  • Visibility {Public, Unlisted, FollowersOnly, Direct, Circle, LocalGeo}
  • AuthorAccountId, To, Cc, ActivityURI, Url, ContextURI
  • InReplyToURI, InReplyToAccountId, ReblogOfPostId
  • SpoilerText, Language, Mentions[], Tags[], Media[]
  • counters, Revisions[], DeletedAt

  /clientapi/dm/* moves onto Post, and every read goes through VisibilityPolicy.CanSee.

• Parsing inbound content:

  • NoteParser handles Note, Article, Page and Question. It reads content, then contentMap, then _misskey_content; also summary, name, sensitive, the tag array, attachment (as remote media), inReplyTo, context, audience, updated, and the quote fields (stored as a URI).
  • Addressing classifies Public, Unlisted, FollowersOnly or Direct. Followers-only is detected by the actor's stored followers URL exactly, not by a path suffix.

• Rendering local text:

  • MarkdownRenderer (Markdig) and PlainTextRenderer (for Mastodon API input).
  • Mentions resolved through WebFinger become an h-card plus a Mention tag; hashtags become a Hashtag tag.

• Renderers:

  • ApContext: Mastodon's @context (toot, schema, indexable, blurhash, focalPoint, featured, alsoKnownAs, movedTo, webfinger).
  • Actor and note url point at HTML pages; fields become attachment PropertyValue; the webfinger property is added; published is truncated to the day.
  • A title becomes name and is also prepended to the content as bold text.
  • A CW without a title gets a localised "Content warning" summary.
  • Routes are renamed to the themed set (/groupies, /stalking, /scribbles/{id}, /grunts/{id}, /whispers/{id}), activity URIs resolve, and the outbox gets a first page.

• Web:

  • Razor pages /@{user} and /@{user}/{id}, with content negotiation; /peasants/{u} with text/html redirects there.
  • WebFinger profile-page points at the HTML page.
  • NodeInfo 2.1 (avatars counted as users, openRegistrations from config) and FEDERATION.md.

• Moderation: DomainBlock (Silence, Suspend, RejectMedia), applied in the receiver, the fetcher and delivery.

• IDs: register PrivacyIdGenerator for Avatar and Group (day-truncated timestamp plus random bytes). Remote post ids are ObjectIds generated from their published time.

P1.2, social graph and timelines:

• Outgoing Follow (Following entity, plus Accept, Reject and outbound Undo). A local target is handled in-process.

• AudienceResolver and OutboxPublisher:

  Visibility	to	cc
  Public	Public	followers + mentions
  Unlisted	followers	Public + mentions
  Followers-only	followers	mentions
  Direct	mentions	(none)

  Replies are also delivered to the parent's author, and group posts to the group. Mentions are delivered to personal inboxes.

• Inbound Create:

  • The interested set is followers of the author, plus addressed or mentioned local avatars, plus the local parent's author, plus group members. If nobody local is interested, the post is dropped.
  • Store the post (idempotent on ObjectURI), then fan out to the TimelineEntry collection ({AvatarId, PostId, AuthorAccountId, ReblogOfPostId}, unique (AvatarId, PostId), paged by PostId).
  • Mastodon's reply, mute and reblog rules apply at write time.
  • Mention notifications are written.

• Notification entity: {AvatarId, Type, FromAccountId, PostId, DedupeKey}, unique on DedupeKey.

• Edits: outbound Update{Note} to the same To/Cc, with revisions kept. Profile edits send Update{Person}.

• Deletes: soft delete, a Delete sent to the stored audience, and the object then answers 410 with a Tombstone (a deleted actor answers 410 too).

• Inbound Like and Announce: a Favourite entity, reblog rows with the original refetched, Undo of both, counters and notifications.

• Threads: a FetchAncestors backfill job, depth at most 10.

• Test endpoints: /clientapi/timeline/home, /clientapi/follow and /clientapi/notifications, usable before P2.

P2 Mastodon client API

• OAuth: OpenIddict 7.7 + OpenIddict.MongoDb.
  • Storage is IMongoDatabase registered as DB.Default.Database().
  • Endpoints: /oauth/authorize, /oauth/token, /oauth/revoke and /.well-known/oauth-authorization-server.
  • Flows: authorization code (PKCE optional) and client_credentials.
  • Scopes: read, write, follow, push and the granular ones.
  • Tokens are non-expiring reference tokens. The signing and encryption keys are persistent, in /etc/privapub/oidc-*.pem.
  • The oob code page is supported, as is force_login.
  • Authorize UX: log in with the root password (a short-lived cookie limited to /oauth), choose an avatar, consent.
  • The token's sub is the avatar id, and no root claim is ever in the principal. A banned or deleted root invalidates its tokens.
  • POST /api/v1/apps creates OpenIddict applications dynamically and prunes unused ones.
• Plumbing:
  • MastodonJson: snake_case, explicit nulls and empty arrays (Tusky breaks on missing fields).
  • A binder that merges Rails-style query, form and JSON parameters.
  • {"error"} responses.
  • Link paging on max_id, since_id and min_id; CORS exposes Link.
  • Idempotency-Key honoured on posting statuses.
• Endpoints:
  • Instance: v1 and v2, advertising version 4.2.0 (compatible; PrivaPub).
  • Accounts: verify_credentials, update_credentials, :id, :id/statuses, lookup, relationships, search, followers and following, follow and unfollow, follow_requests.
  • Statuses: CRUD, edit, context, history, source, favourite and reblog plus their undos (outbound Like and Announce), reblogged_by, favourited_by.
  • Timelines and the rest: home, public, tag; notifications; markers; conversations; /api/v2/search with resolve.
  • Stubs: custom_emojis, filters, lists, announcements, trends, suggestions, followed_tags, preferences.
• Mapping:
  • Avatar, ForeignAvatar and Group all map to Account (group: true for groups). acct uses a WebFinger-verified handle; created_at is truncated to the day; avatar and header always have a placeholder URL.
  • Post maps to Status. Reblogs wrap the original. Per-viewer flags are loaded in batches.
  • LocalGeo never appears in the Mastodon API.
• New Avatar settings: IsLocked, IsDiscoverable, IsIndexable, IsBot, HideCollections, DefaultVisibility, DefaultSensitive, DefaultLanguage.

P3 Social features

• Media pipeline: MediaAttachment entity plus /api/v1/media and /api/v2/media.
  • NetVips: autorotate, strip all metadata, 4096 px cap, 640 px thumbnail.
  • Blurhash.Core.
  • FFMpegCore: remux only, with -map_metadata -1.
  • Stored in /var/lib/privapub/media, outside the web root that deploys replace. nginx serves /media/ with nosniff and a strict CSP, and allows larger uploads only on the media endpoints.
  • A job deletes unattached uploads.
• Remote media proxy: /media/proxy/{hmac}/{url}, going through the safe HTTP handler, with a ~5 GB disk LRU. The API and avatars use proxied URLs, so clients never contact remote hosts.
• Outbound attachments: mediaType, alt text, blurhash, focalPoint and dimensions. Profile avatar and header images are sent as Update{Person}.
• Bookmarks and pins: pins are served as the featured collection at /peasants/{name}/trophies, with featured tags at /tattoos.
• Blocks and mutes: per avatar. Block is not federated: it sends Reject or Undo Follow instead and drops the blocked actor's traffic. Domain blocks per account as well.
• Reports: inbound Flag becomes a Report. Outbound Flag is sent by the instance actor, so the reporting persona isn't revealed. Moderator endpoints on /clientapi.
• Locked accounts: the follow-request flow.

P4 Groups and privacy features

• Group model: Group.Kind, PostingPolicy, Rules; GroupMember.State; new collections for moderators (/wardens), members (/flock) and featured (/trophies).
• Communities (FEP-1b12):
  • GroupDistributor announces the full activity (Create, Update, Delete, Like or Undo) with audience to the group's followers. New posts additionally get Announce(object), so Mastodon shows them.
  • Top-level posts render as Page with a title; comments as Note.
  • Posting from non-followers is allowed according to PostingPolicy.
  • Inbound Announces from remote groups are refetched from their origin.
  • A Mastodon client posts into a group by mentioning @group@host.
• Circles:
  • The actor is undiscoverable, with manual approval and invitation-only joining.
  • Posts are addressed to: [members, group] and delivered to members' personal inboxes, never with Announce.
  • SignedFetchAuthorizer serves circle objects only to a signed request from a member, or from the instance actor of a member's server. Anything else gets 404.
  • FEDERATION.md documents the Mastodon limitation: Mastodon members' replies reach only whoever they mention.
• Local-only location posts:
  • Post.Location becomes a GeoJSON point rounded to 2 decimals (about 1 km), with RangeKm clamped to 1–50 and a 2dsphere index. Visibility is LocalGeo.
  • /clientapi/post/nearby runs $geoNear filtered by each post's own radius. The viewer's position is never stored.
  • The audience is always empty; ActivityPub endpoints and the Mastodon API never expose these posts.
• Persona hardening: optionally re-key existing avatar ids; an optional SecureMode (signed GETs required); no suggestions or directories that could relate sibling avatars.

P5 and beyond: rich content and the rest of the fediverse

Rewritten on 2026-10-01 from the research in docs/INTEROP.md, which holds the per-platform evidence and the priorities. The goal is a future client that shows and links every kind of fediverse content in one place, with a details view of where each object came from. So the server keeps everything it receives: typed where it understands it, raw where it doesn't.

P5 Lose nothing (wire tolerance and full objects)

• Cheap fixes that were wrong (done in v1.7.0):
  • summary is a content warning only on a Note/Question, or when sensitive is set; elsewhere it is an excerpt (INTEROP W2). Remote titles now show in the Mastodon API for non-Note objects.
  • Persona usernames match Mastodon's and Misskey's pattern (groups already did).
  • Every term we emit is defined in our JSON-LD context.
  • Vary: Accept.
  • Not done on purpose: see "Deliberately not done" below.
• Owner decisions that were small (done in v1.7.0): blocks federate (Block, Undo{Block}); a persona's and a group's published is a random day up to two weeks before its creation (migration _007).
• Parsing every shape (done in v1.8.0, Federation/Objects/ObjectShapes.cs):
  • url, icon, image, attachment and attributedTo as a value, an object or an array (W1).
  • A Markdown content and source (W3).
  • Inferring a missing mediaType (W4).
  • Every thumbnail location (W5).
  • Language from @context (W11); alt text from summary (W12).
  • A null id, a 200 Tombstone, a Delete before its Create (W9, W13).
• Typed payloads (done in v1.8.0 for link, video, audio, event and the cover image; review and thread are still open): Post.ObjectType plus Link, Video, Audio, Event, CoverURL and Source (INTEROP §4.1). The Mastodon API view: the title above the body, an event's "when · where" line, a card made from the object without fetching, attachments with their thumbnail and duration. Video and audio playback through the proxy is P6.
• Raw capture for the details view (done in v1.8.0): the object as received, how it arrived, the signature scheme and key, received versus published, the extensions detected, and the origin's software (§4.3). Exposed at /api/privapub/v1/statuses/:id/provenance and /api/privapub/v1/instances/:host. Checked live against GoToSocial.
• Routing by object type (done in v1.9.0):
  • Accept/Reject act only on a Follow we sent and ignore any other object (W6); routing to RSVPs and interaction approvals comes with those features.
  • ChatMessage in as a direct message.
  • Dislike and its Undo in a downvote ledger, shown with favourites as privapub.votes.
  • A Join answered with Ignore until RSVP exists.
  • Friendica's thread-Follow is ignored without an error.
  • A deleted object's id is kept for 90 days (W9), and its ObjectRecord is removed with the post.
• Typed details reach clients (done in v1.9.0) as Status.privapub.
• Delivery: a 503 with Retry-After is waited out like a 429 and no longer counts against the host (done in v1.9.0). We answer 503 with Retry-After ourselves when a sender's key cannot be fetched for a temporary reason (v1.9.1), so Mastodon 4.7 retries instead of switching to RFC 9421.
• Deliberately not done: dereferenceable Follow, Like, Block, Accept, Reject and Undo ids. Serving them would publish who follows, likes and blocks whom, which our collections deliberately hide. Every one of them is sent with its object embedded, which is all Misskey needs. Create and Announce ids do dereference.

P6 What people see: emoji, polls, quotes, reactions, cards, players

• Custom emoji on posts, names, fields and poll options, proxied (done in v1.10.0), together with fuller remote profiles: header, fields, locked, published, moved-to, indexable, memorial, image descriptions.
• Polls in and out, with the Misskey, Pleroma and PieFed vote shapes. A count refresh is never an edit (done in v1.10.0; checked live both ways against GoToSocial).
• Quotes (FEP-044f):
  • read every key; verify QuoteAuthorization on all of its fields; handle revocation (done in v1.13.0);
  • personas quote others: quoted_status_id, QuoteRequest, Accept{result} verified, Update with quoteAuthorization; legacy quotes for posts that state no policy (done in v1.13.0); api_versions.mastodon = 7;
  • be quotable (done in v1.15.0): canQuote on our posts (owner's default: anyone, automatically), QuoteRequest answered with a parrot-licence or Reject, licences served and revoked; personas set source[quote_policy], posts quote_approval_policy; POST /statuses/:id/quotes/:quoting_id/revoke, PUT /statuses/:id/interaction_policy.
• Emoji reactions in all three inbound forms, plus outbound EmojiReact, exposed as emoji_reactions (done in v1.11.0; Pleroma's /api/v1/pleroma/statuses/:id/reactions endpoints and pleroma:emoji_reaction notifications).
• Link cards (done in v1.12.0):
  • from the object, or from FEP-8967 preview, without fetching;
  • otherwise the server reads the page (owner decision 1): public posts only, 0–60 s after arrival, cached per address for 7 days across the whole server, Federation:FetchLinkPreviews to switch it off. Checked live against a GoToSocial profile page.
• Media (done in v1.14.0 except where noted):
  • video playback through the proxy: range requests streamed from the origin, the poster, a playable attachment picked from MP4 or fragmented MP4 files (HLS playlists are not rewritten; not needed while PeerTube publishes the files);
  • audio attachments;
  • JPEG/PNG for Pixelfed: uploads were already re-encoded to JPEG, PNG or GIF;
  • an article reader view: the data is in Status.privapub (title, excerpt, cover, full HTML in content); the view itself belongs to the client.

P7 Threads, communities and the social graph

• Thread backfill:
  • read in order: contextHistory, then context (paged, with ETag), then replies;
  • group by the root post;
  • publish our own context and a paged replies.
• Lemmy, PieFed and Mbin:
  • the moderation set: removals, locks, bans, featured, moderators, Warn, Resolve;
  • votes in and out; link posts; flairs; Feed actors; community polls; post Move;
  • the outbound shape Lemmy requires;
  • communities we host announce to the author's own instance too;
  • Flags from a Service-typed reporter actor.
• GoToSocial interaction policies: stored and shown; send ReplyRequest/LikeRequest where approval is needed; handle Accept{result}.
• Accounts and follows:
  • inbound Move with Mastodon's checks;
  • re-run WebFinger on a rename;
  • inbound Block, plus Add/Remove of pins;
  • FEP-8fcf followers sync;
  • indexable/discoverable/searchableBy;
  • edit history from formerRepresentations;
  • PeerTube reply rules and ApproveReply.
• Events: structured RSVP (Join/Leave with stable ids).

P8 Signatures, discovery and the long tail

• Signatures:
  • RFC 9421 inbound (RSA and Ed25519, Content-Digest);
  • outbound double-knock, remembered per host;
  • publicKey arrays and FEP-521a Multikey;
  • FEP-8b32 proof verification;
  • hs2019 with SHA-512.
• Discovery:
  • a relay client for both relay styles;
  • instance actor discovery (FEP-d556, FEP-2677);
  • implements (FEP-844e).
• Mastodon API: streaming WebSocket, Web Push, grouped notifications. Then advertise an honest version.
• Backfill: an author's outbox after following them.
• Long tail:
  • MFM rendering data;
  • Misskey actor extras;
  • book reviews (relatedWith, rating);
  • Funkwhale and Castopod metadata;
  • Pixelfed place (display only, never re-federated);
  • Bluesky bridging per persona.

Sweep and statistics (T and M, planned 2026-10-03, between P6 and P7)

Two tracks, interleaved so that the test host exists before the ledger, and the ledger starts collecting early, because statistics gain value with every day recorded.

Step	Content	Tag
T1–T4	Tests stop sharing state they don't own. CI runs every test against a throwaway mongod. The whole server runs under test (WebApplicationFactory). Nothing answers 500.	v1.15.1
M1–M6	The interaction ledger: inbox answers, handler verdicts, delivery attempts, outbound requests, served and client traffic. Provenance fixes (fetched records, stored extensions, group-wrapped Update/Delete).	v1.16.0
T5–T8	Coverage sweep over HTTP: OAuth, /clientapi, the Mastodon API, federation GETs, inbox gaps, jobs, migrations, pages.	v1.17.0
M7–M10	Daily rollups (InstanceDay, ServerDay). Every touched server described weekly (NodeInfo usage, instance API, snapshots). Geolocation (DB-IP Lite city and ASN). Admin statistics API under /clientapi/admin/statistics.	v1.17.0 (planned v1.18.0)
T9–T14	The pasture as plugins. GoToSocial gaps, then Mastodon, Misskey/Sharkey, Akkoma and Lemmy 1.0, each run also checking that peer's statistics.	v1.17.0, v1.17.1
M11	The opt-in crawler (PrivaPub-Stargazer) and the /stargazing explainer.	v1.17.0 (planned v1.19.0)

Later, and not part of these steps: the public /stargazing statistics. It is anonymous, cached and rate-limited, and covers per-server software, self-published counts, location as projected by the rule above, availability buckets and the public activity mix. Per-server activity is shown only as order-of-magnitude buckets, and only for servers that report at least 10 users; smaller ones fold into one "small servers" aggregate.

Cut or deferred (deliberately)

• Cut:
  • Link-preview cards fetched from the linked page by default. Cards built from the object, or from FEP-8967 preview data, are in P6; fetching the page is an owner decision (INTEROP §6.1).
  • Translation, trends, directory and lists (filters stay as stubs).
  • Scheduled posts.
  • The Mastodon admin API: moderation stays on /clientapi.
  • POST /api/v1/accounts registration: avatars are created through /clientapi.
  • Local custom emoji.
  • S3 storage.
  • JSON-LD and LD-signature processing. FEP-8b32 proofs need neither (JCS), so they are in P8.
• Deferred: video transcoding (remux only for now).
• Out of scope: moving decePubClient onto the Mastodon API.

Verification (per phase)

• Unit tests (PrivaPub.Tests) on every phase. Fixture JSON is captured per peer into Fixtures/{mastodon,gotosocial,misskey,akkoma,lemmy}/. Renderers are checked against golden JSON. Inbound scenarios run as integration tests on a throwaway mongod.

• Interop: Fediverse Pasture runs on the workstation with podman: Mastodon, GoToSocial, Misskey/Sharkey, Akkoma, plus Lemmy for P4. As built, tools/pasture/run.sh runs PrivaPub and GoToSocial behind one Caddy on a podman network, with AllowPrivateNetworks, AllowPlainHttp and AcceptAnyCertificate; startup refuses all three in Production. tools/pasture/interop.sh drives the matrix for GoToSocial; the other peers are still to be added. The matrix, per peer:

  • follow both ways;
  • public, unlisted, followers-only and direct posts both ways;
  • a reply landing in the remote thread;
  • a mention producing a notification;
  • edit, and delete answering 410;
  • like and boost counters;
  • CW and title rendering on Mastodon;
  • Pasture's odd-payload inputs always answered 202 or 4xx, never 500.

  Also run verify.funfedi.dev against /peasants/privapub and one avatar.

• P0: curl the inbox with junk and get 400/401. Resolving 127.0.0.1.nip.io or a host with an ::1 record is refused. Swagger answers 404 in production.

• P1: 500 deliveries queued for a dead host don't delay deliveries to live hosts.

• P2: Tusky, Elk, Phanpy and Ivory each log in with two avatars of one root as separate accounts. Home paginates both ways; post, reply, DM, follow-by-search, notifications, CW, and edit or delete-and-redraft all work. tools/smoke/mastodon-api.sh runs after every deploy. A test asserts that no response for avatar A contains avatar B's id.

• P3:

  • exiftool shows no GPS data on an uploaded original.
  • Elk renders the blurhash.
  • Phanpy's network tab shows no requests to remote hosts.
  • Like and boost round-trip with Mastodon and GtS, and a block cuts the follow both ways.
  • A Flag reaches the Mastodon admin UI.

• P4:

  • Lemmy follows a community; posts and comments flow both ways; votes show; moderator deletes propagate.
  • A Mastodon member of a circle gets a limited post. An unsigned or non-member GET answers 404.
  • Two local avatars, 3 km and 8 km away, see a 5 km post correctly, and the post creates no delivery jobs.

• Every deploy: /build.json reports the commit. privapub.thepra.dev/peasants/privapub and NodeInfo answer.

Risks and sequencing

• Unique indexes fail on existing duplicates, so the dedupe migration runs first. Run a mongodump before every tagged deploy from P0 on.
• Merge DmPost into Post and register PrivacyIdGenerator before P2 exposes ids to clients.
• The publish grows by about 20 MB (OpenIddict, NetVips.Native, FFMpegCore). Keep deploy.yml's asserted file list current, and add ffmpeg in deploy/max/setup.sh.
• Pasture's private-network and plain-HTTP allowances stay out of appsettings.Production.json, and startup asserts they're off.
• Scale: this is several weeks of work. Each phase is a separate tagged release, and I report after each one.