18 Commits
Author SHA1 Message Date
thepraandClaude Opus 5.5 2645dea26f T8: inbox gaps, jobs, migrations and pages; a deleted remote account's posts are hidden
Owner decision (2026-10-03, "A remote account deletes itself"): its posts are kept but
hidden everywhere.
- Post.AuthorGone (additive bool). DeleteHandler's actor-delete branch sets it on every
  post whose ActorURI is the actor (one update-many), besides dropping its follows and
  timeline rows as before. RemotePosts.Build sets it on a post stored later for an
  account already marked Deleted.
- One rule in VisibilityPolicy: IsShown (not deleted, author not gone), IsPublic and
  CanSee exclude AuthorGone, plus Shown(post) for loaded posts.
- Lookups by id answer 404 through CanSee (statuses/:id and every sub-route, context,
  bookmarks, favourites, polls, reactions, search); provenance, account statuses,
  home/public/tag timelines, notifications, conversations, reblogged_by, the clientapi
  home and post/DM lists, a community's outbox and our Announces filter on IsShown or
  IsPublic; the Mastodon mapper never renders a hidden post or a boost of one.

Tests (30 new):
- AuthorGoneTests: the rule, the handler (posts kept, boosts included, follows and rows
  gone), a post fetched after the delete, and 20 Mastodon/ActivityPub lookups over HTTP
  seen before and hidden after.
- InboxGapTests: actor Update refresh (name, sanitised summary, key rotation in place
  and to a new key id) even with an older `updated`; Undo{Follow} by activity id and by
  object; Reject of our QuoteRequest (and a stranger's ignored); group-wrapped
  Announce{Like} and Announce{Undo{Like}}; a locked persona's pending follow,
  FollowRequest notification, and Decide accepting and rejecting with the original Follow.
- JobHandlerTests: AncestorsJobHandler up to its depth limit; PollRefreshJob and
  PollCloseJob (local and remote polls); InstanceDescriber from a peer's NodeInfo and
  the weekly dedupe through ObjectRecords; LinkPreviews for public posts only;
  DeliveryJobHandler outcomes (2xx, 404/410, 429/503 with Retry-After in seconds and as
  a date, 5xx) and a signature and Digest the peer can verify; MediaJanitor.Sweep;
  OAuthPruner.Prune.
- MigrationTests: _003, _004, _006 and _007 on seeded rows.
- PublicPagesTests: /@user and /@user/{id} (visibility, junk ids, exact CSP,
  Referrer-Policy and nosniff), circle 404, community page, the instance actor,
  ActivityPub redirects, and markup escaped in posts, titles and bios.

Production changes besides the rule:
- LinkPreviews.Handle re-checks that a post is still shown and public (the rule
  Wanted applies) before fetching anything; before, only enqueueing checked it.
- The legacy /clientapi post and DM lists no longer return soft-deleted posts.
- MediaJanitor.Sweep and OAuthPruner.Prune are the loop bodies, now public and tested.
- InstanceDescriber.Address: a protected virtual identity seam so a test can point
  the https NodeInfo addresses at a plain-http peer; production behaviour unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ELjqpznMFMNrJoJUj6K5p2
2026-10-03 11:53:23 +02:00
thepraandClaude Opus 5.5 2cfea7b60c T7: the federation surface over HTTP
Tests through the real routes of PeasantsController, WellKnownController and
UsersController, on the whole server under test (34 new tests):

- FederationGetTests: the actor document (activity+json, SPKI key at
  #main-key owned by the actor, sharedInbox, published = PublishedOn's day,
  no creation date, no root); ld+json; browsers sent to /@name; Vary: Accept;
  /users 301; the outbox's totalItems and ?page=true&max_id paging across
  the 20-item boundary, boosts as Announces, and no followers-only, direct,
  located, federated-copy or deleted post; /groupies and /stalking naming
  nobody; /trophies (public pins, newest first) and /tattoos; /scribbles
  (public and unlisted 200, browsers redirected, followers-only, direct and
  located 404, deleted 410 Tombstone); a circle post only for a signed member
  or its instance actor; a circle's /groupies, /flock and /wardens only for
  members; /grunts create- and announce- ids; /parrot-licences 200, revoked
  410, wrong author 404; secure mode's 401 for every unsigned GET but the
  instance actor's.
- WellKnownTests: WebFinger by acct:, @-prefixed, bare, upper-case and actor
  URI; other domains, unknown names, a root's login name and no resource;
  the instance actor, a community, a circle (answered: current behaviour);
  NodeInfo links, 2.0 and 2.1 naming no root, unknown versions 404; usage
  counting only public, unlisted, non-boost local posts (Exclusive).
- InboxRouteTests: all three inboxes accept a signed delivery and refuse
  junk (400), unsigned (401), a bad Digest, a two-hour-old Date, a signature
  for another host and a swapped body (401); an unknown persona's /mouth is
  404; ld+json with the ActivityStreams profile is accepted; a signer whose
  actor answers 503 gets 503 with Retry-After; the 301st unsigned POST from
  one address is 429 while a signed server from it is not.
- PersonaSeparationHttpTests: with a sibling persona and its community on
  the same login, every GET under /api (filled with the persona's ids) plus
  search, lookup and relationships, and a crawl of everything federation
  publishes about the persona (actor, outbox pages, collections, scribbles,
  grunts, WebFinger, NodeInfo, /@ pages), never name the sibling, its
  community or the login.

Fixed:
- A circle's /groupies told anyone how many followers (members) it has,
  while its /flock and /wardens were already for members only; it now
  answers 404 to anyone but a signed member or a member's instance actor.
- WebFinger answered 404 to a bare user@domain or @user@domain resource,
  which Mastodon, GoToSocial and Pleroma all accept; it now treats them as
  acct: (noted in docs/INTEROP.md).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ELjqpznMFMNrJoJUj6K5p2
2026-10-03 11:53:23 +02:00
thepraandClaude Opus 5.5 4fa53f63bd M2: every inbox answer is recorded
InboxReceiver records each answer once, in a finally, with a reason: too-large, not-json,
not-activity, missing-type-or-actor, no-signature, the signature check's own codes
(headers-unsigned, digest-mismatch, header-unreadable, date-skew, expired,
algorithm-unsupported), signature-invalid, actor-not-key-owner, key-unavailable,
id-cross-origin, undo-foreign, misattributed, unknown-recipient, and for 202s queued,
duplicate, suspended or self-delete-unknown-key. Each event carries the activity and object
type, the inbox, the signature scheme, the bytes and the time taken. A 404 for an unknown
/mouth and a rate-limited inbox (429, from OnRejected) are recorded too.

Until the signature verifies, the host is only claimed, so it is kept only if the server is
already known. A suspended server is recorded under its own name.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ELjqpznMFMNrJoJUj6K5p2
2026-10-03 10:59:55 +02:00
thepraandClaude Opus 5.5 e6729c77e7 P6 done: personas can be quoted, under the owner's default of anyone, automatically
Build / Build (push) Successful in 1m7s
Deploy / privapub.thepra.dev (push) Successful in 1m15s
- Our public and unlisted posts state interactionPolicy.canQuote. The policy comes from the post, then the persona
  (`source[quote_policy]`: public, followers or nobody; default public as the owner chose), and is always nobody for
  followers-only posts and DMs.
- QuoteRequests are answered with Accept{result} naming a parrot-licence at /peasants/{name}/parrot-licences/{id}
  (the route name the owner chose), or with Reject. Followers-only checks that the requester really follows.
- A licence is a QuoteAuthorization naming both posts; revoking it (POST /api/v1/statuses/:id/quotes/:quoting_id/revoke)
  marks it 410, sends Delete{licence} to the quoter and the persona's followers, and revokes our own copy of the quote.
- A quote that arrives with one of our licences is accepted only if that licence is ours, unrevoked and names exactly
  that quoting post. One persona quoting another gets a licence too.
- Mastodon API: quote_approval for our posts (automatic, followers, current_user), `quote_approval_policy` when posting,
  PUT /api/v1/statuses/:id/interaction_policy, `source.quote_policy`.

Checked live: GoToSocial still accepts our posts with the policy stated, and leaves likes, replies and boosts open.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB
2026-10-01 19:53:13 +02:00
thepraandClaude Opus 5.5 81de470f64 P5 done: a key we cannot fetch for now gets 503, and follow/like/block ids stay private on purpose
Build / Build (push) Successful in 59s
Deploy / privapub.thepra.dev (push) Successful in 1m11s
- When a sender's key cannot be fetched because its server timed out or answered 5xx, the inbox answers 503 with
  Retry-After: 300 instead of 401, so Mastodon 4.7 retries rather than switching to RFC 9421 signatures we do not
  verify yet. The fetcher's failure cache now remembers whether a failure was temporary.
- Follow, Like, Block, Accept, Reject and Undo ids are deliberately not dereferenceable: serving them would publish
  who follows, likes and blocks whom. They are always sent with their object embedded.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB
2026-10-01 18:25:18 +02:00
thepraandClaude Opus 5.5 bb9bd71391 P5, first batch: summaries stop hiding articles, personas get reachable names, blocks federate
Build / Build (push) Successful in 50s
Deploy / privapub.thepra.dev (push) Successful in 1m4s
- A remote `summary` is a content warning only on a Note or Question, or when `sensitive` is set. On Articles,
  Events, Videos, Pages and Audio it is an excerpt (WordPress teasers, Mobilizon dates and places, Mbin titles) and
  is now kept as `Post.Excerpt` instead of hiding the post. `Post.ObjectType` records the remote type, and the
  Mastodon API shows a remote non-Note object's title above its body again.
- Persona usernames must match `^[a-z0-9_]+$`, as groups already did; a name outside it was unreachable from
  Mastodon and Misskey.
- `postingRestrictedToMods` is defined in our JSON-LD context (Iceshrimp.NET drops undefined terms).
- `Vary: Accept` on actor and object URLs.
- Owner decision: blocks federate. A block sends `Block`, an unblock `Undo{Block}`; checked live against GoToSocial.
- Owner decision: a persona's and a group's `published`, and the day in new ids, is a random day up to two weeks
  before creation, so personas made the same day no longer share a date. Migration _007 gives existing ones theirs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB
2026-10-01 17:39:58 +02:00
thepraandClaude Opus 5.5 a5d9a89445 Communities follow FEP-1b12, circles federate to their members only
Communities:
- a post addressed to a community (to, cc or audience) is accepted
  according to its posting policy - followers, anyone, or moderators -
  and GroupDistributor announces the whole activity with `audience` to
  the community's followers, plus the object for new posts so Mastodon
  shows them; updates and deletes of community content are announced too;
- top-level posts are Pages with a name (the title, or a headline from
  the text); /flock counts members, /wardens lists moderators;
- a Mastodon client posts into a community by mentioning it, or into a
  remote group, which sets `audience`;
- an Announce of an activity from a remote group a persona follows (Lemmy)
  is followed through: the object is fetched from its own origin, kept with
  its AudienceURI, and fanned out to the group's local followers; updates
  are applied in place and deletes checked against the origin.

Circles stop being local-only: an undiscoverable Group actor whose follows
are all requests the owner approves; posts addressed to the circle and its
/flock and delivered to members' own inboxes, never announced, never
public; a remote member's post into the circle is accepted from members
only. SignedFetchAuthorizer serves circle posts and collections only to a
signed request from a member or a member server's instance actor - 404 for
anyone else. Circles never surface in search, lookups, mentions, account
ids or profile pages.

Federation:SecureMode requires a valid signature on every GET under
/peasants except the instance actor. Group forms take a posting policy.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB
2026-10-01 12:30:57 +02:00
thepraandClaude Opus 5.5 8f75317050 Blocks, mutes, bookmarks, pins and reports
- Blocks are per persona and never federate (a Block activity would tell
  the other server who blocked whom): the blocked account is removed as a
  follower, with a Reject{Follow} if it is remote, unfollowed, cleared from
  home and notifications, and refused with Reject if it follows again.
- Mutes (optionally timed, optionally sparing notifications) and per-
  persona domain blocks keep authors out of home timelines, notifications
  and every status list the API returns; the boosts of a hidden author's
  posts are hidden too.
- Bookmarks and pins (at most five, public or unlisted, own posts) with
  their Mastodon endpoints and flags; pinned posts are the actor's
  `featured` collection at /trophies, and `featuredTags` points at
  /tattoos.
- Reports: /api/v1/reports stores the report and, when forwarding to a
  remote account, sends Flag from the instance actor, so the reporting
  persona is never named to the other server. An inbound Flag about a local
  persona or its posts becomes a report; moderators list and resolve them
  under /clientapi/moderator/reports.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB
2026-10-01 12:13:57 +02:00
thepraandClaude Opus 5.5 b54cdf78b2 One StatusService behind both client APIs, with outbound likes and boosts
Domain/Statuses/StatusService takes a persona, not a root, and is what
/clientapi and the Mastodon API share:
- Publish renders Markdown (clientapi) or plain text (Mastodon clients),
  checks the persona may see the post it replies to, opens or reuses the
  conversation for a direct post from its recipients and mentions, fans
  out and hands the Create to the outbox;
- Edit keeps a revision and sends Update{Note}; Remove soft-deletes and
  returns the post, so a client can delete and redraft;
- Favourite and Reblog work on anything the persona can see and send Like,
  Announce and their Undo to the author (and, for boosts, to followers);
  boosts are refused for anything but public and unlisted posts.

PostsService is now the clientapi wrapper that checks the root owns the
persona. A persona's own boost is a local row: the outbox renders it as
Announce, /grunts/announce-{id} resolves, and object endpoints, profile
pages and NodeInfo skip it. ContentFormat gains Plain.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB
2026-10-01 11:55:22 +02:00
thepraandClaude Opus 5.5 a76cc34557 Posts reach their audience, edits follow them, deletes leave a tombstone
OutboxPublisher works out who a post goes to, Mastodon's way: followers'
shared inboxes for public, unlisted and followers-only; every mentioned
remote account's own inbox; the recipients only for direct; the parent's
author for a reply; a community's followers for a post in it; nobody for
a circle. Creates, updates and deletes all use it.

- Local posts take a visibility (public, unlisted, followers-only) and a
  spoiler text next to the content-warning flag.
- /clientapi/post/update keeps the previous version as a revision,
  re-renders, and sends Update{Note} with `updated` to the same to/cc.
- Deleting is now soft: the content, title, spoiler, media and revisions
  are cleared, timeline entries removed, the parent's reply count goes
  down, Delete goes to the stored audience, and the object answers 410
  with a Tombstone instead of 404.
- Editing a persona's profile sends Update{Person} to its followers.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB
2026-10-01 11:34:35 +02:00
thepraandClaude Opus 5.5 1c78da34a9 One visibility rule for every public read
VisibilityPolicy.IsPublic is the single expression for "anyone may see
this" (Public or Unlisted, not deleted); the outbox, the object and
activity endpoints, the HTML pages and NodeInfo all use it instead of
spelling it out. CanSee answers for a persona: the author, a mentioned
local persona, a conversation member for Direct, a circle member for
Circle; followers-only waits for P1.2's follows.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB
2026-10-01 11:26:51 +02:00
thepraandClaude Opus 5.5 9ec1f9930b Profile and post pages, NodeInfo 2.1, and FEDERATION.md
/@user and /@user/{id} are Razor pages showing an avatar's or community's
public and unlisted posts: no scripts, a strict CSP, no-referrer, noindex,
and an alternate link to the ActivityPub document. A client asking them
for activity+json is redirected to the actor or note, and the actor and
note redirect browsers here.

NodeInfo answers 2.1 as well as 2.0 (repository, homepage, a link to
FEDERATION.md) and counts only public local posts.

FEDERATION.md (FEP-67ff) lists the protocols, FEPs, route names,
activities and the security rules a peer will notice.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB
2026-10-01 11:23:03 +02:00
thepraandClaude Opus 5.5 e54e17708f Outbound documents speak Mastodon's dialect under PrivaPub's route names
Local text (Domain/Content/ContentRenderer): Markdig with raw HTML off and
autolinks, or plain text for API clients; then @user and @user@host are
resolved (locally or through WebFinger) into Mastodon's h-card markup and
#tags into hashtag links, skipping code, links and e-mail addresses. A
post stores the result, its mentions and its tags, and is delivered to
the mentioned actors and the parent's author as well as to followers.

Renderer:
- Mastodon's @context (toot, schema PropertyValue, discoverable,
  indexable, blurhash, focalPoint, featured, alsoKnownAs, movedTo) and
  FEP-2c59's webfinger;
- an actor's url is its HTML page, its fields are PropertyValue
  attachments, published is cut to the day and indexable is false;
- a note's content is the stored rendering; a title becomes name and a
  bold first line, a content warning without its own text is summarised
  by the title or "Content warning"; Mention and Hashtag tags, contentMap,
  updated, and to/cc by visibility.

Routes take the agreed names: /groupies, /stalking, /scribbles/{id},
/grunts/{id} (a Create resolves), /whispers/{id} for a DM's context. The
outbox is a collection with a first page, paged by max_id. A browser
asking for an actor or a note is redirected to /@user, and WebFinger's
profile-page points there.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB
2026-10-01 11:21:31 +02:00
thepraandClaude Opus 5.5 e9529310fe One Post model for every visibility, read from notes the way peers write them
Post gains what federation and the Mastodon API need: Visibility (Public,
Unlisted, FollowersOnly, Direct, Circle, LocalGeo), the author's account
id, to/cc, the Create's id, url, context, quote, InReplyToURI and the
parent's author, a separate SpoilerText next to the title, language,
mentions, hashtags, remote attachments (alt text, blurhash, focus, size),
reply/favourite/reblog counters, revisions, EditedAt and DeletedAt.

Direct messages are Posts with Visibility Direct and a ConversationId;
migration _005 copies DmPost rows across with their ids and fills the new
fields of existing posts. DmPost is left in place so a rollback still sees
the old messages.

Inbound:
- NoteParser reads Note, Article, Page, Question and media types: content,
  then contentMap, then _misskey_content; summary as the spoiler and name
  as the title; Mention and Hashtag tags; attachments; a PeerTube-style
  list attribution prefers the person over the channel; quote URIs.
- Addressing classifies like Mastodon, finding followers-only by the
  author's own followers URL (now stored on ForeignAvatar), not by a
  "/followers" suffix.
- Create keeps a post when a local persona is addressed or mentioned, when
  it replies to a local post (the parent's reply count goes up) or when a
  community it follows is addressed; an unsolicited public post is not
  stored. Update keeps the previous version as a revision.

The outbox and object endpoints serve only Public and Unlisted posts.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB
2026-10-01 11:17:14 +02:00
thepraandClaude Opus 5.5 796f06acf4 The inbox answers once it has verified, and processes from the job queue
InboxService is split the way the roadmap lays out Federation/Inbox:
- InboxReceiver reads and verifies the request exactly as before, runs
  the checks that need no fetch (the activity id's origin, a Follow of a
  missing or local-only actor, an Undo of someone else's activity, an
  embedded object attributed to someone else), queues a ProcessInbox job
  and answers 202. The job's dedupe key is the activity id, so a peer that
  delivers the same activity twice is processed once.
- InboxProcessor (two at a time, eight attempts) loads the verified actor
  and hands the activity to the handler for its type.
- Handlers/{Follow,Undo,Create,Delete,Update}Handler are the old methods,
  unchanged except that they no longer produce status codes; the JSON
  helpers live in Objects/ActivityJson and the group membership helpers in
  Inbox/ForeignMembers.

A slow fetch of an object or a remote actor now delays the job, not the
sender's HTTP request.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB
2026-10-01 11:12:34 +02:00
thepraandClaude Opus 5.5 d80cd42a0c Rate limits on accounts and inboxes, a database dump before every deploy
- Sign-up, login, the invitation flows and password recovery allow ten
  requests a minute per client address; the inboxes give each sending
  origin (the keyId's) a bucket of 300 that refills at 300 a minute, and
  answer 429 beyond it, which peers retry.
- deploy.yml dumps the PrivaPub database to /var/backups before it stops
  the service (the last seven are kept), and after the swap checks that
  Swagger answers 404 and that the shared inbox answers junk with 400 and
  an unsigned activity with 401.
- ActivityPubClient and PostBoost, never used, are gone.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB
2026-10-01 11:02:21 +02:00
thepraandClaude Opus 5.5 2eb2a63f1e Conversations are keyed by who is in them, and circles stay home
S8: a direct message joins a conversation only when its participants are
exactly that conversation's members, found through a new
DmGroup.ParticipantsKey (a hash of the sorted members). A remote context
no longer decides anything: it let anyone who knew a conversation's
context post into it, and joining by context while dropping a participant
would have shown a reply to someone it was not addressed to. A context is
kept only when it is on the author's origin. Sending a DM to the same
people again reuses their conversation instead of opening a new one.

S9: Group.Kind is Circle or Community. A circle is not a federated actor:
its actor, collections, WebFinger and inbox answer 404, a remote Follow is
refused, and posts in it are IsLocalOnly - never delivered, never in an
outbox, never served. Communities keep today's behaviour until P4.
Migration _003 makes every existing group a circle, marks their posts
local-only and backfills the conversation keys.

End-to-end inbox tests sign real deliveries from a fake peer: a context
injection, a forged activity id, a note attributed to someone else, a
cross-origin object, a bad signature, junk bodies and a Follow of a circle.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB
2026-10-01 10:57:01 +02:00
thepraandClaude Opus 5.5 034b792801 Move the federation code under Federation/, namespaces only
Build / Build (push) Successful in 16s
Services/Federation and Controllers/ServerToServer become
Federation/{Actors,Signing,Inbox,Outbox,Rendering,Controllers}, the first step
of the roadmap's layout. No type, route or behaviour changes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB
2026-10-01 10:39:41 +02:00