Commit Graph
10 Commits
Author SHA1 Message Date
thepraandClaude Opus 5.5 8f25bf056d Everything on, phase 4a: one answer everywhere for counts, search, collections and the instance API
Owner decision 2026-10-04: fix the mismatches and every other mismatch of the same kind.

- One counting rule (Domain/Privacy/Counted), Mastodon's. It is used for a persona's statuses_count, its outbox
  totalItems, NodeInfo localPosts and the instance status_count, which used to count four different things. It
  counts every post that is neither deleted nor a DM, boosts included, and circle and located posts too (owner
  decision). A group's count includes its remote members' posts.
- Users. Personas of banned or deleted roots no longer count, and are not found in search. NodeInfo now gives
  activeMonth and activeHalfyear, and the v2 instance gives active_month instead of a constant 0.
- replies_count counts only public and unlisted replies, so it no longer tells anyone that a private reply exists.
  Migration _012 recounts it.
- A remote account that deletes itself takes everything out of every count (GoneActors): its likes, downvotes,
  reactions and poll votes go and their counters come back, as do its boosts', replies' and quotes' counts, and its
  notifications. Lookups, account lists, search and favourited_by no longer show it. Migration _012 applies this to
  accounts already gone.
- Deleting a post also deletes its pins and the local boosts of it.
- /stalking gives the same total as following_count. Members are still never listed, and hide_collections is now
  always true, since the setting never did anything.
- Joining a community by invitation is following it, so /flock and /groupies agree; leaving unfollows.
- Search. Anyone may search, as on Mastodon; resolve and offset need a sign-in, offset pages, and deleted accounts
  are never found.
- notifications/unread_count counts what the list shows, and the owner's follower and following lists page with
  Link.
- The instance API advertises what is enforced:
  - max_characters, now enforced with a 422;
  - max_pinned_statuses = MaxPins;
  - the media types and limits MediaService and MediaOptions accept;
  - PollService's limits;
  - the configured languages;
  - no streaming URL until streaming exists.

  domain_count counts the servers we have exchanged with; which ones stays unpublished (peers is empty).
- Routes Mastodon answers now answer instead of 404:
  - directory, tags/{name}, timelines/link and identity_proofs;
  - instance/languages, translation_languages, domain_blocks and privacy_policy;
  - the v1 and v2 notification policy, and notification requests.

Also, from phase 3: a recovered password ends /clientapi sessions through a per-root SessionStamp claim instead of
comparing the JWT's whole-second nbf with the change time. That comparison let a token issued in the same second
survive, which made a test flaky.

671 tests pass.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ELjqpznMFMNrJoJUj6K5p2
2026-10-04 03:48:40 +02:00
thepraandClaude Opus 5.5 8c2eba6cbb Everything on, phase 3: sign-in and recovery tell nothing, recovered passwords end sessions, deleted roots are gone everywhere
Owner decision 2026-10-04: fix the account privacy findings.

- Sign-in. Every failure answers "That username and password do not match." after the same work: an unknown login
  is hashed against a decoy, and the comparison is constant-time. "Banned" is told only to someone who gave the right
  password. This covers /clientapi/user/login, /invitation/login and /oauth/login.
- Recovery.
  - Every request answers the same sentence and queues a SendRecovery job, whether or not the account exists or has an
    email. The lookup, the code and SMTP move to RecoveryJob, so neither the answer nor its timing says anything.
  - Codes are kept only as a SHA-256 hash, for one hour. Migration _011 drops the plaintext ones, which never expired.
  - A recovered password ends every session of the root. RootSessions sets CredentialsChangedAt, which JwtEvents
    checks against the JWT's issue time, now stamped as nbf, and revokes each persona's OAuth tokens and authorizations.
- Deleting a root (RootRemoval: the admin route, or the restored self-delete at /clientapi/user/delete, which asks for
  the password).
  - Its sessions end.
  - Each persona and each group it owns sends Delete{Actor} to its followers, its members and the accounts it follows.
  - The personas' posts are emptied.
  - /peasants/{name} answers 410 with a Tombstone (formerType Person or Group), as do its inbox and WebFinger, through
    LocalActorService.Gone. The names stay reserved.
  - The root keeps only a unique `deleted-{id}` name; the second deletion on an instance used to collide on
    "Deleted user".

Also, from phase 2's pasture: GoToSocial files a circle post like a DM and shows it only to accounts it mentions. Each
member's copy, and a member's refetch, now also mentions that member silently. The GoToSocial scenario checks circle
posts in conversations, like DMs, and they pass there now, as on Mastodon.

657 tests pass.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ELjqpznMFMNrJoJUj6K5p2
2026-10-04 03:16:59 +02:00
thepraandClaude Opus 5.5 5f56681c01 Everything on, phase 1: geolocation fetches itself, the deploy signs in as @thepra, the crawler is on, sign-up by invitation
Build / Build (push) Successful in 5m1s
Deploy / privapub.thepra.dev (push) Successful in 5m39s
Owner decisions (2026-10-04, recorded in docs/ROADMAP.md): production runs everything that is built, and nothing waits
on a person running a command.

- Geolocation updates itself. GeoUpdater, a hosted service, checks daily whether each DB-IP Lite database was built this
  month. If not, it fetches this month's, or last month's early in the month. It installs a file only once it opens as
  the right kind of database, then swaps it in atomically, and the locator reloads at once. Lookups now run under the
  lock, so a reload can no longer dispose a reader mid-lookup. The systemd timer, its script and their setup.sh lines
  are gone: the root step they needed never happened, and none is needed now. /stargazing names the database in use.
- The admin CLI runs after the app is built, with every service and nothing started.
  - `create-root <login> [--admin]` takes the password on stdin; it is how the first login is made while sign-up is
    closed.
  - `smoke <persona>` keeps the root `deploy-smoke` and an undiscoverable persona, and gives the root a new password
    on every run.
- The deploy signs in as @thepra. It runs the CLI, gets a token through the real OAuth flow (tools/smoke/oauth.sh,
  moved out of the pasture's privapub_token, which now uses it), checks the signed-in API and that @thepra is
  undiscoverable, then revokes the token. PRIVAPUB_SMOKE_TOKEN is gone.
- The deploy also fails when:
  - NodeInfo and the instance API disagree about registrations;
  - /stargazing does not say the crawler is on;
  - the geolocation databases are missing or more than 40 days old.
- The crawler is on in production, seeded with ten large servers of different kinds. FEDERATION.md now describes it
  and how to opt out.
- One registrations switch (Registrations:Mode, default Invitations; Open in tests and the pasture). It is read by
  open sign-up (403 when closed), NodeInfo `openRegistrations`, and v1 and v2 of the instance API, so they can no longer
  disagree. Before, NodeInfo said open and the instance API said closed. Group invitations always work, so
  invites_enabled is true.
- A persona edit through /clientapi no longer resets what the Mastodon API set (discoverable, locked, quote policy…):
  the theme is merged into the settings instead of replacing them.

650 tests pass. The deploy's smoke step was rehearsed against the pasture's PrivaPub.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ELjqpznMFMNrJoJUj6K5p2
2026-10-04 02:37:38 +02:00
thepraandClaude Opus 5.5 2cfea7b60c T7: the federation surface over HTTP
Tests through the real routes of PeasantsController, WellKnownController and
UsersController, on the whole server under test (34 new tests):

- FederationGetTests: the actor document (activity+json, SPKI key at
  #main-key owned by the actor, sharedInbox, published = PublishedOn's day,
  no creation date, no root); ld+json; browsers sent to /@name; Vary: Accept;
  /users 301; the outbox's totalItems and ?page=true&max_id paging across
  the 20-item boundary, boosts as Announces, and no followers-only, direct,
  located, federated-copy or deleted post; /groupies and /stalking naming
  nobody; /trophies (public pins, newest first) and /tattoos; /scribbles
  (public and unlisted 200, browsers redirected, followers-only, direct and
  located 404, deleted 410 Tombstone); a circle post only for a signed member
  or its instance actor; a circle's /groupies, /flock and /wardens only for
  members; /grunts create- and announce- ids; /parrot-licences 200, revoked
  410, wrong author 404; secure mode's 401 for every unsigned GET but the
  instance actor's.
- WellKnownTests: WebFinger by acct:, @-prefixed, bare, upper-case and actor
  URI; other domains, unknown names, a root's login name and no resource;
  the instance actor, a community, a circle (answered: current behaviour);
  NodeInfo links, 2.0 and 2.1 naming no root, unknown versions 404; usage
  counting only public, unlisted, non-boost local posts (Exclusive).
- InboxRouteTests: all three inboxes accept a signed delivery and refuse
  junk (400), unsigned (401), a bad Digest, a two-hour-old Date, a signature
  for another host and a swapped body (401); an unknown persona's /mouth is
  404; ld+json with the ActivityStreams profile is accepted; a signer whose
  actor answers 503 gets 503 with Retry-After; the 301st unsigned POST from
  one address is 429 while a signed server from it is not.
- PersonaSeparationHttpTests: with a sibling persona and its community on
  the same login, every GET under /api (filled with the persona's ids) plus
  search, lookup and relationships, and a crawl of everything federation
  publishes about the persona (actor, outbox pages, collections, scribbles,
  grunts, WebFinger, NodeInfo, /@ pages), never name the sibling, its
  community or the login.

Fixed:
- A circle's /groupies told anyone how many followers (members) it has,
  while its /flock and /wardens were already for members only; it now
  answers 404 to anyone but a signed member or a member's instance actor.
- WebFinger answered 404 to a bare user@domain or @user@domain resource,
  which Mastodon, GoToSocial and Pleroma all accept; it now treats them as
  acct: (noted in docs/INTEROP.md).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ELjqpznMFMNrJoJUj6K5p2
2026-10-03 11:53:23 +02:00
thepraandClaude Opus 5.5 b54cdf78b2 One StatusService behind both client APIs, with outbound likes and boosts
Domain/Statuses/StatusService takes a persona, not a root, and is what
/clientapi and the Mastodon API share:
- Publish renders Markdown (clientapi) or plain text (Mastodon clients),
  checks the persona may see the post it replies to, opens or reuses the
  conversation for a direct post from its recipients and mentions, fans
  out and hands the Create to the outbox;
- Edit keeps a revision and sends Update{Note}; Remove soft-deletes and
  returns the post, so a client can delete and redraft;
- Favourite and Reblog work on anything the persona can see and send Like,
  Announce and their Undo to the author (and, for boosts, to followers);
  boosts are refused for anything but public and unlisted posts.

PostsService is now the clientapi wrapper that checks the root owns the
persona. A persona's own boost is a local row: the outbox renders it as
Announce, /grunts/announce-{id} resolves, and object endpoints, profile
pages and NodeInfo skip it. ContentFormat gains Plain.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB
2026-10-01 11:55:22 +02:00
thepraandClaude Opus 5.5 1c78da34a9 One visibility rule for every public read
VisibilityPolicy.IsPublic is the single expression for "anyone may see
this" (Public or Unlisted, not deleted); the outbox, the object and
activity endpoints, the HTML pages and NodeInfo all use it instead of
spelling it out. CanSee answers for a persona: the author, a mentioned
local persona, a conversation member for Direct, a circle member for
Circle; followers-only waits for P1.2's follows.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB
2026-10-01 11:26:51 +02:00
thepraandClaude Opus 5.5 9ec1f9930b Profile and post pages, NodeInfo 2.1, and FEDERATION.md
/@user and /@user/{id} are Razor pages showing an avatar's or community's
public and unlisted posts: no scripts, a strict CSP, no-referrer, noindex,
and an alternate link to the ActivityPub document. A client asking them
for activity+json is redirected to the actor or note, and the actor and
note redirect browsers here.

NodeInfo answers 2.1 as well as 2.0 (repository, homepage, a link to
FEDERATION.md) and counts only public local posts.

FEDERATION.md (FEP-67ff) lists the protocols, FEPs, route names,
activities and the security rules a peer will notice.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB
2026-10-01 11:23:03 +02:00
thepraandClaude Opus 5.5 e54e17708f Outbound documents speak Mastodon's dialect under PrivaPub's route names
Local text (Domain/Content/ContentRenderer): Markdig with raw HTML off and
autolinks, or plain text for API clients; then @user and @user@host are
resolved (locally or through WebFinger) into Mastodon's h-card markup and
#tags into hashtag links, skipping code, links and e-mail addresses. A
post stores the result, its mentions and its tags, and is delivered to
the mentioned actors and the parent's author as well as to followers.

Renderer:
- Mastodon's @context (toot, schema PropertyValue, discoverable,
  indexable, blurhash, focalPoint, featured, alsoKnownAs, movedTo) and
  FEP-2c59's webfinger;
- an actor's url is its HTML page, its fields are PropertyValue
  attachments, published is cut to the day and indexable is false;
- a note's content is the stored rendering; a title becomes name and a
  bold first line, a content warning without its own text is summarised
  by the title or "Content warning"; Mention and Hashtag tags, contentMap,
  updated, and to/cc by visibility.

Routes take the agreed names: /groupies, /stalking, /scribbles/{id},
/grunts/{id} (a Create resolves), /whispers/{id} for a DM's context. The
outbox is a collection with a first page, paged by max_id. A browser
asking for an actor or a note is redirected to /@user, and WebFinger's
profile-page points there.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB
2026-10-01 11:21:31 +02:00
thepraandClaude Opus 5.5 2eb2a63f1e Conversations are keyed by who is in them, and circles stay home
S8: a direct message joins a conversation only when its participants are
exactly that conversation's members, found through a new
DmGroup.ParticipantsKey (a hash of the sorted members). A remote context
no longer decides anything: it let anyone who knew a conversation's
context post into it, and joining by context while dropping a participant
would have shown a reply to someone it was not addressed to. A context is
kept only when it is on the author's origin. Sending a DM to the same
people again reuses their conversation instead of opening a new one.

S9: Group.Kind is Circle or Community. A circle is not a federated actor:
its actor, collections, WebFinger and inbox answer 404, a remote Follow is
refused, and posts in it are IsLocalOnly - never delivered, never in an
outbox, never served. Communities keep today's behaviour until P4.
Migration _003 makes every existing group a circle, marks their posts
local-only and backfills the conversation keys.

End-to-end inbox tests sign real deliveries from a fake peer: a context
injection, a forged activity id, a note attributed to someone else, a
cross-origin object, a bad signature, junk bodies and a Follow of a circle.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB
2026-10-01 10:57:01 +02:00
thepraandClaude Opus 5.5 034b792801 Move the federation code under Federation/, namespaces only
Build / Build (push) Successful in 16s
Services/Federation and Controllers/ServerToServer become
Federation/{Actors,Signing,Inbox,Outbox,Rendering,Controllers}, the first step
of the roadmap's layout. No type, route or behaviour changes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB
2026-10-01 10:39:41 +02:00