diff --git a/CLAUDE.md b/CLAUDE.md index 57738be..b8b0f8c 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -182,8 +182,17 @@ group www-data and reaches the private mongod; `sudo -u www-data` works too. `ContentFormat` says what `Text` holds. Remote names are plain text. 8. **Circles federate to members only.** A circle is an undiscoverable Group actor that takes follow requests (the owner approves); its posts are addressed to the circle and its `/flock`, delivered to members' personal inboxes, never - announced, and served only to a signed request from a member or a member's instance actor - (`SignedFetchAuthorizer`), 404 otherwise. Circles never appear in search, lookups, mentions or profile pages. + announced. Each member's copy also names that member in `cc` (`OutboxPublisher.Naming`, owner decision + 2026-10-04), because Mastodon and GoToSocial keep a post only when it names one of their accounts; Create, every + Update and the Delete all go through `OutboxPublisher.Publish` for that. A reply to a circle post stays in the circle, + and a circle post never asks a non-member for a quote. Circles never appear in search, lookups, mentions or profile + pages. + **A post that is not public is served only to a signed request from someone it was for** (`SignedFetchAuthorizer.MayRead`): + a follower or an addressed account for followers-only, an addressed account for a DM, a member for a circle, or the + instance actor of a server where one of them lives; 404 to anyone else, 410 to them once it is deleted. A circle + refetch names the requesting member, or the members on the requesting server. A DM's `context` + (`/peasants/{name}/whispers/{id}`) lists the conversation's posts for its participants. Mastodon deletes its copy + when a refetch answers 404, which is why this matters. **Communities** are FEP-1b12 groups: `GroupDistributor` announces the whole activity (plus the object for new posts, for Mastodon), top-level posts are `Page`s with a `name`, posting follows `Group.PostingPolicy`. Located posts (`LocalGeo`) are the only local-only posts. @@ -419,7 +428,8 @@ tools/pasture/run.sh down # removes e Its token comes from `rails runner` (no password grant). Without Elasticsearch its status search finds nothing, so deliveries are checked through `/api/v1/accounts/:id/statuses` of the sender as Mastodon knows them, or `Status.exists?` through `rails runner`. Its actors are numbered (`/ap/users/`), so look URIs up rather than - build them. 49 checks; circle posts are an expected failure (see `docs/INTEROP.md`, Mastodon). + build them. Circle posts and a followers-only post survive its signed refetch (`ActivityPub::FetchRemoteStatusService` + through `rails runner`). Inbound Block is the one expected failure until P7. - **Misskey (2026.10.0):** one container on the shared Postgres and Redis. A new Misskey federates with nobody (`federation: none`) until `admin/update-meta` says `all`, which `misskey_up` does. Its API is `POST /api/` with the token as `i` (`mk` in the scenario); `users/relation` answers a list, `users/notes` leaves replies out unless diff --git a/FEDERATION.md b/FEDERATION.md index 4d08e65..026ca3c 100644 --- a/FEDERATION.md +++ b/FEDERATION.md @@ -73,8 +73,10 @@ A group is either a **community** or a **circle**. top-level post is a `Page` with a `name`. Members are counted at `/flock`; moderators are listed at `/wardens`, which the actor's `attributedTo` points to, with `postingRestrictedToMods` as Lemmy expects. A mention of a community posts into it. - A **circle** is private. Its actor is not discoverable and every follow is a request. Its posts are addressed to the - circle and its members collection, delivered to each member's own inbox and never announced. They are served only - to a signed request from a member, or from the instance actor of a member's server; anyone else gets 404. A + circle and its members collection, delivered to each member's own inbox and never announced. Each member's copy also + names that member in `cc`, so servers that keep only posts naming one of their accounts (Mastodon, GoToSocial) keep + it; it names no other member. The posts are served only to a signed request from a member, or from the instance actor + of a member's server, and that copy names the member (or the members on that server); anyone else gets 404. A Mastodon member's replies reach only the people they mention. - Announces from **remote** groups (Lemmy communities) are followed through to the activity: the object is fetched from its own origin, never taken from the announce. @@ -192,6 +194,13 @@ Posts with a location (shown to nearby users of this server) never leave the ser deletes, must be on its actor's origin. An embedded object from another origin is fetched from that origin. - **Fetching.** All fetches are signed by the instance actor. They go only to public addresses, follow at most three redirects and read at most 1 MB. +- **Reading our documents (SecureMode).** privapub.thepra.dev answers ActivityPub GETs only when they are signed, like + Mastodon's authorized fetch; the instance actor `/peasants/privapub` is the exception, since its key is needed first. + A browser asking for HTML is redirected to the public page instead. +- **Posts that are not public** (followers-only, direct, circle) are served to a signed request from someone they were + for, or from the instance actor of a server where someone they were for lives, and to nobody else (404). Once deleted + they answer those same readers 410. A direct message's `context`, `/peasants/{name}/whispers/{id}`, is an + `OrderedCollection` of the conversation's posts for its participants. - **HTML.** Received HTML is sanitised to Mastodon's allowlist. - **Keys we cannot fetch for now.** When a sender's key cannot be fetched because its server timed out or answered 5xx, the inbox answers 503 with `Retry-After: 300` rather than 401. diff --git a/PrivaPub.Tests/Federation/GroupTests.cs b/PrivaPub.Tests/Federation/GroupTests.cs index c7283fc..36592c8 100644 --- a/PrivaPub.Tests/Federation/GroupTests.cs +++ b/PrivaPub.Tests/Federation/GroupTests.cs @@ -9,6 +9,7 @@ using PrivaPub.Models.Federation; using PrivaPub.Models.Group; using PrivaPub.Models.Post; using PrivaPub.Models.Social; +using PrivaPub.StaticServices; using PrivaPub.Tests.Support; using System.Text.Json.Nodes; @@ -138,21 +139,83 @@ namespace PrivaPub.Tests.Federation Assert.Equal(PostVisibility.Circle, outcome.Post.Visibility); var create = Assert.Single(await _harness.Outgoing(member.Id + "/inbox")); Assert.Equal(new[] { circle.Uri, circle.Flock }, create["object"]!["to"]!.AsArray().Select(t => t!.GetValue())); + // the member's copy names the member, so Mastodon and GoToSocial keep it, and names nobody else + Assert.Equal(new[] { member.Id }, create["cc"]!.AsArray().Select(t => t!.GetValue())); + Assert.Equal(new[] { member.Id }, create["object"]!["cc"]!.AsArray().Select(t => t!.GetValue())); Assert.DoesNotContain(Addressing.Public, create.ToJsonString()); Assert.Empty(await _harness.Outgoing(follower.SharedInbox)); Assert.Empty((await _harness.Outgoing(member.SharedInbox)).Where(a => a["type"]!.GetValue() == "Announce")); - var authorizer = new SignedFetchAuthorizer(_harness.Remote); + var authorizer = new SignedFetchAuthorizer(_harness.Remote, new DbEntities()); var path = new Uri(outcome.Post.ObjectURI).AbsolutePath; var asMember = await authorizer.Requester(member.Get(Harness.Host, path), token); var asOutsider = await authorizer.Requester(outsider.Get(Harness.Host, path), token); Assert.True(SignedFetchAuthorizer.MayReadCircle(circleEntity, asMember)); Assert.False(SignedFetchAuthorizer.MayReadCircle(circleEntity, asOutsider)); Assert.False(SignedFetchAuthorizer.MayReadCircle(circleEntity, default)); + Assert.True(await authorizer.MayRead(outcome.Post, asMember, token)); + Assert.False(await authorizer.MayRead(outcome.Post, asOutsider, token)); + Assert.Equal(new[] { member.Id }, SignedFetchAuthorizer.CircleReaders(circleEntity, asMember)); Assert.True(circle.IsCircle); Assert.False(circle.Discoverable); } + [Fact] + public async Task A_circle_posts_edit_and_delete_reach_each_member_naming_only_that_member() + { + var token = TestContext.Current.CancellationToken; + var (_, alice) = await _harness.Persona("alice"); + var first = new RemoteActor(_harness.Peer, "first"); + var second = new RemoteActor(_harness.Peer, "second", _harness.Peer.B); + var (_, circle) = await Group(GroupKind.Circle, PostingPolicy.Followers, alice.Id, first.Id, second.Id); + await _harness.Remote.GetActor(first.Id, refresh: false, token); + await _harness.Remote.GetActor(second.Id, refresh: false, token); + + var outcome = await _harness.Statuses.Publish(alice, new StatusDraft { Text = "just us", GroupId = circle.Id }, token); + await _harness.Statuses.Edit(alice, outcome.Post.ID, new StatusDraft { Text = "just us, edited" }, token); + await _harness.Statuses.Remove(alice, outcome.Post.ID, token); + + foreach (var (member, other) in new[] { (first, second), (second, first) }) + { + var sent = await _harness.Outgoing(member.Id + "/inbox"); + Assert.Equal(new[] { "Create", "Update", "Delete" }, sent.Select(a => a["type"]!.GetValue())); + Assert.All(sent, a => Assert.Contains(member.Id, a["cc"]!.AsArray().Select(c => c!.GetValue()))); + Assert.All(sent, a => Assert.DoesNotContain(other.Id, a.ToJsonString())); + } + } + + [Fact] + public async Task A_reply_to_a_circle_post_stays_in_it_and_a_circle_post_asks_nobody_outside_for_a_quote() + { + var token = TestContext.Current.CancellationToken; + var (_, alice) = await _harness.Persona("alice"); + var (_, bob) = await _harness.Persona("bob"); + var (entity, circle) = await Group(GroupKind.Circle, PostingPolicy.Followers, alice.Id); + entity.Members.Add(new GroupMember { AvatarId = bob.Id }); + await DB.Default.SaveAsync(entity, token); + var outsider = new RemoteActor(_harness.Peer, "asked"); + await _harness.Remote.GetActor(outsider.Id, refresh: false, token); + var asksFirst = new Post + { + ObjectURI = $"{Origin(outsider)}/notes/{Guid.NewGuid():N}", + ActorURI = outsider.Id, + IsFederatedCopy = true, + Visibility = PostVisibility.Public, + ContentHtml = "

ask me first

", + QuotePolicy = new InteractionRule { Manual = new() { Addressing.Public } } + }; + await DB.Default.SaveAsync(asksFirst, token); + + var root = await _harness.Statuses.Publish(alice, new StatusDraft { Text = "just us", GroupId = circle.Id }, token); + var reply = await _harness.Statuses.Publish(bob, new StatusDraft { Text = "still just us", InReplyTo = root.Post.ID }, token); + var quote = await _harness.Statuses.Publish(alice, new StatusDraft { Text = "look", GroupId = circle.Id, QuotedStatusId = asksFirst.ID }, token); + + Assert.Equal(PostVisibility.Circle, reply.Post.Visibility); + Assert.Equal(circle.Id, reply.Post.GroupId); + Assert.Equal(422, quote.Status); + Assert.Empty(await _harness.Outgoing(outsider.Id + "/inbox")); + } + [Fact] public async Task Only_circle_members_can_post_into_a_circle() { diff --git a/PrivaPub.Tests/Http/FederationGetTests.cs b/PrivaPub.Tests/Http/FederationGetTests.cs index 2879f7e..f0c6ad9 100644 --- a/PrivaPub.Tests/Http/FederationGetTests.cs +++ b/PrivaPub.Tests/Http/FederationGetTests.cs @@ -322,11 +322,66 @@ namespace PrivaPub.Tests.Http Assert.Equal(new[] { circle.Uri, circle.Uri + "/flock" }, Strings(asMember.Json["to"])); Assert.DoesNotContain(ActivityPubRenderer.Public, asMember.Text); Assert.Equal(HttpStatusCode.OK, asMemberServer.Status); + // a refetch names the member, as the member's copy did; the instance actor sees the members on its server only + Assert.Equal(new[] { member.Id }, Strings(asMember.Json["cc"])); + Assert.Equal(new[] { member.Id }, Strings(asMemberServer.Json["cc"])); Assert.Equal(HttpStatusCode.NotFound, (await _client.Fetch(path)).Status); Assert.Equal(HttpStatusCode.NotFound, (await _client.Fetch(path, Browser)).Status); foreach (var outsider in new[] { neighbour, stranger, strangerServer }) Assert.Equal(HttpStatusCode.NotFound, (await _client.Fetch(outsider.SignedGet(path))).Status); - Assert.Equal(HttpStatusCode.NotFound, (await _client.Fetch($"/peasants/{owner.UserName}/grunts/create-{post.ID}")).Status); + var create = $"/peasants/{owner.UserName}/grunts/create-{post.ID}"; + Assert.Equal(HttpStatusCode.NotFound, (await _client.Fetch(create)).Status); + var createAsMember = await _client.Fetch(member.SignedGet(create)); + Assert.Equal(HttpStatusCode.OK, createAsMember.Status); + Assert.Equal(post.ObjectURI, createAsMember.Json["object"]!["id"]!.GetValue()); + Assert.Equal(new[] { member.Id }, Strings(createAsMember.Json["object"]!["cc"])); + } + + [Fact] + public async Task Followers_only_and_direct_posts_are_served_to_signed_fetches_from_those_they_were_for() + { + var token = TestContext.Current.CancellationToken; + var owner = await _host.Persona(await _host.SignUp(), "private"); + var follower = new RemoteActor(_peer, "follower"); + var followerServer = new RemoteActor(_peer, "instance", type: "Application"); + var recipient = new RemoteActor(_peer, "recipient"); + var stranger = new RemoteActor(_peer, "stranger", _peer.B); + _peer.WebFinger(recipient); + await DB.Default.SaveAsync(new Follower + { + LocalActorId = owner.Id, + LocalActorKind = LocalActorKind.Person, + ActorURI = follower.Id, + InboxURL = follower.Id + "/inbox" + }, token); + var quiet = await _host.Publish(owner, "for followers", PostVisibility.FollowersOnly); + var direct = await _host.Publish(owner, new StatusDraft { Text = $"@{recipient.Handle()} just you", PlainText = true, Visibility = PostVisibility.Direct }); + var quietPath = $"/peasants/{owner.UserName}/scribbles/{quiet.ID}"; + var directPath = $"/peasants/{owner.UserName}/scribbles/{direct.ID}"; + + Assert.Equal(HttpStatusCode.OK, (await _client.Fetch(follower.SignedGet(quietPath))).Status); + Assert.Equal(HttpStatusCode.OK, (await _client.Fetch(followerServer.SignedGet(quietPath))).Status); + Assert.Equal(HttpStatusCode.NotFound, (await _client.Fetch(stranger.SignedGet(quietPath))).Status); + Assert.Equal(HttpStatusCode.NotFound, (await _client.Fetch(quietPath)).Status); + var asRecipient = await _client.Fetch(recipient.SignedGet(directPath)); + Assert.Equal(HttpStatusCode.OK, asRecipient.Status); + Assert.Contains(recipient.Id, Strings(asRecipient.Json["to"])); + Assert.Equal(HttpStatusCode.NotFound, (await _client.Fetch(follower.SignedGet(directPath))).Status); + Assert.Equal(HttpStatusCode.NotFound, (await _client.Fetch(directPath)).Status); + + // the DM's context is the conversation, for its participants only + var context = PathOf(asRecipient.Json["context"]!.GetValue()); + var conversation = await _client.Fetch(recipient.SignedGet(context)); + Assert.Equal(HttpStatusCode.OK, conversation.Status); + Assert.Equal(new[] { direct.ObjectURI }, Strings(conversation.Json["orderedItems"])); + Assert.Equal(HttpStatusCode.NotFound, (await _client.Fetch(stranger.SignedGet(context))).Status); + Assert.Equal(HttpStatusCode.NotFound, (await _client.Fetch(context)).Status); + + // once deleted, those it was for learn it is gone; everyone else still learns nothing + await _host.Remove(owner, quiet); + Assert.Equal(HttpStatusCode.Gone, (await _client.Fetch(follower.SignedGet(quietPath))).Status); + Assert.Equal(HttpStatusCode.NotFound, (await _client.Fetch(stranger.SignedGet(quietPath))).Status); + Assert.Equal(HttpStatusCode.NotFound, (await _client.Fetch(quietPath)).Status); } [Fact] @@ -463,6 +518,10 @@ namespace PrivaPub.Tests.Http Assert.Equal("Application", instance.Json["type"]!.GetValue()); foreach (var path in paths[..^1]) Assert.True((await client.Fetch(reader.SignedGet(path))).Status == HttpStatusCode.OK, $"{path} refused a signed GET"); + // a browser is not asked for a signature: it is only sent to the public pages + var browser = await client.Fetch($"/peasants/{persona.UserName}", Browser); + Assert.Equal(HttpStatusCode.Redirect, browser.Status); + Assert.Equal(HttpStatusCode.Redirect, (await client.Fetch($"/peasants/{persona.UserName}/scribbles/{post.ID}", Browser)).Status); } } } diff --git a/PrivaPub/Api/Mastodon/Controllers/StatusesController.cs b/PrivaPub/Api/Mastodon/Controllers/StatusesController.cs index 4daa43b..017c91d 100644 --- a/PrivaPub/Api/Mastodon/Controllers/StatusesController.cs +++ b/PrivaPub/Api/Mastodon/Controllers/StatusesController.cs @@ -273,7 +273,7 @@ namespace PrivaPub.Api.Mastodon.Controllers await DB.Default.Update().MatchID(post.ID).Modify(p => p.LocalQuotePolicy, policy).ExecuteAsync(token); post.LocalQuotePolicy = policy; if (!post.IsLocalOnly) - await _outbox.Publish(Me, post, ActivityPubRenderer.UpdateOf(post, Me, $"policy-{DateTime.UtcNow.Ticks}"), token); + await _outbox.PublishUpdate(Me, post, $"policy-{DateTime.UtcNow.Ticks}", token); return Json(await _mapper.Status(post, MyId, token)); } diff --git a/PrivaPub/Domain/Statuses/PollService.cs b/PrivaPub/Domain/Statuses/PollService.cs index 0994478..fbb0fa1 100644 --- a/PrivaPub/Domain/Statuses/PollService.cs +++ b/PrivaPub/Domain/Statuses/PollService.cs @@ -233,7 +233,7 @@ namespace PrivaPub.Domain.Statuses return JobOutcome.Done; await Closing(post, author, token); if (!post.IsLocalOnly) - await _outbox.Publish(author, post, ActivityPubRenderer.UpdateOf(post, author, $"poll-{DateTime.UtcNow.Ticks}"), token); + await _outbox.PublishUpdate(author, post, $"poll-{DateTime.UtcNow.Ticks}", token); return JobOutcome.Done; } diff --git a/PrivaPub/Domain/Statuses/QuoteService.cs b/PrivaPub/Domain/Statuses/QuoteService.cs index 62a33c9..9e2a61e 100644 --- a/PrivaPub/Domain/Statuses/QuoteService.cs +++ b/PrivaPub/Domain/Statuses/QuoteService.cs @@ -239,7 +239,7 @@ namespace PrivaPub.Domain.Statuses .ExecuteAsync(token); await DB.Default.Update().MatchID(quoted.ID).Modify(b => b.Inc(p => p.QuotesCount, 1)).ExecuteAsync(token); if (!post.IsLocalOnly) - await _outbox.Publish(author, post, ActivityPubRenderer.UpdateOf(post, author, "quote-approved"), token); + await _outbox.PublishUpdate(author, post, "quote-approved", token); return true; } diff --git a/PrivaPub/Domain/Statuses/StatusService.cs b/PrivaPub/Domain/Statuses/StatusService.cs index 3f27c21..cc1e6e1 100644 --- a/PrivaPub/Domain/Statuses/StatusService.cs +++ b/PrivaPub/Domain/Statuses/StatusService.cs @@ -119,6 +119,11 @@ namespace PrivaPub.Domain.Statuses return StatusOutcome.Fail(StatusCodes.Status404NotFound, "Record not found"); if (parent != default && !await VisibilityPolicy.CanSee(parent, author.Id, token)) return StatusOutcome.Fail(StatusCodes.Status404NotFound, "Record not found"); + // a reply in a circle stays in the circle, whichever client wrote it: Mastodon clients know nothing of groups + if (group == default && parent is { Visibility: PostVisibility.Circle } && !string.IsNullOrEmpty(parent.GroupId) + && await _dbEntities.Groups.MatchID(parent.GroupId).ExecuteFirstAsync(token) is { DeletionAt: null } parentCircle + && await MayPost(parentCircle, author, token)) + group = _localActors.FromGroup(parentCircle); PostEntity quoted = default; var quotePermission = QuotePermission.Denied; @@ -161,6 +166,9 @@ namespace PrivaPub.Domain.Statuses var visibility = located ? PostVisibility.LocalGeo : group is { IsCircle: true } ? PostVisibility.Circle : draft.Visibility is PostVisibility.Circle or PostVisibility.LocalGeo ? PostVisibility.Public : draft.Visibility; + // asking a quoted post's author for permission would show them the circle post + if (visibility == PostVisibility.Circle && quoted != default && quotePermission != QuotePermission.Granted) + return StatusOutcome.Fail(StatusCodes.Status422UnprocessableEntity, "Validation failed: A circle post can only quote a post that needs no permission"); var post = new PostEntity { GroupUserId = author.Id, @@ -340,7 +348,7 @@ namespace PrivaPub.Domain.Statuses { var delete = ActivityPubRenderer.Delete(author, post.ObjectURI, $"delete-{post.ID}", new JsonArray(post.To.Select(t => (JsonNode)t).ToArray()), new JsonArray(post.Cc.Select(c => (JsonNode)c).ToArray())); - await _delivery.Enqueue(author, audience, delete, token); + await _outbox.Publish(author, post, delete, token);//the post in hand still has its group and mentions var group = string.IsNullOrEmpty(post.GroupId) ? default : await _localActors.FindById(LocalActorKind.Group, post.GroupId, token); if (group is { IsCircle: false }) await _groups.Announce(group, delete, post.ObjectURI, isNewPost: false, token); diff --git a/PrivaPub/Federation/Controllers/PeasantsController.cs b/PrivaPub/Federation/Controllers/PeasantsController.cs index 5cce633..972d447 100644 --- a/PrivaPub/Federation/Controllers/PeasantsController.cs +++ b/PrivaPub/Federation/Controllers/PeasantsController.cs @@ -17,6 +17,7 @@ using PrivaPub.Domain.Privacy; using PrivaPub.Federation.Actors; using PrivaPub.Federation.Rendering; using PrivaPub.Federation.Inbox; +using PrivaPub.Federation.Outbox; using PrivaPub.Federation.Signing; using PrivaPub.Infrastructure.Http; using PrivaPub.Models.Group; @@ -185,11 +186,13 @@ namespace PrivaPub.Federation.Controllers public async Task Post(string actor, string postId, CancellationToken token) { var (local, post) = await PublicPost(actor, postId, token); - if (post == default && await CirclePost(actor, postId, token) is { } circlePost) + if (post == default && await SignedPost(actor, postId, token) is { } signed) { - var circleNote = ActivityPubRenderer.Note(circlePost.Post, circlePost.Author, circlePost.Circle, circlePost.Post.InReplyToURI); - circleNote["@context"] = ActivityPubRenderer.Context(); - return Activity(circleNote); + if (signed.Post.DeletedAt.HasValue) + return TombstoneOf(signed.Author, signed.Post); + var signedNote = (JsonObject)signed.Note.DeepClone(); + signedNote["@context"] = ActivityPubRenderer.Context(); + return Activity(signedNote); } if (post == default) return await Tombstone(actor, postId, token) ?? NotFound(); @@ -243,7 +246,36 @@ namespace PrivaPub.Federation.Controllers if (!activityId.StartsWith("create-", StringComparison.Ordinal)) return NotFound(); var (local, post) = await PublicPost(actor, activityId["create-".Length..], token); - return post == default ? NotFound() : Activity(await CreateFor(post, local, token)); + if (post != default) + return Activity(await CreateFor(post, local, token)); + return await SignedPost(actor, activityId["create-".Length..], token) is { Post.DeletedAt: null } signed + ? Activity(ActivityPubRenderer.Create(signed.Author, (JsonObject)signed.Note.DeepClone(), activityId)) + : NotFound(); + } + + // A DM's `context`: the conversation's posts, for its participants (or their servers' instance actors) only. + [HttpGet, Route("{actor}/whispers/{conversationId}")] + public async Task Whispers(string actor, string conversationId, CancellationToken token) + { + var local = await _localActors.FindByUserName(actor, token); + if (local is not { IsFederated: true, Kind: LocalActorKind.Person }) + return NotFound(); + var uri = local.ConversationUri(conversationId); + var conversation = await _dbEntities.DmGroups.Match(g => g.ID == conversationId && g.ConversationURI == uri && !g.DeletionAt.HasValue).ExecuteFirstAsync(token); + if (conversation == default || !SignedFetchAuthorizer.MayReadConversation(conversation, await _fetches.Requester(Request, token))) + return NotFound(); + var posts = await _dbEntities.Posts + .Match(p => p.ConversationId == conversationId && p.Visibility == PostVisibility.Direct && !p.DeletedAt.HasValue) + .Sort(p => p.CreationDate, Order.Ascending) + .ExecuteAsync(token); + return Activity(new JsonObject + { + ["@context"] = ActivityPubRenderer.ActivityStreams, + ["id"] = uri, + ["type"] = "OrderedCollection", + ["totalItems"] = posts.Count, + ["orderedItems"] = new JsonArray(posts.Select(p => (JsonNode)p.ObjectURI).ToArray()) + }); } [HttpPost, Route("{actor}/mouth"), EnableRateLimiting(RateLimiting.Inbox)] @@ -275,18 +307,35 @@ namespace PrivaPub.Federation.Controllers return (local, post); } - async Task<(LocalActor Author, LocalActor Circle, PostEntity Post)?> CirclePost(string actor, string postId, CancellationToken token) + // A followers-only, direct or circle post (deleted ones included), for a signed request from someone it was for + // (SignedFetchAuthorizer.MayRead). It is rendered as it was delivered: a circle post also names, in cc, the requesting + // member, or the members on the requesting instance actor's server. + async Task<(LocalActor Author, PostEntity Post, JsonObject Note)?> SignedPost(string actor, string postId, CancellationToken token) { var local = await _localActors.FindByUserName(actor, token); - if (local is not { Kind: LocalActorKind.Person }) + if (local is not { IsFederated: true, Kind: LocalActorKind.Person }) return default; var post = await _dbEntities.Posts - .Match(p => p.ID == postId && p.GroupUserId == local.Id && !p.IsFederatedCopy && !p.DeletedAt.HasValue && p.Visibility == PostVisibility.Circle) + .Match(p => p.ID == postId && p.GroupUserId == local.Id && !p.IsFederatedCopy && p.ReblogOfPostId == null + && (p.Visibility == PostVisibility.FollowersOnly || p.Visibility == PostVisibility.Direct || p.Visibility == PostVisibility.Circle)) .ExecuteFirstAsync(token); - var circle = post == default ? default : await _dbEntities.Groups.MatchID(post.GroupId).ExecuteFirstAsync(token); - if (circle == default || !SignedFetchAuthorizer.MayReadCircle(circle, await _fetches.Requester(Request, token))) + if (post == default) return default; - return (local, _localActors.FromGroup(circle), post); + var requester = await _fetches.Requester(Request, token); + if (!await _fetches.MayRead(post, requester, token)) + return default; + if (post.Visibility == PostVisibility.Circle) + { + var circle = await _dbEntities.Groups.MatchID(post.GroupId).ExecuteFirstAsync(token); + var note = ActivityPubRenderer.Note(post, local, _localActors.FromGroup(circle), post.InReplyToURI); + return (local, post, OutboxPublisher.Naming(note, SignedFetchAuthorizer.CircleReaders(circle, requester))); + } + var rendered = post.Visibility == PostVisibility.Direct + ? ActivityPubRenderer.DirectNote(post, local, Array.Empty<(string, string)>(), post.ContextURI) + : ActivityPubRenderer.Note(post, local, default, post.InReplyToURI); + rendered["to"] = new JsonArray(post.To.Select(t => (JsonNode)t).ToArray()); + rendered["cc"] = new JsonArray(post.Cc.Select(c => (JsonNode)c).ToArray()); + return (local, post, rendered); } async Task Tombstone(string actor, string postId, CancellationToken token) @@ -298,8 +347,11 @@ namespace PrivaPub.Federation.Controllers .Match(p => p.ID == postId && p.GroupUserId == local.Id && !p.IsFederatedCopy && p.DeletedAt.HasValue && (p.Visibility == PostVisibility.Public || p.Visibility == PostVisibility.Unlisted)) .ExecuteFirstAsync(token); - if (deleted == default) - return default; + return deleted == default ? default : TombstoneOf(local, deleted); + } + + ContentResult TombstoneOf(LocalActor local, PostEntity deleted) + { var tombstone = new JsonObject { ["@context"] = ActivityPubRenderer.ActivityStreams, @@ -349,7 +401,9 @@ namespace PrivaPub.Federation.Controllers { if (HttpMethods.IsGet(Request.Method)) Response.Headers.Vary = "Accept"; - if (_federation.CurrentValue.SecureMode && HttpMethods.IsGet(Request.Method) + // SecureMode asks every reader of ActivityPub documents for a signature, except for the instance actor, whose key + // peers need first, and except for browsers, which only get redirected to the public pages + if (_federation.CurrentValue.SecureMode && HttpMethods.IsGet(Request.Method) && !WantsHtml() && !string.Equals(context.RouteData.Values["actor"] as string, LocalActorService.InstanceUserName, StringComparison.OrdinalIgnoreCase) && await _fetches.Requester(Request, HttpContext.RequestAborted) == default) { diff --git a/PrivaPub/Federation/Outbox/OutboxPublisher.cs b/PrivaPub/Federation/Outbox/OutboxPublisher.cs index 2354882..dfbaa44 100644 --- a/PrivaPub/Federation/Outbox/OutboxPublisher.cs +++ b/PrivaPub/Federation/Outbox/OutboxPublisher.cs @@ -14,6 +14,7 @@ namespace PrivaPub.Federation.Outbox { Task> Audience(LocalActor author, PostEntity post, CancellationToken token); Task Publish(LocalActor author, PostEntity post, JsonObject activity, CancellationToken token); + Task PublishUpdate(LocalActor author, PostEntity post, string reason, CancellationToken token); Task PublishProfile(LocalActor actor, CancellationToken token); } @@ -73,25 +74,63 @@ namespace PrivaPub.Federation.Outbox return inboxes.Where(i => !string.IsNullOrEmpty(i)).Distinct(StringComparer.Ordinal).ToList(); } - async Task> CircleMembers(string groupId, CancellationToken token) + async Task> CircleMembers(string groupId, CancellationToken token) => + (await CircleRecipients(groupId, token)).Select(r => r.Inbox).Distinct(StringComparer.Ordinal).ToList(); + + async Task> CircleRecipients(string groupId, CancellationToken token) { var circle = string.IsNullOrEmpty(groupId) ? default : await _dbEntities.Groups.MatchID(groupId).ExecuteFirstAsync(token); if (circle == default) - return Array.Empty(); + return Array.Empty<(string, string)>(); var remote = circle.Members.Where(m => m.IsForeign).Select(m => m.AvatarId).ToList(); if (remote.Count == 0) - return Array.Empty(); + return Array.Empty<(string, string)>(); return (await _dbEntities.ForeignAvatars.Match(a => remote.Contains(a.ActorURI)).ExecuteAsync(token)) - .Select(a => a.InboxURL).Where(i => !string.IsNullOrEmpty(i)).Distinct(StringComparer.Ordinal).ToList(); + .Where(a => !string.IsNullOrEmpty(a.InboxURL)) + .Select(a => (a.ActorURI, a.InboxURL)) + .ToList(); } public async Task Publish(LocalActor author, PostEntity post, JsonObject activity, CancellationToken token) { + if (post.Visibility == PostVisibility.Circle) + { + foreach (var (member, inbox) in await CircleRecipients(post.GroupId, token)) + await _delivery.Enqueue(author, new[] { inbox }, Naming(activity, new[] { member }), token); + return; + } var inboxes = await Audience(author, post, token); if (inboxes.Count > 0) await _delivery.Enqueue(author, inboxes, activity, token); } + public async Task PublishUpdate(LocalActor author, PostEntity post, string reason, CancellationToken token) + { + var group = string.IsNullOrEmpty(post.GroupId) ? default : await _localActors.FindById(LocalActorKind.Group, post.GroupId, token); + await Publish(author, post, ActivityPubRenderer.UpdateOf(post, author, group, reason), token); + } + + // Mastodon and GoToSocial keep a post only when it names one of their own accounts, and a circle post names only the + // circle. So each member's copy, or a member's refetch, also names that member in cc (owner decision 2026-10-04): + // it tells each member nothing but that they are in the circle. + public static JsonObject Naming(JsonObject activityOrObject, IEnumerable members) + { + var copy = (JsonObject)activityOrObject.DeepClone(); + Name(copy, members); + if (copy["object"] is JsonObject inner && inner.ContainsKey("to")) + Name(inner, members); + return copy; + } + + static void Name(JsonObject node, IEnumerable members) + { + var cc = node["cc"] as JsonArray ?? new JsonArray(); + foreach (var member in members) + if (!cc.Any(c => c?.GetValue() == member)) + cc.Add(member); + node["cc"] = cc; + } + public async Task PublishProfile(LocalActor actor, CancellationToken token) { var document = ActivityPubRenderer.Actor(actor); diff --git a/PrivaPub/Federation/Rendering/ActivityPubRenderer.cs b/PrivaPub/Federation/Rendering/ActivityPubRenderer.cs index 49c81c1..b3d25ed 100644 --- a/PrivaPub/Federation/Rendering/ActivityPubRenderer.cs +++ b/PrivaPub/Federation/Rendering/ActivityPubRenderer.cs @@ -178,11 +178,11 @@ namespace PrivaPub.Federation.Rendering return note; } - public static JsonObject UpdateOf(PostEntity post, LocalActor author, string reason) + public static JsonObject UpdateOf(PostEntity post, LocalActor author, LocalActor group, string reason) { var note = post.Visibility == PostVisibility.Direct ? DirectNote(post, author, Array.Empty<(string, string)>(), post.ContextURI) - : Note(post, author, default, post.InReplyToURI); + : Note(post, author, group, post.InReplyToURI); note["to"] = new JsonArray(post.To.Select(t => (JsonNode)t).ToArray()); note["cc"] = new JsonArray(post.Cc.Select(c => (JsonNode)c).ToArray()); return new JsonObject diff --git a/PrivaPub/Federation/Signing/SignedFetchAuthorizer.cs b/PrivaPub/Federation/Signing/SignedFetchAuthorizer.cs index 80a4612..8984c45 100644 --- a/PrivaPub/Federation/Signing/SignedFetchAuthorizer.cs +++ b/PrivaPub/Federation/Signing/SignedFetchAuthorizer.cs @@ -1,23 +1,34 @@ using PrivaPub.Federation.Actors; using PrivaPub.Federation.Objects; +using PrivaPub.Models.Federation; +using PrivaPub.Models.Post; using PrivaPub.Models.User; +using PrivaPub.StaticServices; using GroupEntity = PrivaPub.Models.Group.Group; +using PostEntity = PrivaPub.Models.Post.Post; namespace PrivaPub.Federation.Signing { public interface ISignedFetchAuthorizer { Task Requester(HttpRequest request, CancellationToken token); + Task MayRead(PostEntity post, ForeignAvatar requester, CancellationToken token); } + // Who may refetch a post that is not public (owner decision 2026-10-04). A server that received a followers-only post, + // a DM or a circle post refetches it, signed by the account it was for or by its instance actor, and Mastodon deletes its + // copy when the refetch answers 404. So the post is served to a signed request from someone it was for, or from the + // instance actor of a server where someone it was for lives; everyone else still gets 404. public class SignedFetchAuthorizer : ISignedFetchAuthorizer { readonly IRemoteActorService _remoteActors; + readonly DbEntities _dbEntities; - public SignedFetchAuthorizer(IRemoteActorService remoteActors) + public SignedFetchAuthorizer(IRemoteActorService remoteActors, DbEntities dbEntities) { _remoteActors = remoteActors; + _dbEntities = dbEntities; } public async Task Requester(HttpRequest request, CancellationToken token) @@ -33,8 +44,48 @@ namespace PrivaPub.Federation.Signing return actor != default && HttpSignatures.Verify(actor.PublicKey, signingString, parameters.Signature) ? actor : default; } + public async Task MayRead(PostEntity post, ForeignAvatar requester, CancellationToken token) + { + if (post == default || requester == default) + return false; + switch (post.Visibility) + { + case PostVisibility.Public or PostVisibility.Unlisted: + return true; + case PostVisibility.Circle: + var circle = string.IsNullOrEmpty(post.GroupId) ? default : await _dbEntities.Groups.MatchID(post.GroupId).ExecuteFirstAsync(token); + return circle != default && MayReadCircle(circle, requester); + case PostVisibility.Direct: + return Addressed(post).Any(uri => Is(requester, uri)); + case PostVisibility.FollowersOnly: + if (Addressed(post).Any(uri => Is(requester, uri))) + return true; + var followers = await _dbEntities.Followers + .Match(f => f.LocalActorId == post.GroupUserId && f.LocalActorKind == LocalActorKind.Person && f.IsAccepted) + .ExecuteAsync(token); + return followers.Any(f => Is(requester, f.ActorURI)); + default: + return false; + } + } + + static IEnumerable Addressed(PostEntity post) => + post.To.Concat(post.Cc).Concat(post.Mentions.Select(m => m.ActorURI)).Where(uri => !string.IsNullOrEmpty(uri)); + + // the account itself, or the instance actor of the server it lives on + static bool Is(ForeignAvatar requester, string actorUri) => + actorUri == requester.ActorURI || requester.AvatarType == AvatarType.Application && Origin.Same(actorUri, requester.ActorURI); + public static bool MayReadCircle(GroupEntity circle, ForeignAvatar requester) => - requester != default && circle.Members.Any(m => m.IsForeign - && (m.AvatarId == requester.ActorURI || requester.AvatarType == AvatarType.Application && Origin.Same(m.AvatarId, requester.ActorURI))); + requester != default && circle.Members.Any(m => m.IsForeign && Is(requester, m.AvatarId)); + + public static bool MayReadConversation(Models.Group.DmGroup conversation, ForeignAvatar requester) => + requester != default && conversation.Members.Any(m => m.IsForeign && Is(requester, m.AvatarId)); + + // the members a refetch names in cc: the requesting member, or the members on an instance actor's server + public static IReadOnlyList CircleReaders(GroupEntity circle, ForeignAvatar requester) => + requester == default + ? Array.Empty() + : circle.Members.Where(m => m.IsForeign && Is(requester, m.AvatarId)).Select(m => m.AvatarId).ToList(); } } diff --git a/tools/pasture/scenarios/gts.sh b/tools/pasture/scenarios/gts.sh index e2a4459..98111f5 100644 --- a/tools/pasture/scenarios/gts.sh +++ b/tools/pasture/scenarios/gts.sh @@ -165,11 +165,9 @@ until_true 20 '[ "$(gcurl -s -H "$GH" "$G/api/v1/accounts/relationships?id[]=$ci curl -s -o /dev/null -X POST $P/clientapi/post/insert -H 'Content-Type: application/json' -H "Authorization: Bearer $jwt" \ -d "{\"avatarId\":\"$alice_id\",\"text\":\"only the circle on GoToSocial sees this\",\"groupId\":\"$circle\"}" circle_uri=$(podman exec pasture-mongo mongosh --quiet PrivaPub --eval 'print(db.Post.findOne({Text:/only the circle on GoToSocial/}).ObjectURI)') -if until_true 15 '[ "$(on_gts "$circle_uri" | j "print(len(d[\"statuses\"]))")" = "1" ]'; then - ok "a circle post reaches its GoToSocial member" -else - xf "a circle post reaches its GoToSocial member (GoToSocial keeps no post addressed only to a collection it does not know)" -fi +# GoToSocial keeps no post addressed only to a collection it does not know, so each member's copy names that member +until_true 15 '[ "$(on_gts "$circle_uri" | j "print(len(d[\"statuses\"]))")" = "1" ]' \ + && ok "a circle post, naming its member, reaches its GoToSocial member" || ko "circle post missing on GoToSocial" [ "$(pfetch -o /dev/null -w '%{http_code}' -H 'Accept: application/activity+json' "$circle_uri")" = "404" ] && ok "the circle post is not served unsigned" || ko "circle post served unsigned" echo "locked personas" diff --git a/tools/pasture/scenarios/mastodon.sh b/tools/pasture/scenarios/mastodon.sh index f38e64e..479440b 100644 --- a/tools/pasture/scenarios/mastodon.sh +++ b/tools/pasture/scenarios/mastodon.sh @@ -40,6 +40,9 @@ until_true 30 '[ "$(m_status_by_uri "$alice_on_m" "$cw_uri" | j "print(d and d[\ fo_uri=$(curl -s -X POST -H "$AH" $P/api/v1/statuses -d 'status=only for followers&visibility=private' | j "print(d['uri'])") until_true 30 '[ "$(m_status_by_uri "$alice_on_m" "$fo_uri" | j "print(d and d[\"visibility\"])")" = "private" ]' && ok "a followers-only post reaches Mastodon as private" || ko "followers-only post missing or widened on Mastodon" [ "$(pfetch -o /dev/null -w '%{http_code}' -H 'Accept: application/activity+json' "$fo_uri")" = "404" ] && ok "the followers-only post is not served unsigned" || ko "followers-only post served unsigned" +# Mastodon deletes its copy when a refetch answers 404, so a signed refetch from a follower's server is answered +fo_refetch=$(podman exec pasture-mastodon bin/rails runner "s = ActivityPub::FetchRemoteStatusService.new.call('$fo_uri'); puts(s.present? ? s.visibility : 'lost')" 2>/dev/null | tail -1) +[ "$fo_refetch" = "private" ] && ok "Mastodon's signed refetch of the followers-only post keeps it private" || ko "Mastodon's refetch of the followers-only post: $fo_refetch" echo " replies" mcurl -o /dev/null -X POST -H "$MH" "$M/api/v1/statuses" -d "status=@alice_masto@privapub.test replying from Mastodon&in_reply_to_id=$a_post_on_m&visibility=public" @@ -144,14 +147,13 @@ circle_post=$(curl -s -X POST $P/clientapi/post/insert -H 'Content-Type: applica -d "{\"avatarId\":\"$alice_id\",\"text\":\"only the circle sees this\",\"groupId\":\"$circle\"}") circle_uri=$(podman exec pasture-mongo mongosh --quiet PrivaPub --eval 'print(db.Post.findOne({Text:/only the circle sees this/},{ObjectURI:1}).ObjectURI)') # Mastodon 4.7 learns whose personal inbox a delivery reached only from /users//inbox, never from the numeric -# /ap/users//inbox it now advertises, and keeps a post naming no local account only for that recipient -# (InboxesController#account_required?, Create#addresses_local_accounts?). A circle post names only the circle. +# /ap/users//inbox it now advertises, and keeps a post naming no local account of its own only for that recipient +# (InboxesController#account_required?, Create#addresses_local_accounts?). So each member's copy names that member. m_stored() { podman exec pasture-mastodon bin/rails runner "puts Status.exists?(uri: '$1')" 2>/dev/null | tail -1; } -if until_true 15 '[ "$(m_stored "$circle_uri")" = "true" ]'; then - ok "a circle post reaches its Mastodon member" -else - xf "a circle post reaches its Mastodon member (Mastodon 4.7 loses the recipient of numeric-inbox deliveries; owner decision pending)" -fi +until_true 15 '[ "$(m_stored "$circle_uri")" = "true" ]' && ok "a circle post, naming its member, reaches its Mastodon member" || ko "circle post missing on Mastodon" +# a signed refetch, as Mastodon does it, is answered for a member's server and names the member, so the copy stays +refetched=$(podman exec pasture-mastodon bin/rails runner "s = ActivityPub::FetchRemoteStatusService.new.call('$circle_uri'); puts(s.present? ? 'kept' : 'lost')" 2>/dev/null | tail -1) +[ "$refetched" = "kept" ] && ok "Mastodon's signed refetch of the circle post keeps it" || ko "Mastodon's refetch of the circle post failed ($refetched)" mastodon_user outsider OT=$(mastodon_token outsider) alice_on_m_o=$(mcurl -H "Authorization: Bearer $OT" "$M/api/v2/search?q=@alice_masto@privapub.test&resolve=true&type=accounts" | j "print(d['accounts'][0]['id'])")