From bd18de35a66d437d031bc3ac867ac444b482831b Mon Sep 17 00:00:00 2001 From: thepra Date: Tue, 6 Oct 2026 07:37:03 +0200 Subject: [PATCH] Deliveries to followers carry a digest of them, per server MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Owner decision of 2026-10-06 (FEP-8fcf). A persona's delivery addressed to its followers carries a signed Collection-Synchronization header naming its followers, its roll-call (…/groupies/roll-call) and the digest of its accepted followers on the receiving server only. The roll-call answers a signed request with the persona's followers on the signer's server and nobody else's. Mastodon gives every Undo{Follow} it sends after reading a roll-call the same id (…#follows//undo), so a second one looked like a copy: an Undo of a Follow that comes again while the follow it ends exists again is now kept once per follow. Checked live (scenarios/followsync.sh): Mastodon drops a follow PrivaPub lost, and undoes one it lost itself. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw --- CLAUDE.md | 4 + FEDERATION.md | 10 +- .../FollowersSynchronizationTests.cs | 200 ++++++++++++++++++ .../Actors/FollowersSynchronization.cs | 70 ++++++ .../Controllers/PeasantsController.cs | 14 ++ PrivaPub/Federation/Inbox/InboxReceiver.cs | 17 ++ PrivaPub/Federation/Outbox/DeliveryService.cs | 11 +- PrivaPub/Federation/Signing/HttpSignatures.cs | 9 +- docs/ROADMAP.md | 3 +- tools/pasture/scenarios/followsync.sh | 51 +++++ 10 files changed, 385 insertions(+), 4 deletions(-) create mode 100644 PrivaPub.Tests/Federation/FollowersSynchronizationTests.cs create mode 100644 PrivaPub/Federation/Actors/FollowersSynchronization.cs create mode 100644 tools/pasture/scenarios/followsync.sh diff --git a/CLAUDE.md b/CLAUDE.md index 8e94991..588553b 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -562,6 +562,10 @@ tools/pasture/run.sh down # removes e - **Threads (`scenarios/threads.sh`, needs mastodon):** mastouser follows alice, carol answers alice's public post, and opening the thread on Mastodon finds carol's reply through alice's `replies` (the scenario makes Mastodon's copy look ten minutes old: Mastodon reads a thread's replies only five minutes after it first holds the post). 5 checks. +- **Followers synchronisation (`scenarios/followsync.sh`, needs mastodon):** PrivaPub forgets mastouser's follow of + alice (mongo), alice's followers-only post naming mastouser makes Mastodon read her roll-call and drop it; then + Mastodon forgets a follow (rails) and alice's next followers-only post makes it send the `Undo` PrivaPub applies. + Follows are checked in Mastodon's database: its API caches relationships for a day. 8 checks. - **Pins (`scenarios/pins.sh`, needs mastodon):** a Mastodon account pins and unpins while alice follows it, alice pins and unpins while it follows her, a fresh account's earlier pin shows once PrivaPub resolves it, and following it brings its earlier posts (its outbox). 9 checks. diff --git a/FEDERATION.md b/FEDERATION.md index 2536f82..96dd8ee 100644 --- a/FEDERATION.md +++ b/FEDERATION.md @@ -71,9 +71,11 @@ covered by unit tests written in their documents' shape. - [FEP-c0e0: Emoji reactions](https://codeberg.org/fediverse/fep/src/branch/main/fep/c0e0/fep-c0e0.md) (`EmojiReact`, and Misskey's `Like` with content) - [FEP-5feb: Search indexing consent](https://codeberg.org/fediverse/fep/src/branch/main/fep/5feb/fep-5feb.md) (`indexable`) +- [FEP-8fcf: Followers collection synchronization across servers](https://codeberg.org/fediverse/fep/src/branch/main/fep/8fcf/fep-8fcf.md) + (sent; see "Followers synchronisation") Planned (see `docs/ROADMAP.md`, phases P7 and P8, and the per-platform notes in `docs/INTEROP.md`): FEP-9098 (custom -emoji), FEP-7888 and FEP-f228 (threads), FEP-7628 (Move), FEP-8fcf (followers synchronisation), FEP-8967 (link +emoji), FEP-7888 and FEP-f228 (threads), FEP-7628 (Move), FEP-8967 (link attachments), FEP-521a and FEP-8b32 (keys and integrity proofs), FEP-ae0c (relays). ## Actors @@ -171,6 +173,12 @@ persona's posts passed on to its followers. A deleted post answers 410 with a `T which a reply inherits from its parent, ours or another server's. Both list only the public and unlisted posts PrivaPub holds, the context every one under the root (at most 500); followers-only, circle, direct and local-only posts name neither, and their collections answer 404. Mastodon completes a thread from them. +- **Followers synchronisation** (FEP-8fcf, owner decision 2026-10-06). A persona's delivery addressed to its followers + carries a signed `Collection-Synchronization` header: its `followers`, its roll-call (`…/groupies/roll-call`) and the + digest (XOR of each id's SHA-256) of its accepted followers on the receiving server only. The roll-call answers a + signed request with the persona's followers on the signer's server and nobody else's; unsigned, 401. A server whose + view differs mends itself from it, Mastodon both ways (checked live). Mastodon's `Undo{Follow}` for a follow it never + knew of has one id per account (`…#follows//undo`); when it comes again after a new follow, it ends that follow too. - **Pinned posts** are the actor's `featured` collection (`/trophies`); `featuredTags` is `/tattoos`. A pin or an unpin is told to the post's audience as `Add` or `Remove` on `featured`, as Mastodon tells it. - **Blocks are sent.** A blocked remote account receives `Block` from the blocking account (and `Reject{Follow}` if it diff --git a/PrivaPub.Tests/Federation/FollowersSynchronizationTests.cs b/PrivaPub.Tests/Federation/FollowersSynchronizationTests.cs new file mode 100644 index 0000000..0a2de78 --- /dev/null +++ b/PrivaPub.Tests/Federation/FollowersSynchronizationTests.cs @@ -0,0 +1,200 @@ +using Microsoft.Extensions.Caching.Memory; +using Microsoft.Extensions.Logging.Abstractions; + +using MongoDB.Entities; + +using PrivaPub.Federation.Actors; +using PrivaPub.Federation.Outbox; +using PrivaPub.Federation.Signing; +using PrivaPub.Infrastructure.Http; +using PrivaPub.Infrastructure.Jobs; +using PrivaPub.Models.Federation; +using PrivaPub.Models.Jobs; +using PrivaPub.Tests.Support; +using PrivaPub.Tests.Support.Host; + +using System.Net; +using System.Security.Cryptography; +using System.Text; +using System.Text.Json.Nodes; + +namespace PrivaPub.Tests.Federation +{ + // FEP-8fcf (owner decision 2026-10-06): a delivery addressed to a persona's followers tells the receiving server, in a signed + // header, a digest of the persona's followers there; the roll-call it names lists them, to that server only + [Trait("Category", "Integration")] + [Xunit.Collection(nameof(Exclusive))] + public sealed class FollowersSynchronizationTests : IAsyncLifetime + { + static readonly string[] PeerHosts = { "localhost", "127.0.0.1" }; + + Harness _harness; + + public async ValueTask InitializeAsync() + { + Assert.SkipUnless(MongoFixture.Enabled, MongoFixture.Skip); + _harness = await Harness.Start(); + await DB.Default.DeleteAsync(i => PeerHosts.Contains(i.Host)); + } + + public async ValueTask DisposeAsync() + { + if (_harness != default) + await _harness.DisposeAsync(); + } + + static CancellationToken Token => TestContext.Current.CancellationToken; + + [Fact] + public void The_digest_is_the_xor_of_each_ids_sha256_whatever_the_order() + { + Assert.Equal(new string('0', 64), FollowersSynchronization.Digest([])); + Assert.Equal(FollowersSynchronization.Digest(["https://a.example/users/one", "https://a.example/users/two"]), + FollowersSynchronization.Digest(["https://a.example/users/two", "https://a.example/users/one"])); + Assert.Equal(Convert.ToHexStringLower(SHA256.HashData(Encoding.UTF8.GetBytes("https://a.example/users/one"))), + FollowersSynchronization.Digest(["https://a.example/users/one"])); + } + + async Task Delivered(LocalActor signer, string inbox, JsonObject activity) + { + await _harness.Delivery.Enqueue(signer, new[] { inbox }, activity, Token); + var job = await DB.Default.Find().Match(j => j.DedupeKey == $"{activity["id"]!.GetValue()}|{inbox}").ExecuteSingleAsync(Token); + var handler = new DeliveryJobHandler(_harness.Local, Peer.Http(), new HostCircuitBreaker(new MemoryCache(new MemoryCacheOptions())), + NullLogger.Instance); + Assert.Equal(JobResult.Done, (await handler.Handle(job, Token)).Result); + return Assert.Single(_harness.Peer.Requests, r => r.Body?.Contains(activity["id"]!.GetValue()) == true); + } + + static JsonObject Create(LocalActor author, params string[] to) => new() + { + ["id"] = $"{author.Uri}/grunts/{Guid.NewGuid():N}", ["type"] = "Create", ["actor"] = author.Uri, ["to"] = new JsonArray(to.Select(t => (JsonNode)t).ToArray()), + ["object"] = new JsonObject { ["type"] = "Note", ["content"] = "hi", ["to"] = new JsonArray(to.Select(t => (JsonNode)t).ToArray()) } + }; + + [Fact] + public async Task A_delivery_to_followers_carries_a_signed_digest_of_the_followers_on_that_server_only() + { + var (_, alice) = await _harness.Persona("alice"); + var one = new RemoteActor(_harness.Peer, "one"); + var two = new RemoteActor(_harness.Peer, "two"); + var far = new RemoteActor(_harness.Peer, "far", _harness.Peer.B); + await _harness.FollowedBy(alice, one); + await _harness.FollowedBy(alice, two); + await _harness.FollowedBy(alice, far); + await DB.Default.SaveAsync(new Follower + { + LocalActorId = alice.Id, LocalActorKind = alice.Kind, ActorURI = _harness.Peer.A + "/users/asking", InboxURL = _harness.Peer.A + "/users/asking/inbox", IsAccepted = false + }, Token); + _harness.Peer.Answer("/inbox", 202); + + var received = await Delivered(alice, _harness.Peer.A + "/inbox", Create(alice, alice.Followers)); + + Assert.Equal(alice.Followers, Value(received, "collectionId")); + Assert.Equal(alice.Followers + "/roll-call", Value(received, "url")); + Assert.Equal(FollowersSynchronization.Digest([one.Id, two.Id]), Value(received, "digest")); + var signature = HttpSignatures.Parse(received.Signature); + Assert.Equal(new[] { "(request-target)", "host", "date", "digest", "collection-synchronization" }, signature.Headers); + var signingString = $"(request-target): post /inbox\nhost: {received.Headers["Host"]}\ndate: {received.Headers["Date"]}\ndigest: {received.Headers["Digest"]}\n" + + $"collection-synchronization: {received.Headers[FollowersSynchronization.Header]}"; + using var key = RSA.Create(); + key.ImportFromPem(alice.PublicKeyPem); + Assert.True(key.VerifyData(Encoding.UTF8.GetBytes(signingString), signature.Signature, HashAlgorithmName.SHA256, RSASignaturePadding.Pkcs1)); + } + + static string Value(HttpRequestRecord received, string name) + { + var values = received.Headers[FollowersSynchronization.Header].Split(',', StringSplitOptions.TrimEntries) + .Select(p => p.Split('=', 2)).ToDictionary(p => p[0], p => p[1].Trim('"')); + return values[name]; + } + + [Fact] + public async Task A_delivery_that_is_not_for_followers_carries_none() + { + var (_, alice) = await _harness.Persona("alice"); + var recipient = new RemoteActor(_harness.Peer, "recipient"); + await _harness.FollowedBy(alice, recipient); + _harness.Peer.Answer("/inbox", 202); + + var received = await Delivered(alice, _harness.Peer.A + "/inbox", Create(alice, recipient.Id)); + + Assert.False(received.Headers.ContainsKey(FollowersSynchronization.Header)); + Assert.DoesNotContain("collection-synchronization", HttpSignatures.Parse(received.Signature).Headers); + } + + // Mastodon undoes a follow it never knew of with the same id each time ({actor}#follows//undo): when it comes again, + // after a new follow, it ends that one too; a copy of it, with nothing to end, stays a copy + [Fact] + public async Task An_undo_of_a_follow_sent_again_after_a_new_follow_ends_that_one_too() + { + var (_, alice) = await _harness.Persona("alice"); + var bob = new RemoteActor(_harness.Peer, "bob"); + var undo = new JsonObject + { + ["id"] = bob.Id + "#follows//undo", ["type"] = "Undo", ["actor"] = bob.Id, + ["object"] = new JsonObject { ["id"] = bob.Id + "#follows/", ["type"] = "Follow", ["actor"] = bob.Id, ["object"] = alice.Uri } + }; + Task Follows() => DB.Default.Find().Match(f => f.LocalActorId == alice.Id && f.ActorURI == bob.Id).ExecuteAnyAsync(Token); + + await _harness.FollowedBy(alice, bob); + await _harness.Deliver(bob, "/human-centipede", undo.DeepClone()); + Assert.False(await Follows()); + + await _harness.FollowedBy(alice, bob); + await _harness.Deliver(bob, "/human-centipede", undo.DeepClone()); + Assert.False(await Follows()); + + Assert.Equal("duplicate", (await _harness.Deliver(bob, "/human-centipede", undo.DeepClone())).Reason); + } + } + + [Trait("Category", "Integration")] + public sealed class RollCallTests : IAsyncLifetime + { + PrivaPubHost _host; + HttpClient _client; + Peer _peer; + + public async ValueTask InitializeAsync() + { + Assert.SkipUnless(MongoFixture.Enabled, MongoFixture.Skip); + _host = await PrivaPubHost.Shared(); + _client = _host.Client(); + _peer = await Peer.Start(); + } + + public async ValueTask DisposeAsync() + { + _client?.Dispose(); + if (_peer != default) + await _peer.DisposeAsync(); + } + + static string[] Items(JsonObject collection) => collection["orderedItems"]!.AsArray().Select(i => i!.GetValue()).ToArray(); + + [Fact] + public async Task The_roll_call_lists_the_followers_on_the_signing_server_and_asks_for_a_signature() + { + var token = TestContext.Current.CancellationToken; + var persona = await _host.Persona(await _host.SignUp(), "rollcall"); + var near = new RemoteActor(_peer, "near"); + var server = new RemoteActor(_peer, "instance", type: "Application"); + var far = new RemoteActor(_peer, "far", _peer.B); + foreach (var follower in new[] { near, far }) + await DB.Default.SaveAsync(new Follower + { + LocalActorId = persona.Id, LocalActorKind = LocalActorKind.Person, ActorURI = follower.Id, InboxURL = follower.Id + "/inbox" + }, token); + var path = $"/peasants/{persona.UserName}/groupies/roll-call"; + + var asServer = await _client.Fetch(server.SignedGet(path)); + var asFar = await _client.Fetch(far.SignedGet(path)); + + Assert.Equal(HttpStatusCode.OK, asServer.Status); + Assert.Equal(persona.ActorUri() + "/groupies/roll-call", asServer.Json["id"]!.GetValue()); + Assert.Equal(new[] { near.Id }, Items(asServer.Json)); + Assert.Equal(new[] { far.Id }, Items(asFar.Json)); + Assert.Equal(HttpStatusCode.Unauthorized, (await _client.Fetch(path)).Status); + } + } +} diff --git a/PrivaPub/Federation/Actors/FollowersSynchronization.cs b/PrivaPub/Federation/Actors/FollowersSynchronization.cs new file mode 100644 index 0000000..f10a580 --- /dev/null +++ b/PrivaPub/Federation/Actors/FollowersSynchronization.cs @@ -0,0 +1,70 @@ +using MongoDB.Bson; +using MongoDB.Entities; + +using PrivaPub.Models.Federation; + +using System.Security.Cryptography; +using System.Text; +using System.Text.Json.Nodes; +using System.Text.RegularExpressions; + +namespace PrivaPub.Federation.Actors +{ + // FEP-8fcf (owner decision 2026-10-06): a delivery addressed to a persona's followers carries a digest of its followers on + // the receiving server, and that server reads which of its own accounts they are at the persona's roll-call, signed. A + // server whose view differs (a Follow or an Undo lost on the way) mends it from there. No server learns of followers + // anywhere else. + public static class FollowersSynchronization + { + public const string Header = "Collection-Synchronization"; + + public static bool Synchronizes(LocalActor actor) => actor is { Kind: LocalActorKind.Person, IsCircle: false }; + + public static string RollCall(LocalActor actor) => actor.Followers + "/roll-call"; + + // a server, as Mastodon cuts a URL to tell where its accounts live: scheme and authority + public static string Origin(string uri) => + Uri.TryCreate(uri, UriKind.Absolute, out var parsed) && parsed.Scheme is "https" or "http" ? $"{parsed.Scheme}://{parsed.Authority}" : default; + + // the actor's accepted followers whose ids live under origin + public static async Task> On(LocalActor actor, string origin, CancellationToken token) + { + if (origin == default) + return []; + var under = new BsonRegularExpression("^" + Regex.Escape(origin + "/")); + return await DB.Default.Find() + .Match(f => f.LocalActorId == actor.Id && f.LocalActorKind == actor.Kind && f.IsAccepted) + .Match(f => f.Regex(x => x.ActorURI, under)) + .Project(f => f.ActorURI) + .ExecuteAsync(token); + } + + // the XOR of each id's SHA-256, in lower-case hex: the same set gives the same digest in any order + public static string Digest(IEnumerable ids) + { + var digest = new byte[SHA256.HashSizeInBytes]; + foreach (var id in ids) + { + var hash = SHA256.HashData(Encoding.UTF8.GetBytes(id)); + for (var i = 0; i < digest.Length; i++) + digest[i] ^= hash[i]; + } + return Convert.ToHexStringLower(digest); + } + + public static string HeaderValue(LocalActor actor, string digest) => + $"collectionId=\"{actor.Followers}\", url=\"{RollCall(actor)}\", digest=\"{digest}\""; + + // whether the activity, or the object it carries, is addressed to the actor's followers + public static bool ForFollowers(JsonObject activity, LocalActor actor) => + Addressed(activity, actor.Followers) || activity["object"] is JsonObject inner && Addressed(inner, actor.Followers); + + static bool Addressed(JsonObject node, string followers) => + new[] { "to", "cc", "bto", "bcc" }.Any(field => node[field] switch + { + JsonArray many => many.Any(a => a is JsonValue value && value.TryGetValue(out var text) && text == followers), + JsonValue one => one.TryGetValue(out var text) && text == followers, + _ => false + }); + } +} diff --git a/PrivaPub/Federation/Controllers/PeasantsController.cs b/PrivaPub/Federation/Controllers/PeasantsController.cs index d28b11a..d531366 100644 --- a/PrivaPub/Federation/Controllers/PeasantsController.cs +++ b/PrivaPub/Federation/Controllers/PeasantsController.cs @@ -143,6 +143,20 @@ namespace PrivaPub.Federation.Controllers return Activity(ActivityPubRenderer.OrderedCollection(local.Followers, (int)count, default)); } + // FEP-8fcf: the persona's followers on the server that signs the request, and nobody else's (FollowersSynchronization) + [HttpGet, Route("{actor}/groupies/roll-call")] + public async Task RollCall(string actor, CancellationToken token) + { + var local = await _localActors.FindByUserName(actor, token); + if (local is not { IsFederated: true } || !FollowersSynchronization.Synchronizes(local)) + return NotFound(); + var requester = await _fetches.Requester(Request, token); + if (requester == default) + return StatusCode(StatusCodes.Status401Unauthorized); + var there = await FollowersSynchronization.On(local, FollowersSynchronization.Origin(requester.ActorURI), token); + return Activity(ActivityPubRenderer.OrderedCollection(FollowersSynchronization.RollCall(local), there.Count, there.Select(f => (JsonNode)f))); + } + [HttpGet, Route("{actor}/stalking")] public async Task Following(string actor, CancellationToken token) { diff --git a/PrivaPub/Federation/Inbox/InboxReceiver.cs b/PrivaPub/Federation/Inbox/InboxReceiver.cs index eabed4b..460bfef 100644 --- a/PrivaPub/Federation/Inbox/InboxReceiver.cs +++ b/PrivaPub/Federation/Inbox/InboxReceiver.cs @@ -199,10 +199,27 @@ namespace PrivaPub.Federation.Inbox // true copy carries the same if (!queued && dedupe != default && CarriesOther(await _queue.Payload(dedupe, token), activity)) queued = await _queue.Enqueue(JobKind.ProcessInbox, queuedPayload, host, $"{dedupe}|{Digest(activity)}", token); + // an Undo of a Follow that comes again while the follow it ends exists again is meant for that follow: Mastodon + // gives every Undo it sends after reading a roll-call (FEP-8fcf) one id, `{actor}#follows//undo`. It is kept + // once per follow, so a retry still counts as a copy + if (!queued && dedupe != default && type == "Undo" && await FollowAgain(activity, actorUri, token) is { } follow) + queued = await _queue.Enqueue(JobKind.ProcessInbox, queuedPayload, host, $"{dedupe}|{Digest(activity)}|{follow}", token); _logger.LogInformation("Inbox {Recipient}: {Type} from {Actor} queued", recipient?.Handle ?? "shared", type, actorUri); return new(StatusCodes.Status202Accepted, Reason: !queued ? "duplicate" : forwardedBy != default ? "forwarded" : "queued"); } + // the follow an Undo of a Follow would end, when there is one: its record's id + async Task FollowAgain(JsonNode activity, string actorUri, CancellationToken token) + { + if (activity["object"] is not JsonObject inner || Value(inner, "type") != "Follow" || Id(inner["object"]) is not { } targetUri + || await _localActors.FindByUri(targetUri, token) is not { } target) + return default; + var follower = await MongoDB.Entities.DB.Default.Find() + .Match(f => f.ActorURI == actorUri && f.LocalActorId == target.Id && f.LocalActorKind == target.Kind) + .ExecuteFirstAsync(token); + return follower?.ID; + } + static string HostOf(string uri) => Uri.TryCreate(uri, UriKind.Absolute, out var parsed) ? parsed.Host.ToLowerInvariant() : default; static bool CarriesOther(string earlierPayload, JsonNode activity) diff --git a/PrivaPub/Federation/Outbox/DeliveryService.cs b/PrivaPub/Federation/Outbox/DeliveryService.cs index 8fad02f..c4f6a27 100644 --- a/PrivaPub/Federation/Outbox/DeliveryService.cs +++ b/PrivaPub/Federation/Outbox/DeliveryService.cs @@ -170,6 +170,15 @@ namespace PrivaPub.Federation.Outbox }); } + // FEP-8fcf: what a delivery to the persona's followers tells the receiving server of its followers there + static async Task<(string, string)[]> Synchronization(LocalActor signer, string body, Uri inbox, CancellationToken token) + { + if (!FollowersSynchronization.Synchronizes(signer) || JsonNode.Parse(body) is not JsonObject activity || !FollowersSynchronization.ForFollowers(activity, signer)) + return []; + var there = await FollowersSynchronization.On(signer, FollowersSynchronization.Origin(inbox.AbsoluteUri), token); + return [(FollowersSynchronization.Header, FollowersSynchronization.HeaderValue(signer, FollowersSynchronization.Digest(there)))]; + } + async Task Deliver(Job job, Attempt attempt, CancellationToken token) { var payload = JsonSerializer.Deserialize(job.Payload); @@ -198,7 +207,7 @@ namespace PrivaPub.Federation.Outbox var body = Encoding.UTF8.GetBytes(payload.Body); using var request = new HttpRequestMessage(HttpMethod.Post, inbox) { Content = new ByteArrayContent(body) }; request.Content.Headers.ContentType = MediaTypeHeaderValue.Parse(RemoteActorService.ActivityJson); - HttpSignatures.Sign(request, signer, body); + HttpSignatures.Sign(request, signer, body, await Synchronization(signer, payload.Body, inbox, token)); var started = Stopwatch.GetTimestamp(); try diff --git a/PrivaPub/Federation/Signing/HttpSignatures.cs b/PrivaPub/Federation/Signing/HttpSignatures.cs index 55a3b9a..1442f02 100644 --- a/PrivaPub/Federation/Signing/HttpSignatures.cs +++ b/PrivaPub/Federation/Signing/HttpSignatures.cs @@ -17,7 +17,8 @@ namespace PrivaPub.Federation.Signing public static string Digest(byte[] body) => "SHA-256=" + Convert.ToBase64String(SHA256.HashData(body)); - public static void Sign(HttpRequestMessage request, LocalActor signer, byte[] body) + // extra: further headers the request carries, signed with the rest (FEP-8fcf's Collection-Synchronization) + public static void Sign(HttpRequestMessage request, LocalActor signer, byte[] body, params (string Name, string Value)[] extra) { var date = DateTime.UtcNow.ToString("r", CultureInfo.InvariantCulture); var signed = new List<(string Name, string Value)> @@ -35,6 +36,12 @@ namespace PrivaPub.Federation.Signing signed.Add(("digest", digest)); } + foreach (var (name, value) in extra) + { + request.Headers.TryAddWithoutValidation(name, value); + signed.Add((name.ToLowerInvariant(), value)); + } + var signingString = string.Join("\n", signed.Select(h => $"{h.Name}: {h.Value}")); using var rsa = RSA.Create(); rsa.ImportFromPem(signer.PrivateKeyPem); diff --git a/docs/ROADMAP.md b/docs/ROADMAP.md index 39cbf06..7a9ba50 100644 --- a/docs/ROADMAP.md +++ b/docs/ROADMAP.md @@ -672,7 +672,8 @@ it, raw where it doesn't. - re-run WebFinger on a rename; - inbound `Block`; `Add`/`Remove` of pins: **done 2026-10-05** (both ways, a community's pins too, the `featured` collection read with an account's counts; checked live against Mastodon); - - FEP-8fcf followers sync; + - FEP-8fcf followers sync: sending **done 2026-10-06** (owner decision; the digest of a persona's followers on the + receiving server, and a signed roll-call listing only them; Mastodon mends both ways from it, checked live); - `indexable`/`discoverable`/`searchableBy`; - edit history from `formerRepresentations`; - PeerTube reply rules and `ApproveReply`. diff --git a/tools/pasture/scenarios/followsync.sh b/tools/pasture/scenarios/followsync.sh new file mode 100644 index 0000000..e43091e --- /dev/null +++ b/tools/pasture/scenarios/followsync.sh @@ -0,0 +1,51 @@ +# Followers synchronisation (FEP-8fcf, owner decision 2026-10-06): alice's followers-only post tells Mastodon, in a signed +# header, a digest of her followers there; when Mastodon's view differs it reads her roll-call and mends itself. Both ways: +# a follow PrivaPub lost (Mastodon drops it), and a follow Mastodon lost (it sends the Undo PrivaPub then applies). The +# roll-call lists Mastodon's accounts only, and only to a signed request. Needs the mastodon peer. +M=https://mastodon.test:6443 +mcurl() { curl -sk --resolve mastodon.test:6443:127.0.0.1 "$@"; } +. "$here/peers/mastodon.sh" +m_rails() { podman exec pasture-mastodon bin/rails runner "$1" 2>/dev/null | tail -1; } +p_mongo() { podman exec pasture-mongo mongosh --quiet PrivaPub --eval "$1"; } + +echo "followsync" +AT=$(privapub_token alice_sync) +AH="Authorization: Bearer $AT" +[ -n "$AT" ] && ok "PrivaPub token for alice_sync" || { ko "PrivaPub token for alice_sync"; return 1; } +MT=$(mastodon_token) +MH="Authorization: Bearer $MT" +run=$(date +%s) +alice_acct=$(curl -s -H "$AH" "$P/api/v1/accounts/verify_credentials") +alice_id=$(echo "$alice_acct" | j "print(d['id'])") +alice_uri=$(echo "$alice_acct" | j "print(d['url'])" | sed 's|/@|/peasants/|') +alice_followers() { curl -s -H "$AH" "$P/api/v1/accounts/verify_credentials" | j "print(d['followers_count'])"; } +alice_on_m=$(mcurl -H "$MH" "$M/api/v2/search?q=@alice_sync@privapub.test&resolve=true&type=accounts" | j "print(d['accounts'][0]['id'])") +# (from Mastodon's database: its API caches relationships a day, and the scenario changes them behind its back) +m_follows() { m_rails "puts Follow.exists?(account: Account.find_local(\"mastouser\"), target_account: Account.find_by(uri: \"$alice_uri\")) ? \"True\" : \"False\""; } +mastouser_uri=$(m_rails 'puts ActivityPub::TagManager.instance.uri_for(Account.find_local("mastouser"))') +follow_from_m() { + mcurl -o /dev/null -X POST -H "$MH" "$M/api/v1/accounts/$alice_on_m/follow" + until_true 45 '[ "$(m_follows)" = "True" ] && [ "$(alice_followers)" = "1" ]' +} +quiet_post() { curl -s -o /dev/null -X POST -H "$AH" "$P/api/v1/statuses" -d "status=$1&visibility=private"; } + +[ "$(m_follows)" = "True" ] || follow_from_m +[ "$(m_follows)" = "True" ] && [ "$(alice_followers)" = "1" ] && ok "mastouser follows alice" || ko "mastouser never followed alice" + +echo " the roll-call" +[ "$(pfetch -s -o /dev/null -w '%{http_code}' -H 'Accept: application/activity+json' "$alice_uri/groupies/roll-call")" = "401" ] \ + && ok "an unsigned roll-call is refused" || ko "the roll-call answered an unsigned request" + +echo " a follow PrivaPub lost" +p_mongo "db.Follower.deleteMany({LocalActorId:'$alice_id', ActorURI:'$mastouser_uri'})" >/dev/null +[ "$(alice_followers)" = "0" ] && ok "PrivaPub forgets mastouser" || ko "PrivaPub still counts mastouser" +# (nothing goes to Mastodon for a follower PrivaPub does not know of: alice names mastouser, so her post still goes there) +quiet_post "@mastouser@mastodon.test for my followers $run" +until_true 60 '[ "$(m_follows)" = "False" ]' && ok "Mastodon reads alice's roll-call and drops the follow" || ko "Mastodon still has mastouser following alice" + +echo " a follow Mastodon lost" +follow_from_m && ok "mastouser follows alice again" || ko "mastouser could not follow alice again" +m_rails "a = Account.find_local(\"mastouser\"); t = Account.find_by(uri: \"$alice_uri\"); Follow.where(account: a, target_account: t).destroy_all" >/dev/null +[ "$(m_follows)" = "False" ] && ok "Mastodon forgets the follow" || ko "Mastodon still has the follow" +quiet_post "for my followers again $run" +until_true 60 '[ "$(alice_followers)" = "0" ]' && ok "Mastodon reads the roll-call and undoes the follow PrivaPub had" || ko "PrivaPub still counts mastouser"