From 9ab87b277980183fcc08cde90b0941c131a7e84a Mon Sep 17 00:00:00 2001 From: thepra Date: Tue, 6 Oct 2026 09:01:21 +0200 Subject: [PATCH] Personas prove what goes to relays; the server says what it reads FEP-521a and FEP-8b32. Every persona has an Ed25519 key of its own (Avatar.SigningKey; migration 014 gives the earlier ones theirs), named in its actor's assertionMethod as a Multikey, the terms defined in the actor's own context. A persona's activity going to a relay carries an eddsa-jcs-2022 proof (JSON canonicalised by RFC 8785, Jcs), so what Activity-Relay forwards reaches Mastodon, which verifies it with its own code. Nothing else carries one: Mitra takes a proof over the HTTP signature and refuses one by a key it has not read, without reading the actor again. Received: an actor's own Multikeys are kept, and a forwarded activity whose proof one of them verifies is taken as it came instead of being read again from its origin. Discovery: WebFinger for the server's origin links its instance actor (FEP-d556), NodeInfo links it as the application actor (FEP-2677), and actors name RFC 9421 under implements (FEP-844e). Checked live: relay 16 (Activity-Relay's forward of alice's post reaches Mastodon), Mitra, GoToSocial and Mastodon unchanged (165 in all). Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw --- CLAUDE.md | 13 +- FEDERATION.md | 33 +++- .../Federation/IntegrityProofTests.cs | 175 ++++++++++++++++++ PrivaPub.Tests/Http/WellKnownTests.cs | 25 +++ PrivaPub.Tests/Support/RemoteActor.cs | 21 ++- PrivaPub/Federation/Actors/ActorDocument.cs | 28 +++ .../Federation/Actors/LocalActorService.cs | 3 + .../Federation/Actors/RemoteActorService.cs | 1 + .../Controllers/WellKnownController.cs | 21 ++- PrivaPub/Federation/Inbox/Forwarded.cs | 14 +- PrivaPub/Federation/Inbox/InboxProcessor.cs | 4 +- PrivaPub/Federation/Outbox/DeliveryService.cs | 21 ++- .../Rendering/ActivityPubRenderer.cs | 36 +++- .../Federation/Signing/IntegrityProofs.cs | 127 +++++++++++++ PrivaPub/Federation/Signing/Jcs.cs | 138 ++++++++++++++ .../_014_personas_have_ed25519_keys.cs | 18 ++ PrivaPub/Models/User/Avatar.cs | 8 + PrivaPub/PrivaPub.csproj | 1 + .../Private/IPrivateAvatarUsersService.cs | 1 + docs/INTEROP.md | 19 +- docs/ROADMAP.md | 14 +- tools/pasture/scenarios/relay.sh | 11 +- 22 files changed, 702 insertions(+), 30 deletions(-) create mode 100644 PrivaPub.Tests/Federation/IntegrityProofTests.cs create mode 100644 PrivaPub/Federation/Signing/IntegrityProofs.cs create mode 100644 PrivaPub/Federation/Signing/Jcs.cs create mode 100644 PrivaPub/Infrastructure/Data/Migrations/_014_personas_have_ed25519_keys.cs diff --git a/CLAUDE.md b/CLAUDE.md index 4843c35..ab3b015 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -84,7 +84,8 @@ PrivaPub/ ASP.NET Core Web API, net10.0 Objects/ Origin, ActivityJson, NoteParser, Addressing, ContentSanitizer Moderation/ DomainBlocks (suspend / silence / reject media) Signing/ HttpSignatures (draft-cavage sign/verify), MessageSignatures (RFC 9421 verify), - RequestSignature (whichever a request carries) + RequestSignature (whichever a request carries), IntegrityProofs (FEP-8b32 + eddsa-jcs-2022 by a persona's Ed25519 key, FEP-521a Multikeys), Jcs (RFC 8785) Inbox/ InboxReceiver (verify, queue, 202) → InboxProcessor (job) → Handlers/{Follow,Accept,Reject, Undo,Create,Update,Delete,Like,Announce}; RemotePosts (build, fetch parents, FetchAncestors); RemoteReplies (FetchReplies: a thread's `context`, else `replies` two levels down); @@ -165,7 +166,10 @@ group www-data and reaches the private mongod; `sudo -u www-data` works too. (its own `geo` client, a fixed HTTPS host, size-capped, the file checked before it is swapped in). 2. **Every fetch is signed by the instance actor** (`privapub`), never by a persona; deliveries are signed by the acting avatar or group. Both are draft-cavage rsa-sha256 over `(request-target) host date` (+ `digest` on bodies). Inbound, - an RFC 9421 signature (`Signature-Input`) is verified too, its target against our public address. + an RFC 9421 signature (`Signature-Input`) is verified too, its target against our public address. A persona's own + activity going to a relay also carries a FEP-8b32 proof by its Ed25519 key (`Avatar.SigningKey`, one per persona, + never shared), and nothing else does: Mitra refuses a proof by a key it has not read. A forwarded activity is taken + as it came only when its proof verifies with a Multikey of its actor's. 3. **A remote document is believed only from its own address.** `RemoteActorService.FetchObject` requires the document's `id` to be the URL it was served from (a same-origin alias is followed once). A key is accepted only if the actor lists it, its `owner` is the actor and it shares the actor's origin. @@ -600,8 +604,9 @@ tools/pasture/run.sh down # removes e `peers/aoderelay.sh` aode-relay 0.3.129 as `aoderelay.test`):** `appsettings.Pasture.json` names both in `Federation:Relays`, so PrivaPub subscribes a minute after it starts. `scenarios/relay.sh` has Mastodon subscribe to each in turn, checks that a post of an account nobody here follows reaches the federated timeline (forwarded by one, - announced by the other), that alice's public post goes to the relays (and through aode-relay to Mastodon) and nothing - less public, and has Mastodon leave again, so the town sees no relayed posts. 15 checks. + announced by the other), that alice's public post goes to the relays (and through both to Mastodon: Activity-Relay's + forward on its FEP-8b32 proof, after the scenario makes Mastodon read alice's keys anew) and nothing less public, and + has Mastodon leave again, so the town sees no relayed posts. 16 checks. - **Smithereen (1.0.3):** its image on the shared MySQL (database `smithereen`, its schema from the image's commit), with imgproxy and a file server behind Caddy as `smithereen.test` (`/i` and `/s`), trusting the CA through a JDK store with it added (`JAVA_TOOL_OPTIONS`). MySQL takes its stored functions only with diff --git a/FEDERATION.md b/FEDERATION.md index ffad14a..61e467a 100644 --- a/FEDERATION.md +++ b/FEDERATION.md @@ -11,6 +11,10 @@ PrivaPub is an ActivityPub server written in C#. This document follows - [HTTP Message Signatures](https://www.rfc-editor.org/rfc/rfc9421) (RFC 9421) and Content-Digest (RFC 9530), verified on deliveries and signed fetches: `rsa-v1_5-sha256` and `rsa-pss-sha512` with RSA keys - [NodeInfo](https://nodeinfo.diaspora.software/) 2.0 and 2.1 +- Object integrity proofs ([FEP-8b32](https://codeberg.org/fediverse/fep/src/branch/main/fep/8b32/fep-8b32.md), + `eddsa-jcs-2022`, JSON canonicalised by [RFC 8785](https://www.rfc-editor.org/rfc/rfc8785)) by Ed25519 keys published + as Multikeys ([FEP-521a](https://codeberg.org/fediverse/fep/src/branch/main/fep/521a/fep-521a.md)), on what personas + send to relays and verified on what is forwarded ## Tested against @@ -64,6 +68,12 @@ covered by unit tests written in their documents' shape. - [FEP-67ff: FEDERATION.md](https://codeberg.org/fediverse/fep/src/branch/main/fep/67ff/fep-67ff.md) - [FEP-f1d5: NodeInfo in Fediverse Software](https://codeberg.org/fediverse/fep/src/branch/main/fep/f1d5/fep-f1d5.md) - [FEP-2c59: Discovery of a WebFinger address from an ActivityPub actor](https://codeberg.org/fediverse/fep/src/branch/main/fep/2c59/fep-2c59.md) +- [FEP-d556: Server-Level Actor Discovery Using WebFinger](https://codeberg.org/fediverse/fep/src/branch/main/fep/d556/fep-d556.md) + (WebFinger for the server's origin links its instance actor as `…#Service`) and + [FEP-2677: Identifying the Application Actor](https://codeberg.org/fediverse/fep/src/branch/main/fep/2677/fep-2677.md) + (`/.well-known/nodeinfo` links it as `…#Application`) +- [FEP-844e: Capability discovery](https://codeberg.org/fediverse/fep/src/branch/main/fep/844e/fep-844e.md): the instance + actor's `implements`, and every other actor's `generator`, name RFC 9421 (verified with RSA keys) - [FEP-1b12: Group federation](https://codeberg.org/fediverse/fep/src/branch/main/fep/1b12/fep-1b12.md) (communities; see "Groups") - [FEP-044f: Consent-respecting quote posts](https://codeberg.org/fediverse/fep/src/branch/main/fep/044f/fep-044f.md) (`QuoteAuthorization` at `/parrot-licences/{id}`; checked with Mastodon both ways) @@ -71,12 +81,15 @@ covered by unit tests written in their documents' shape. - [FEP-c0e0: Emoji reactions](https://codeberg.org/fediverse/fep/src/branch/main/fep/c0e0/fep-c0e0.md) (`EmojiReact`, and Misskey's `Like` with content) - [FEP-5feb: Search indexing consent](https://codeberg.org/fediverse/fep/src/branch/main/fep/5feb/fep-5feb.md) (`indexable`) +- [FEP-521a: Representing actor's public keys](https://codeberg.org/fediverse/fep/src/branch/main/fep/521a/fep-521a.md) and + [FEP-8b32: Object Integrity Proofs](https://codeberg.org/fediverse/fep/src/branch/main/fep/8b32/fep-8b32.md) (see + "Integrity proofs") - [FEP-8fcf: Followers collection synchronization across servers](https://codeberg.org/fediverse/fep/src/branch/main/fep/8fcf/fep-8fcf.md) (sent and honoured; see "Followers synchronisation") Planned (see `docs/ROADMAP.md`, phases P7 and P8, and the per-platform notes in `docs/INTEROP.md`): FEP-9098 (custom emoji), FEP-7888 and FEP-f228 (threads), FEP-7628 (Move), FEP-8967 (link -attachments), FEP-521a and FEP-8b32 (keys and integrity proofs), FEP-ae0c (relays). +attachments), FEP-ae0c (relays). ## Actors @@ -297,8 +310,8 @@ forwards as its author sent it (Activity-Relay), read again from its origin like announces (aode-relay), kept as its author's and never as the relay's boost. Nothing else is taken from a relay. A persona's public post outside any group, its edit and its deletion also go to the relays that accepted us (owner decision 2026-10-06), as Mastodon sends them; nothing less public, and no boost. Mastodon takes a post Activity-Relay forwards -only with an LD signature or an FEP-8b32 proof, which PrivaPub does not yet add, so it reaches Mastodon through relays -that announce (aode-relay). +only with an LD signature or an FEP-8b32 proof: PrivaPub's carry a proof, so they reach Mastodon through both kinds of +relay (checked live). ## Server descriptions and the crawler @@ -379,5 +392,15 @@ Posts with a location (shown to nearby users of this server) never leave the ser - Collections expose counts, not members: `/groupies` and `/stalking` give the same totals as the Mastodon API's follower and following counts, and the outbox's `totalItems` is the persona's post count, though only public posts are listed. `/api/v1/instance/peers` is empty: which servers this one talks to is not published. -- Only RSA keys are verified, under draft-cavage or RFC 9421; Ed25519 (FEP-521a) is planned. What PrivaPub sends is - signed with draft-cavage only, which every server reads. +- HTTP signatures are verified with RSA keys only, under draft-cavage or RFC 9421. What PrivaPub sends is signed with + draft-cavage only, which every server reads. +- **Integrity proofs** (FEP-8b32, FEP-521a). Every persona has an Ed25519 key of its own (never shared between personas), + named in its actor's `assertionMethod` as a `Multikey` (`…#ed25519-key`, `publicKeyMultibase`); the terms are defined + in the actor's own context, so no context document has to be fetched. A persona's activity going to a relay carries a + `DataIntegrityProof` (`eddsa-jcs-2022`, `proofPurpose` `assertionMethod`) over the activity as delivered; Mastodon + 4.7 verifies it (checked against its own verifier), so what the relay forwards reaches it. Nothing else carries one: + Mitra takes a proof over the HTTP signature, and refuses one by a key it has not read yet without reading the actor + again, which every server that knew a persona before it had its key would do. What a persona passes on of others' + carries theirs or none. Received: an actor's own Multikeys (at most five, under its id, controlled by it) are kept; + a forwarded activity whose proof one of them verifies is taken as it came, instead of being read again from its origin + (an unknown key of the actor's has the actor read again first). diff --git a/PrivaPub.Tests/Federation/IntegrityProofTests.cs b/PrivaPub.Tests/Federation/IntegrityProofTests.cs new file mode 100644 index 0000000..26f1a6a --- /dev/null +++ b/PrivaPub.Tests/Federation/IntegrityProofTests.cs @@ -0,0 +1,175 @@ +using MongoDB.Entities; + +using PrivaPub.Federation.Actors; +using PrivaPub.Federation.Rendering; +using PrivaPub.Federation.Signing; +using PrivaPub.Models.Federation; +using PrivaPub.Models.User; +using PrivaPub.Tests.Support; + +using System.Text; +using System.Text.Json.Nodes; + +namespace PrivaPub.Tests.Federation +{ + // FEP-8b32 proofs (eddsa-jcs-2022) by a persona's Ed25519 key, which its actor names as a FEP-521a Multikey + public sealed class IntegrityProofTests + { + [Fact] + public void Canonical_json_is_rfc_8785s() + { + // RFC 8785, section 3.2.2 + var input = JsonNode.Parse("{\"numbers\":[333333333.33333329,1E30,4.50,2e-3,0.000000000000000000000000001]," + + "\"string\":\"\\u20ac$\\u000F\\u000aA'\\u0042\\u0022\\u005c\\\\\\\"\\/\",\"literals\":[null,true,false]}"); + + Assert.Equal("{\"literals\":[null,true,false],\"numbers\":[333333333.3333333,1e+30,4.5,0.002,1e-27],\"string\":\"€$\\u000f\\nA'B\\\"\\\\\\\\\\\"/\"}", + Jcs.Serialize(input)); + } + + [Fact] + public void Base58_and_multikeys_read_back_what_they_write() + { + Assert.Equal("2NEpo7TZRRrLZSi2U", IntegrityProofs.Base58(Encoding.ASCII.GetBytes("Hello World!"))); + Assert.Equal("112", IntegrityProofs.Base58(new byte[] { 0, 0, 1 })); + Assert.Equal(new byte[] { 0, 0, 1 }, IntegrityProofs.FromBase58("112")); + + var publicKey = IntegrityProofs.PublicKey(IntegrityProofs.NewSeed()); + var multikey = IntegrityProofs.Multikey(publicKey); + Assert.StartsWith("z6Mk", multikey); + Assert.Equal(publicKey, IntegrityProofs.FromMultikey(multikey)); + } + + [Fact] + public void A_proof_verifies_with_its_key_and_with_nothing_changed() + { + var seed = IntegrityProofs.NewSeed(); + var activity = new JsonObject + { + ["@context"] = ActivityPubRenderer.Context(), ["id"] = "https://privapub.test/peasants/alice/grunts/1", ["type"] = "Create", + ["actor"] = "https://privapub.test/peasants/alice", ["object"] = new JsonObject { ["type"] = "Note", ["content"] = "

ciao, è così

", ["width"] = 4.5 } + }; + activity["proof"] = IntegrityProofs.Create(activity, "https://privapub.test/peasants/alice#ed25519-key", seed, DateTime.UtcNow); + + var delivered = JsonNode.Parse(activity.ToJsonString())!.AsObject(); + Assert.True(IntegrityProofs.Verify(delivered, IntegrityProofs.PublicKey(seed))); + Assert.Equal("assertionMethod", delivered["proof"]!["proofPurpose"]!.GetValue()); + + var changed = delivered.DeepClone().AsObject(); + changed["object"]!["content"] = "

something else

"; + Assert.False(IntegrityProofs.Verify(changed, IntegrityProofs.PublicKey(seed))); + Assert.False(IntegrityProofs.Verify(delivered, IntegrityProofs.PublicKey(IntegrityProofs.NewSeed()))); + } + + [Fact] + public void A_persona_with_a_key_names_it_as_a_multikey() + { + var seed = IntegrityProofs.NewSeed(); + var actor = ActivityPubRenderer.Actor(new LocalActor { Id = "a", UserName = "alice", BaseAddress = Harness.Base, Kind = LocalActorKind.Person, SigningKey = seed }); + var keyless = ActivityPubRenderer.Actor(new LocalActor { Id = "b", UserName = "bob", BaseAddress = Harness.Base, Kind = LocalActorKind.Person }); + + var key = Assert.Single(actor["assertionMethod"]!.AsArray())!; + Assert.Equal(($"{Harness.Base}/peasants/alice#ed25519-key", "Multikey", $"{Harness.Base}/peasants/alice"), + (key["id"]!.GetValue(), key["type"]!.GetValue(), key["controller"]!.GetValue())); + Assert.Equal(IntegrityProofs.PublicKey(seed), IntegrityProofs.FromMultikey(key["publicKeyMultibase"]!.GetValue())); + Assert.Null(keyless["assertionMethod"]); + } + } + + [Trait("Category", "Integration")] + [Xunit.Collection(nameof(Exclusive))] + public sealed class DeliveredProofTests : IAsyncLifetime + { + Harness _harness; + + public async ValueTask InitializeAsync() + { + Assert.SkipUnless(MongoFixture.Enabled, MongoFixture.Skip); + _harness = await Harness.Start(); + } + + public async ValueTask DisposeAsync() + { + if (_harness != default) + await _harness.DisposeAsync(); + } + + // what goes to a relay carries the persona's proof, to be forwarded on its strength; what goes to a server directly + // does not (Mitra refuses a proof by a key it has not read yet); another's activity passed on never gets ours + [Fact] + public async Task A_personas_activity_goes_to_a_relay_with_its_proof_and_elsewhere_without() + { + var token = TestContext.Current.CancellationToken; + var (_, persona) = await _harness.Persona("alice"); + var seed = IntegrityProofs.NewSeed(); + await DB.Default.Update().MatchID(persona.Id).Modify(a => a.SigningKey, seed).ExecuteAsync(token); + var alice = _harness.Local.FromAvatar(await DB.Default.Find().MatchID(persona.Id).ExecuteSingleAsync(token)); + var relayInbox = $"{_harness.Peer.A}/relay-{Guid.NewGuid():N}/inbox"; + var serverInbox = _harness.Peer.A + "/proofs/inbox"; + await DB.Default.SaveAsync(new RelaySubscription + { + Configured = relayInbox, ActorURI = relayInbox.Replace("/inbox", "/actor"), InboxURL = relayInbox, State = RelayState.Accepted + }, token); + var own = new JsonObject { ["id"] = alice.ActivityUri(Guid.NewGuid().ToString("N")), ["type"] = "Create", ["actor"] = alice.Uri, ["object"] = new JsonObject { ["type"] = "Note" } }; + var passedOn = new JsonObject { ["id"] = $"https://elsewhere.example/activities/{Guid.NewGuid():N}", ["type"] = "Create", ["actor"] = "https://elsewhere.example/users/bob" }; + + try + { + await _harness.Delivery.Enqueue(alice, new[] { relayInbox, serverInbox }, own, token); + await _harness.Delivery.Enqueue(alice, new[] { relayInbox }, passedOn, token); + + var toRelay = await _harness.Outgoing(relayInbox); + var signed = Assert.Single(toRelay, a => a["actor"]!.GetValue() == alice.Uri); + Assert.Equal(alice.AssertionKeyId, signed["proof"]!["verificationMethod"]!.GetValue()); + Assert.True(IntegrityProofs.Verify(signed, IntegrityProofs.PublicKey(seed))); + Assert.Null(Assert.Single(toRelay, a => a["actor"]!.GetValue() != alice.Uri)["proof"]); + Assert.Null(Assert.Single(await _harness.Outgoing(serverInbox))["proof"]); + Assert.Null(own["proof"]); + } + finally + { + await DB.Default.DeleteAsync(s => s.InboxURL == relayInbox); + } + } + + // a post forwarded by another server, which its origin does not serve: taken on its author's proof, never without + [Fact] + public async Task A_forwarded_post_with_its_authors_proof_is_taken_as_it_came() + { + var token = TestContext.Current.CancellationToken; + var (_, alice) = await _harness.Persona("alice"); + var author = new RemoteActor(_harness.Peer, "author", ed25519: true); + var forwarder = new RemoteActor(_harness.Peer, "forwarder", _harness.Peer.B); + await DB.Default.SaveAsync(new PrivaPub.Models.Social.Following + { + AvatarId = alice.Id, TargetActorURI = author.Id, TargetInboxURL = author.Id + "/inbox", State = PrivaPub.Models.Social.FollowState.Accepted + }, token); + JsonObject Create(string text) + { + var noteId = $"{new Uri(author.Id).GetLeftPart(UriPartial.Authority)}/notes/{Guid.NewGuid():N}"; + return new JsonObject + { + ["@context"] = "https://www.w3.org/ns/activitystreams", ["id"] = noteId + "/activity", ["type"] = "Create", ["actor"] = author.Id, + ["to"] = new JsonArray("https://www.w3.org/ns/activitystreams#Public"), + ["object"] = new JsonObject + { + ["id"] = noteId, ["type"] = "Note", ["attributedTo"] = author.Id, ["content"] = $"

{text}

", + ["to"] = new JsonArray("https://www.w3.org/ns/activitystreams#Public"), ["published"] = DateTime.UtcNow.ToString("O") + } + }; + } + var proven = author.Prove(Create("proven")); + var tampered = author.Prove(Create("as written")); + tampered["object"]!["content"] = "

changed on the way

"; + var bare = Create("unproven"); + + await _harness.Deliver(forwarder, "/human-centipede", proven); + await _harness.Deliver(forwarder, "/human-centipede", tampered); + await _harness.Deliver(forwarder, "/human-centipede", bare); + + Task Held(JsonObject create) => DB.Default.Find().Match(p => p.ObjectURI == create["object"]!["id"]!.GetValue()).ExecuteAnyAsync(token); + Assert.True(await Held(proven)); + Assert.False(await Held(tampered)); + Assert.False(await Held(bare)); + } + } +} diff --git a/PrivaPub.Tests/Http/WellKnownTests.cs b/PrivaPub.Tests/Http/WellKnownTests.cs index 286f627..2584b68 100644 --- a/PrivaPub.Tests/Http/WellKnownTests.cs +++ b/PrivaPub.Tests/Http/WellKnownTests.cs @@ -38,6 +38,31 @@ namespace PrivaPub.Tests.Http static string Link(JsonObject document, string rel) => document["links"]!.AsArray().Single(l => l!["rel"]!.GetValue() == rel)!["href"]!.GetValue(); + // FEP-d556, FEP-2677 and FEP-844e: the server's own actor is found from its origin and from NodeInfo, and says what + // the server reads that its documents do not show + [Fact] + public async Task The_server_actor_is_found_from_the_origin_and_nodeinfo_and_tells_what_it_implements() + { + var instance = $"{Base}/peasants/privapub"; + foreach (var resource in new[] { Base, Base + "/" }) + { + var found = await WebFinger(resource); + Assert.Equal(HttpStatusCode.OK, found.Status); + Assert.Equal(resource, found.Json["subject"]!.GetValue()); + Assert.Equal(instance, Link(found.Json, "https://www.w3.org/ns/activitystreams#Service")); + } + var nodeinfo = await _client.Fetch("/.well-known/nodeinfo", "application/json"); + Assert.Equal(instance, Link(nodeinfo.Json, "https://www.w3.org/ns/activitystreams#Application")); + + var actor = (await _client.Fetch("/peasants/privapub")).Json; + Assert.Contains(actor["implements"]!.AsArray(), i => i!["href"]!.GetValue() == "https://datatracker.ietf.org/doc/html/rfc9421"); + var persona = await _host.Persona(await _host.SignUp(), "generated"); + var personaActor = (await _client.Fetch($"/peasants/{persona.UserName}")).Json; + Assert.Equal("Application", personaActor["generator"]!["type"]!.GetValue()); + Assert.Contains(personaActor["generator"]!["implements"]!.AsArray(), i => i!["href"]!.GetValue() == "https://datatracker.ietf.org/doc/html/rfc9421"); + Assert.StartsWith("z6Mk", personaActor["assertionMethod"]![0]!["publicKeyMultibase"]!.GetValue()); + } + [Fact] public async Task WebFinger_finds_a_persona_by_acct_and_by_actor_uri() { diff --git a/PrivaPub.Tests/Support/RemoteActor.cs b/PrivaPub.Tests/Support/RemoteActor.cs index 5415592..ae4ae48 100644 --- a/PrivaPub.Tests/Support/RemoteActor.cs +++ b/PrivaPub.Tests/Support/RemoteActor.cs @@ -15,11 +15,15 @@ namespace PrivaPub.Tests.Support readonly RSA _key = RSA.Create(2048); readonly bool _namesSharedInbox; readonly bool _hasWall; + readonly string _ed25519; // sharedInbox: whether its document names its server's shared inbox (endpoints.sharedInbox), as Mastodon's do; wall: - // whether it has a wall (sm:wall, Smithereen's) - public RemoteActor(Peer peer, string name, string origin = default, string type = "Person", bool sharedInbox = false, bool wall = false) + // whether it has a wall (sm:wall, Smithereen's); ed25519: whether it has an Ed25519 key (FEP-521a) to prove what it + // sends (FEP-8b32) + public RemoteActor(Peer peer, string name, string origin = default, string type = "Person", bool sharedInbox = false, bool wall = false, + bool ed25519 = false) { + _ed25519 = ed25519 ? PrivaPub.Federation.Signing.IntegrityProofs.NewSeed() : default; Name = $"{name}{Guid.NewGuid():N}"[..20]; Id = $"{origin ?? peer.A}/users/{Name}"; Type = type; @@ -36,6 +40,13 @@ namespace PrivaPub.Tests.Support public string SharedInbox => Id.Split("/users/")[0] + "/inbox"; public string Wall => Id + "/wall"; + // the activity with its proof by this actor's Ed25519 key + public JsonObject Prove(JsonObject activity) + { + activity["proof"] = PrivaPub.Federation.Signing.IntegrityProofs.Create(activity, Id + "#ed25519-key", _ed25519, DateTime.UtcNow); + return activity; + } + public JsonObject Document() { var document = new JsonObject @@ -58,6 +69,12 @@ namespace PrivaPub.Tests.Support document["endpoints"] = new JsonObject { ["sharedInbox"] = SharedInbox }; if (_hasWall) document["wall"] = Wall; + if (_ed25519 != default) + document["assertionMethod"] = new JsonArray(new JsonObject + { + ["id"] = Id + "#ed25519-key", ["type"] = "Multikey", ["controller"] = Id, + ["publicKeyMultibase"] = PrivaPub.Federation.Signing.IntegrityProofs.Multikey(PrivaPub.Federation.Signing.IntegrityProofs.PublicKey(_ed25519)) + }); return document; } diff --git a/PrivaPub/Federation/Actors/ActorDocument.cs b/PrivaPub/Federation/Actors/ActorDocument.cs index cc8244d..38d9d1b 100644 --- a/PrivaPub/Federation/Actors/ActorDocument.cs +++ b/PrivaPub/Federation/Actors/ActorDocument.cs @@ -40,6 +40,7 @@ namespace PrivaPub.Federation.Actors public List Emojis { get; init; } = new(); public Dictionary Fields { get; init; } = new(); public IReadOnlyList Keys { get; init; } = Array.Empty(); + public List AssertionKeys { get; init; } = new(); public List Features { get; init; } = new(); public ActorKey Key(string keyId) => Keys.FirstOrDefault(k => k.Id == keyId); @@ -81,6 +82,7 @@ namespace PrivaPub.Federation.Actors Icon = root.TryGetProperty("icon", out var icon) ? RemoteActorService.Text(icon, "url") : default, Discoverable = !root.TryGetProperty("discoverable", out var discoverable) || discoverable.ValueKind != JsonValueKind.False, Keys = ParseKeys(root, id), + AssertionKeys = ParseAssertionKeys(root, id), Indexable = Flag(root, "indexable"), Locked = Flag(root, "manuallyApprovesFollowers"), Memorial = Flag(root, "memorial"), @@ -121,6 +123,32 @@ namespace PrivaPub.Federation.Actors return fields; } + const int MaxAssertionKeys = 5; + + // its Ed25519 keys (FEP-521a) its document holds, as Mastodon reads them: Multikeys it controls, under its own id + static List ParseAssertionKeys(JsonElement root, string actorId) + { + var keys = new List(); + if (!root.TryGetProperty("assertionMethod", out var methods)) + return keys; + var candidates = methods.ValueKind switch + { + JsonValueKind.Object => new[] { methods }, + JsonValueKind.Array => methods.EnumerateArray().Where(k => k.ValueKind == JsonValueKind.Object).ToArray(), + _ => Array.Empty() + }; + foreach (var key in candidates.Take(MaxAssertionKeys)) + { + var keyId = RemoteActorService.Text(key, "id"); + var publicKey = Signing.IntegrityProofs.FromMultikey(RemoteActorService.Text(key, "publicKeyMultibase")); + if (RemoteActorService.Text(key, "type") != "Multikey" || RemoteActorService.Text(key, "controller") != actorId + || keyId == default || !keyId.StartsWith(actorId + "#", StringComparison.Ordinal) || publicKey == default) + continue; + keys.Add(new Models.User.AssertionKey { Id = keyId, PublicKey = Convert.ToBase64String(publicKey) }); + } + return keys; + } + static List ParseKeys(JsonElement root, string actorId) { var keys = new List(); diff --git a/PrivaPub/Federation/Actors/LocalActorService.cs b/PrivaPub/Federation/Actors/LocalActorService.cs index 795855e..05e41e3 100644 --- a/PrivaPub/Federation/Actors/LocalActorService.cs +++ b/PrivaPub/Federation/Actors/LocalActorService.cs @@ -26,6 +26,7 @@ namespace PrivaPub.Federation.Actors public string ThumbnailURL { get; init; } public string PrivateKeyPem { get; init; } public string PublicKeyPem { get; init; } + public string SigningKey { get; init; }//a persona's Ed25519 seed (FEP-521a), for the proofs its activities carry (FEP-8b32) public bool Discoverable { get; init; } = true; public bool ManuallyApprovesFollowers { get; init; } public bool IsFederated { get; init; } = true; @@ -40,6 +41,7 @@ namespace PrivaPub.Federation.Actors public string Uri => $"{BaseAddress}/peasants/{UserName}"; public string KeyId => $"{Uri}#main-key"; + public string AssertionKeyId => $"{Uri}#ed25519-key"; public string Inbox => $"{Uri}/mouth"; public string Outbox => $"{Uri}/anus"; public string Followers => $"{Uri}/groupies"; @@ -231,6 +233,7 @@ namespace PrivaPub.Federation.Actors ThumbnailURL = avatar.ThumbnailURL, PrivateKeyPem = avatar.PrivateKey, PublicKeyPem = avatar.PublicKey, + SigningKey = avatar.SigningKey, Published = avatar.PublishedOn, Fields = avatar.Fields ?? new Dictionary(), ManuallyApprovesFollowers = avatar.Settings?.IsLocked == true, diff --git a/PrivaPub/Federation/Actors/RemoteActorService.cs b/PrivaPub/Federation/Actors/RemoteActorService.cs index 0faf927..f7101b4 100644 --- a/PrivaPub/Federation/Actors/RemoteActorService.cs +++ b/PrivaPub/Federation/Actors/RemoteActorService.cs @@ -206,6 +206,7 @@ namespace PrivaPub.Federation.Actors .Modify(a => a.FollowingURL, actor.Following) .Modify(a => a.FeaturedURL, Origin.Same(actor.Featured, actor.Id) ? actor.Featured : default) .Modify(a => a.WallURL, Origin.Same(actor.Wall, actor.Id) ? actor.Wall : default) + .Modify(a => a.AssertionKeys, actor.AssertionKeys) .Modify(a => a.SharedInboxURL, actor.SharedInbox) .Modify(a => a.PictureURL, actor.Icon) .Modify(a => a.ThumbnailURL, actor.Header) diff --git a/PrivaPub/Federation/Controllers/WellKnownController.cs b/PrivaPub/Federation/Controllers/WellKnownController.cs index 1645a02..a3b9657 100644 --- a/PrivaPub/Federation/Controllers/WellKnownController.cs +++ b/PrivaPub/Federation/Controllers/WellKnownController.cs @@ -36,6 +36,19 @@ namespace PrivaPub.Federation.Controllers { if (string.IsNullOrEmpty(resource)) return BadRequest(); + // the server itself (FEP-d556): its instance actor + if (resource.TrimEnd('/') == _localActors.BaseAddress) + { + var instance = await _localActors.GetInstanceActor(token); + return Content(new JsonObject + { + ["subject"] = resource, + ["links"] = new JsonArray(new JsonObject + { + ["rel"] = "https://www.w3.org/ns/activitystreams#Service", ["type"] = "application/activity+json", ["href"] = instance.Uri + }) + }.ToJsonString(), "application/jrd+json; charset=utf-8"); + } LocalActor actor; // Mastodon, GoToSocial and Pleroma also take a bare user@domain or @user@domain, so we do too. @@ -72,7 +85,7 @@ namespace PrivaPub.Federation.Controllers } [HttpGet, Route("/.well-known/nodeinfo")] - public IActionResult NodeInfoLinks() + public async Task NodeInfoLinks(CancellationToken token) { var document = new JsonObject { @@ -86,6 +99,12 @@ namespace PrivaPub.Federation.Controllers { ["rel"] = "http://nodeinfo.diaspora.software/ns/schema/2.0", ["href"] = $"{_localActors.BaseAddress}/nodeinfo/2.0" + }, + // the application actor (FEP-2677) + new JsonObject + { + ["rel"] = "https://www.w3.org/ns/activitystreams#Application", + ["href"] = (await _localActors.GetInstanceActor(token)).Uri }) }; return Content(document.ToJsonString(), "application/json; charset=utf-8"); diff --git a/PrivaPub/Federation/Inbox/Forwarded.cs b/PrivaPub/Federation/Inbox/Forwarded.cs index 448b506..2723470 100644 --- a/PrivaPub/Federation/Inbox/Forwarded.cs +++ b/PrivaPub/Federation/Inbox/Forwarded.cs @@ -17,9 +17,21 @@ namespace PrivaPub.Federation.Inbox // activity in its threads. The signature proves who passed it on, never who wrote it, so nothing in it is believed: a // Create or an Update is taken as its object reads at the actor's origin now, a Delete once that origin says the object // is gone, and anything else is let go (a vote or a follow cannot be checked against its origin). An LD signature - // (RsaSignature2017) would prove the author, but needs JSON-LD; integrity proofs (FEP-8b32) will. + // (RsaSignature2017) would prove the author, but needs JSON-LD. An integrity proof (FEP-8b32, eddsa-jcs-2022) by one of + // the actor's Ed25519 keys does: such an activity is taken as forwarded, read again from nowhere. public static class Forwarded { + // whether the activity carries a proof made by one of the actor's own keys (FEP-521a) + public static bool Proven(JsonNode activity, Models.User.ForeignAvatar actor) => + activity is JsonObject document && document["proof"] is JsonObject proof && Value(proof, "verificationMethod") is { } method + && actor.AssertionKeys.FirstOrDefault(k => k.Id == method) is { } key + && Signing.IntegrityProofs.Verify(document, Convert.FromBase64String(key.PublicKey)); + + // whether it names a key of the actor's own that we do not hold (the actor was read before it had one) + public static bool NamesUnknownKey(JsonNode activity, Models.User.ForeignAvatar actor) => + activity is JsonObject document && document["proof"] is JsonObject proof && Value(proof, "verificationMethod") is { } method + && method.StartsWith(actor.ActorURI + "#", StringComparison.Ordinal) && actor.AssertionKeys.All(k => k.Id != method); + // what may be taken from a forwarder: a post of the activity's actor, created, edited or deleted public static bool Takeable(string type, JsonNode activity, string actorUri) { diff --git a/PrivaPub/Federation/Inbox/InboxProcessor.cs b/PrivaPub/Federation/Inbox/InboxProcessor.cs index 5c4cc64..807aab9 100644 --- a/PrivaPub/Federation/Inbox/InboxProcessor.cs +++ b/PrivaPub/Federation/Inbox/InboxProcessor.cs @@ -65,7 +65,9 @@ namespace PrivaPub.Federation.Inbox Record(job, payload, activity, type, started, new ArrivalVerdict(), Interactions.Deferred, "actor-unavailable"); return JobOutcome.Retry("the actor could not be loaded"); } - if (payload.ForwardedBy != default) + if (payload.ForwardedBy != default && Forwarded.NamesUnknownKey(activity, actor)) + actor = await _remoteActors.GetActor(actor.ActorURI, refresh: true, token) ?? actor; + if (payload.ForwardedBy != default && !Forwarded.Proven(activity, actor)) { var (confirmed, drop) = await Forwarded.Confirm(activity, type, actor.ActorURI, _remoteActors, token); if (drop != default) diff --git a/PrivaPub/Federation/Outbox/DeliveryService.cs b/PrivaPub/Federation/Outbox/DeliveryService.cs index c4f6a27..d097c81 100644 --- a/PrivaPub/Federation/Outbox/DeliveryService.cs +++ b/PrivaPub/Federation/Outbox/DeliveryService.cs @@ -47,9 +47,26 @@ namespace PrivaPub.Federation.Outbox { var body = activity.ToJsonString(); var activityId = activity["id"] is JsonValue id && id.TryGetValue(out var text) ? text : default; - var jobs = inboxes + var targets = inboxes .Where(i => !string.IsNullOrEmpty(i) && !i.StartsWith(signer.BaseAddress + "/", StringComparison.OrdinalIgnoreCase)) .Distinct(StringComparer.Ordinal) + .ToList(); + // what goes to a relay carries the signer's proof (FEP-8b32): the relay passes it on as it came, signed with its own + // key, and Mastodon takes it on the proof's strength. Nothing else does: a server that knew the persona before it + // had its key (Mitra) refuses a proof by a key it does not hold, and does not read the actor again + var relayed = new HashSet(StringComparer.Ordinal); + if (!string.IsNullOrEmpty(signer.SigningKey) && activity["actor"] is JsonValue actor && actor.TryGetValue(out var actorUri) && actorUri == signer.Uri) + relayed = (await DB.Default.Find().Match(s => targets.Contains(s.InboxURL)).Project(s => s.InboxURL).ExecuteAsync(token)) + .ToHashSet(StringComparer.Ordinal); + var provenBody = default(string); + if (relayed.Count > 0) + { + var proven = activity.DeepClone().AsObject(); + proven.Remove("proof"); + proven["proof"] = IntegrityProofs.Create(proven, signer.AssertionKeyId, signer.SigningKey, DateTime.UtcNow); + provenBody = proven.ToJsonString(); + } + var jobs = targets .Select(inbox => Uri.TryCreate(inbox, UriKind.Absolute, out var uri) ? (inbox, uri) : default) .Where(target => target.uri != default) .Select(target => new Job @@ -57,7 +74,7 @@ namespace PrivaPub.Federation.Outbox Kind = JobKind.Deliver, Host = target.uri.Host.ToLowerInvariant(), DedupeKey = activityId == default ? default : again == default ? $"{activityId}|{target.inbox}" : $"{activityId}|{target.inbox}|{again}", - Payload = JsonSerializer.Serialize(new DeliveryPayload(signer.Id, signer.Kind, target.inbox, body)) + Payload = JsonSerializer.Serialize(new DeliveryPayload(signer.Id, signer.Kind, target.inbox, relayed.Contains(target.inbox) ? provenBody : body)) }) .ToList(); if (jobs.Count > 0) diff --git a/PrivaPub/Federation/Rendering/ActivityPubRenderer.cs b/PrivaPub/Federation/Rendering/ActivityPubRenderer.cs index a9ada19..c9b44b5 100644 --- a/PrivaPub/Federation/Rendering/ActivityPubRenderer.cs +++ b/PrivaPub/Federation/Rendering/ActivityPubRenderer.cs @@ -79,9 +79,29 @@ namespace PrivaPub.Federation.Rendering ["privacySettings"] = "sm:privacySettings", ["wallPosting"] = "sm:wallPosting", ["wallPostVisibility"] = "sm:wallPostVisibility", - ["allowedTo"] = "sm:allowedTo" + ["allowedTo"] = "sm:allowedTo", + // FEP-521a keys and FEP-8b32 proofs, defined here rather than by the data-integrity and multikey contexts, + // which strict JSON-LD readers would have to fetch + ["assertionMethod"] = new JsonObject { ["@id"] = "sec:assertionMethod", ["@type"] = "@id", ["@container"] = "@set" }, + ["Multikey"] = "sec:Multikey", + ["controller"] = new JsonObject { ["@id"] = "sec:controller", ["@type"] = "@id" }, + ["publicKeyMultibase"] = new JsonObject { ["@id"] = "sec:publicKeyMultibase", ["@type"] = "sec:multibase" }, + ["DataIntegrityProof"] = "sec:DataIntegrityProof", + ["proof"] = new JsonObject { ["@id"] = "sec:proof", ["@type"] = "@id", ["@container"] = "@graph" }, + ["cryptosuite"] = new JsonObject { ["@id"] = "sec:cryptosuite", ["@type"] = "sec:cryptosuiteString" }, + ["proofValue"] = new JsonObject { ["@id"] = "sec:proofValue", ["@type"] = "sec:multibase" }, + ["proofPurpose"] = new JsonObject { ["@id"] = "sec:proofPurpose", ["@type"] = "@vocab" }, + ["verificationMethod"] = new JsonObject { ["@id"] = "sec:verificationMethod", ["@type"] = "@id" }, + ["implements"] = new JsonObject { ["@id"] = "https://w3id.org/fep/844e/implements", ["@type"] = "@id", ["@container"] = "@set" } }); + // what PrivaPub reads that a sender cannot tell from our documents (FEP-844e): RFC 9421 signatures, with RSA keys + static JsonArray Implements() => new(new JsonObject + { + ["href"] = "https://datatracker.ietf.org/doc/html/rfc9421", + ["name"] = "RFC-9421: HTTP Message Signatures" + }); + public static string Html(string markdown) => string.IsNullOrEmpty(markdown) ? string.Empty : Markdown.ToHtml(markdown, Pipeline).Trim(); @@ -150,6 +170,20 @@ namespace PrivaPub.Federation.Rendering document["attributedTo"] = actor.Wardens; document["postingRestrictedToMods"] = actor.PostingRestrictedToModerators; } + // what the server reads (FEP-844e): on the application actor itself, on any other as its generator + if (actor.Kind == LocalActorKind.Application) + document["implements"] = Implements(); + else + document["generator"] = new JsonObject { ["type"] = "Application", ["implements"] = Implements() }; + // its Ed25519 key (FEP-521a), which signs the proofs its activities carry (FEP-8b32) + if (!string.IsNullOrEmpty(actor.SigningKey)) + document["assertionMethod"] = new JsonArray(new JsonObject + { + ["id"] = actor.AssertionKeyId, + ["type"] = "Multikey", + ["controller"] = actor.Uri, + ["publicKeyMultibase"] = Signing.IntegrityProofs.Multikey(Signing.IntegrityProofs.PublicKey(actor.SigningKey)) + }); if (!string.IsNullOrEmpty(actor.PictureURL)) document["icon"] = new JsonObject { ["type"] = "Image", ["url"] = actor.PictureURL }; if (!string.IsNullOrEmpty(actor.ThumbnailURL)) diff --git a/PrivaPub/Federation/Signing/IntegrityProofs.cs b/PrivaPub/Federation/Signing/IntegrityProofs.cs new file mode 100644 index 0000000..23bf722 --- /dev/null +++ b/PrivaPub/Federation/Signing/IntegrityProofs.cs @@ -0,0 +1,127 @@ +using Org.BouncyCastle.Crypto.Parameters; +using Org.BouncyCastle.Crypto.Signers; + +using System.Globalization; +using System.Numerics; +using System.Security.Cryptography; +using System.Text; +using System.Text.Json.Nodes; + +namespace PrivaPub.Federation.Signing +{ + // FEP-8b32 object integrity proofs, eddsa-jcs-2022, by an actor's Ed25519 key, which its document names under + // assertionMethod as a FEP-521a Multikey. An activity that carries one needs no HTTP signature by its actor: Mastodon + // 4.7 takes what a relay forwards on its strength. + public static class IntegrityProofs + { + public const string Cryptosuite = "eddsa-jcs-2022"; + const string Base58Alphabet = "123456789ABCDEFGHJKLMNPQRSTUVWXYZabcdefghijkmnopqrstuvwxyz"; + static readonly byte[] Ed25519Codec = { 0xed, 0x01 }; + + // a new key: its 32-byte seed, in base64 + public static string NewSeed() => Convert.ToBase64String(RandomNumberGenerator.GetBytes(32)); + + public static byte[] PublicKey(string seed) => new Ed25519PrivateKeyParameters(Convert.FromBase64String(seed)).GeneratePublicKey().GetEncoded(); + + // the public key as a Multikey's publicKeyMultibase: base58btc ("z") of the ed25519-pub multicodec and the key + public static string Multikey(byte[] publicKey) => "z" + Base58(Ed25519Codec.Concat(publicKey).ToArray()); + + // the Ed25519 public key a publicKeyMultibase holds, or null + public static byte[] FromMultikey(string multibase) + { + var bytes = multibase is { Length: > 1 } && multibase[0] == 'z' ? FromBase58(multibase[1..]) : default; + return bytes is { Length: 34 } && bytes[0] == Ed25519Codec[0] && bytes[1] == Ed25519Codec[1] ? bytes[2..] : default; + } + + public static JsonObject Create(JsonObject document, string verificationMethod, string seed, DateTime created) + { + var proof = new JsonObject + { + ["type"] = "DataIntegrityProof", + ["cryptosuite"] = Cryptosuite, + ["verificationMethod"] = verificationMethod, + ["proofPurpose"] = "assertionMethod", + ["created"] = DateTime.SpecifyKind(created, DateTimeKind.Utc).ToString("yyyy-MM-ddTHH:mm:ssZ", CultureInfo.InvariantCulture) + }; + var signer = new Ed25519Signer(); + signer.Init(true, new Ed25519PrivateKeyParameters(Convert.FromBase64String(seed))); + var data = SignedData(proof, Unsecured(document)); + signer.BlockUpdate(data, 0, data.Length); + proof["proofValue"] = "z" + Base58(signer.GenerateSignature()); + return proof; + } + + // whether the document's proof was made by the key, as Mastodon checks it: a proof that names an @context makes that + // the document's for the check, which must begin with it + public static bool Verify(JsonObject document, byte[] publicKey) + { + if (document["proof"] is not JsonObject proof || Text(proof, "type") != "DataIntegrityProof" || Text(proof, "cryptosuite") != Cryptosuite + || Text(proof, "proofValue") is not { Length: > 1 } value || value[0] != 'z' || publicKey is not { Length: 32 }) + return false; + var options = proof.DeepClone().AsObject(); + options.Remove("proofValue"); + var unsecured = Unsecured(document); + if (options["@context"] is JsonArray context) + { + if (unsecured["@context"] is not JsonArray own || own.Count < context.Count + || !context.Select((c, i) => JsonNode.DeepEquals(c, own[i])).All(same => same)) + return false; + unsecured["@context"] = context.DeepClone(); + } + var signature = FromBase58(value[1..]); + if (signature is not { Length: 64 }) + return false; + var verifier = new Ed25519Signer(); + verifier.Init(false, new Ed25519PublicKeyParameters(publicKey)); + var data = SignedData(options, unsecured); + verifier.BlockUpdate(data, 0, data.Length); + return verifier.VerifySignature(signature); + } + + static JsonObject Unsecured(JsonObject document) + { + var unsecured = document.DeepClone().AsObject(); + unsecured.Remove("proof"); + return unsecured; + } + + // what eddsa-jcs-2022 signs: the SHA-256 of the canonical proof options, then that of the canonical document + static byte[] SignedData(JsonObject options, JsonObject document) => + SHA256.HashData(Encoding.UTF8.GetBytes(Jcs.Serialize(options))).Concat(SHA256.HashData(Encoding.UTF8.GetBytes(Jcs.Serialize(document)))).ToArray(); + + static string Text(JsonObject node, string name) => node[name] is JsonValue value && value.TryGetValue(out var text) ? text : default; + + public static string Base58(byte[] bytes) + { + var number = new BigInteger(bytes, isUnsigned: true, isBigEndian: true); + var builder = new StringBuilder(); + while (number > 0) + { + number = BigInteger.DivRem(number, 58, out var remainder); + builder.Insert(0, Base58Alphabet[(int)remainder]); + } + foreach (var b in bytes) + { + if (b != 0) + break; + builder.Insert(0, '1'); + } + return builder.ToString(); + } + + public static byte[] FromBase58(string text) + { + var number = BigInteger.Zero; + foreach (var c in text) + { + var digit = Base58Alphabet.IndexOf(c); + if (digit < 0) + return default; + number = number * 58 + digit; + } + var body = number.IsZero ? Array.Empty() : number.ToByteArray(isUnsigned: true, isBigEndian: true); + var zeros = text.TakeWhile(c => c == '1').Count(); + return new byte[zeros].Concat(body).ToArray(); + } + } +} diff --git a/PrivaPub/Federation/Signing/Jcs.cs b/PrivaPub/Federation/Signing/Jcs.cs new file mode 100644 index 0000000..8fe28ca --- /dev/null +++ b/PrivaPub/Federation/Signing/Jcs.cs @@ -0,0 +1,138 @@ +using System.Globalization; +using System.Text; +using System.Text.Json; +using System.Text.Json.Nodes; + +namespace PrivaPub.Federation.Signing +{ + // RFC 8785, the JSON Canonicalization Scheme: members sorted by their names' UTF-16 code units, no whitespace, strings + // escaped only where JSON must be, numbers as ECMAScript writes them. What an eddsa-jcs-2022 proof signs (FEP-8b32). + public static class Jcs + { + public static string Serialize(JsonNode node) + { + var builder = new StringBuilder(); + Write(builder, node); + return builder.ToString(); + } + + static void Write(StringBuilder builder, JsonNode node) + { + switch (node) + { + case null: + builder.Append("null"); + break; + case JsonObject obj: + builder.Append('{'); + var first = true; + foreach (var (name, value) in obj.OrderBy(p => p.Key, StringComparer.Ordinal)) + { + if (!first) + builder.Append(','); + first = false; + String(builder, name); + builder.Append(':'); + Write(builder, value); + } + builder.Append('}'); + break; + case JsonArray array: + builder.Append('['); + for (var i = 0; i < array.Count; i++) + { + if (i > 0) + builder.Append(','); + Write(builder, array[i]); + } + builder.Append(']'); + break; + case JsonValue value: + switch (value.GetValueKind()) + { + case JsonValueKind.String: + String(builder, value.GetValue()); + break; + case JsonValueKind.Number: + builder.Append(Number(double.Parse(value.ToJsonString(), NumberStyles.Float, CultureInfo.InvariantCulture))); + break; + case JsonValueKind.True: + builder.Append("true"); + break; + case JsonValueKind.False: + builder.Append("false"); + break; + default: + builder.Append("null"); + break; + } + break; + } + } + + static void String(StringBuilder builder, string text) + { + builder.Append('"'); + foreach (var c in text) + { + switch (c) + { + case '"': + builder.Append("\\\""); + break; + case '\\': + builder.Append("\\\\"); + break; + case '\b': + builder.Append("\\b"); + break; + case '\f': + builder.Append("\\f"); + break; + case '\n': + builder.Append("\\n"); + break; + case '\r': + builder.Append("\\r"); + break; + case '\t': + builder.Append("\\t"); + break; + default: + if (c < 0x20) + builder.Append("\\u").Append(((int)c).ToString("x4", CultureInfo.InvariantCulture)); + else + builder.Append(c); + break; + } + } + builder.Append('"'); + } + + // ECMAScript's Number.prototype.toString: integers without a fraction, the shortest digits that read back the same, + // an exponent from 1e21 up and below 1e-6 + public static string Number(double value) + { + if (double.IsNaN(value) || double.IsInfinity(value)) + throw new ArgumentException("JSON has no NaN or Infinity", nameof(value)); + if (value == 0) + return "0"; + var abs = Math.Abs(value); + if (abs >= 1e21 || abs < 1e-6) + { + var exponential = value.ToString("R", CultureInfo.InvariantCulture).Replace("E", "e"); + var at = exponential.IndexOf('e'); + if (at < 0) + return exponential; + var mantissa = exponential[..at]; + var exponent = int.Parse(exponential[(at + 1)..], CultureInfo.InvariantCulture); + return $"{mantissa}e{(exponent < 0 ? "-" : "+")}{Math.Abs(exponent)}"; + } + var text = value.ToString("R", CultureInfo.InvariantCulture); + if (!text.Contains('E')) + return text; + // "R" chose an exponent ECMAScript would not: written out in full + return decimal.Parse(text, NumberStyles.Float, CultureInfo.InvariantCulture).ToString(CultureInfo.InvariantCulture); + } + } +} diff --git a/PrivaPub/Infrastructure/Data/Migrations/_014_personas_have_ed25519_keys.cs b/PrivaPub/Infrastructure/Data/Migrations/_014_personas_have_ed25519_keys.cs new file mode 100644 index 0000000..17e975f --- /dev/null +++ b/PrivaPub/Infrastructure/Data/Migrations/_014_personas_have_ed25519_keys.cs @@ -0,0 +1,18 @@ +using MongoDB.Entities; + +using PrivaPub.Federation.Signing; +using PrivaPub.Models.User; + +namespace PrivaPub.Infrastructure.Data.Migrations +{ + // every persona signs the proofs its activities carry (FEP-8b32) with an Ed25519 key of its own (FEP-521a): new ones + // get theirs when made, the earlier ones once here + public class _014_personas_have_ed25519_keys : IMigration + { + public async Task UpgradeAsync() + { + foreach (var avatar in await DB.Default.Find().Match(a => a.SigningKey == null).Project(p => p.Include(a => a.ID)).ExecuteAsync()) + await DB.Default.Update().Match(a => a.ID == avatar.ID && a.SigningKey == null).Modify(a => a.SigningKey, IntegrityProofs.NewSeed()).ExecuteAsync(); + } + } +} diff --git a/PrivaPub/Models/User/Avatar.cs b/PrivaPub/Models/User/Avatar.cs index 5566799..cd2f826 100644 --- a/PrivaPub/Models/User/Avatar.cs +++ b/PrivaPub/Models/User/Avatar.cs @@ -14,6 +14,7 @@ namespace PrivaPub.Models.User public Dictionary SharedPersonalContacts { get; set; } = new(); public string PrivateKey { get; set; } public string PublicKey { get; set; } + public string SigningKey { get; set; }//the seed of its Ed25519 key (FEP-521a), which signs its activities' proofs (FEP-8b32) public AvatarAccountState AccountState { get; set; } = AvatarAccountState.Normal; public AvatarSettings Settings { get; set; } = new(); public Dictionary Fields { get; set; } = new(); @@ -76,6 +77,7 @@ namespace PrivaPub.Models.User public string InboxURL { get; set; } public string OutboxURL { get; set; } public string FeaturedURL { get; set; }//featured: the posts it pins + public List AssertionKeys { get; set; } = new();//its Ed25519 keys (FEP-521a), which prove what it sends (FEP-8b32) public string WallURL { get; set; }//sm:wall (FEP-400e): where others write to it, Smithereen's walls public string MovedToURL { get; set; } public List AlsoKnownAs { get; set; } = new();//alsoKnownAs: the accounts it says it also is @@ -158,4 +160,10 @@ namespace PrivaPub.Models.User Banned, Deleted } + + public class AssertionKey + { + public string Id { get; set; } + public string PublicKey { get; set; }//the raw Ed25519 key, base64 + } } diff --git a/PrivaPub/PrivaPub.csproj b/PrivaPub/PrivaPub.csproj index a32a0e0..61c2b00 100644 --- a/PrivaPub/PrivaPub.csproj +++ b/PrivaPub/PrivaPub.csproj @@ -7,6 +7,7 @@ + diff --git a/PrivaPub/Services/ClientToServer/Private/IPrivateAvatarUsersService.cs b/PrivaPub/Services/ClientToServer/Private/IPrivateAvatarUsersService.cs index 54328ad..1625f6a 100644 --- a/PrivaPub/Services/ClientToServer/Private/IPrivateAvatarUsersService.cs +++ b/PrivaPub/Services/ClientToServer/Private/IPrivateAvatarUsersService.cs @@ -67,6 +67,7 @@ namespace PrivaPub.Services.ClientToServer.Private PersonalNote = form.PersonalNote, PrivateKey = privateKey, PublicKey = publicKey, + SigningKey = Federation.Signing.IntegrityProofs.NewSeed(), Domain = _localActors.BaseAddress }; newAvatar.ID = (string)newAvatar.GenerateNewID(); diff --git a/docs/INTEROP.md b/docs/INTEROP.md index f1db5e1..30a6dde 100644 --- a/docs/INTEROP.md +++ b/docs/INTEROP.md @@ -825,7 +825,10 @@ without a port, before it gives out its OAuth client. ### Mitra 5.9.1 - Signs its deliveries with RSA (draft-cavage) and adds an FEP-8b32 proof made with its Ed25519 key (FEP-521a), which - PrivaPub does not verify yet; the HTTP signature is enough for what it delivers itself. + PrivaPub verifies when the activity comes forwarded; the HTTP signature is enough for what it delivers itself. +- **Prefers a proof to the HTTP signature:** an activity delivered with a proof by a key Mitra has not read is refused + (401, "key not found in cache"), and Mitra does not read the actor again. So nothing PrivaPub delivers directly carries + a proof; only what goes to relays does. - Its Mastodon API resolves an account elsewhere only when the search is not limited to a type (`/api/v2/search?resolve=true`, no `type=accounts`); reactions go through Pleroma's route (`PUT /api/v1/pleroma/statuses/:id/reactions/:emoji`). @@ -840,12 +843,16 @@ without a port, before it gives out its OAuth client. follower whose actor ends in `/relay` (LitePub's way) gets an `Announce` instead. - **aode-relay** takes either kind and announces the post from its own actor. - A forwarded post carries its author's LD signature when Mastodon wrote it; PrivaPub does not verify LD signatures, - so it reads every relayed post again from its origin (one signed request each). Verifying them, or FEP-8b32 proofs, - would save that request. -- **Pasture evidence (2026-10-05, `tools/pasture/scenarios/relay.sh`):** 13 checks pass: PrivaPub's instance actor + so it reads such a relayed post again from its origin (one signed request each). One carrying a FEP-8b32 proof by its + author's Ed25519 key (Mitra's, Fedify's, PrivaPub's) is taken as it came. +- Mastodon 4.7 takes what Activity-Relay forwards only with an LD signature or a FEP-8b32 proof; personas' activities + going to a relay carry a proof, so their public posts reach Mastodon through it too. Mastodon reads a known account's keys again at + most daily, so a persona's key reaches a server that held its actor before at its next refresh. +- **Pasture evidence (2026-10-06, `tools/pasture/scenarios/relay.sh`):** 16 checks pass: PrivaPub's instance actor subscribes to each relay and takes its Accept; Mastodon subscribes; a public post of a Mastodon account nobody here follows reaches the federated timeline, forwarded by Activity-Relay and announced by aode-relay (as its author's, - never as the relay's boost), and nobody's home; nothing of a persona's goes to a relay. + never as the relay's boost), and nobody's home; a persona's public post goes to the relays, nothing less public, and + reaches Mastodon through both (forwarded on its proof, and announced). ### Smithereen 1.0.3 @@ -1165,7 +1172,7 @@ snapshots; `/api/privapub/v1/cdns` and `/cdns/:domain` group servers by CDN, wee | 400 vs 401 | A 400 or 401 makes Mastodon 4.7 and WordPress retry with the other scheme | 401 only for signature failures (we do this); 400 only for bodies that are really malformed | P1 (keep) | | Temporary key failure | Mastodon answers 503 | We should answer 503 too, and treat a 503 as a retry in delivery | P2 | | Keys | `publicKey` can be an array (Mastodon 4.6). FEP-521a `assertionMethod` Multikey is FINAL (Ed25519 `z6Mk…`). GoToSocial key ids have no `#` and point at a stub. | Read all of these | P2 | -| Integrity proofs | FEP-8b32 `eddsa-jcs-2022`: JCS, no JSON-LD. Sent by Mitra, Streams, Hubzilla, Fedify and others; Mastodon verifies them from 4.7 | Verify, so relayed or forwarded objects need no refetch | P2 | +| Integrity proofs | FEP-8b32 `eddsa-jcs-2022`: JCS, no JSON-LD. Sent by Mitra, Streams, Hubzilla, Fedify and others; Mastodon verifies them from 4.7; Mitra prefers a proof to the HTTP signature and refuses one by a key it has not read | Verify, so relayed or forwarded objects need no refetch. **Done 2026-10-06**, both ways: personas' activities to relays carry one, forwarded ones with a valid proof are taken as they came | P2 | | LD signatures | Mastodon still sends `RsaSignature2017` | Ignore, and refetch from origin (we do) | — | | Query string | GoToSocial, Akkoma 3.20 and Misskey 2026.10 sign it; GoToSocial retries without it | Verify both ways | P1 | | `hs2019` | The algorithm comes from the key; some senders hash with SHA-512 | Try rsa-sha256, then sha512 | P2 | diff --git a/docs/ROADMAP.md b/docs/ROADMAP.md index 179cd60..37d8752 100644 --- a/docs/ROADMAP.md +++ b/docs/ROADMAP.md @@ -686,16 +686,20 @@ it, raw where it doesn't. - RFC 9421 inbound (RSA and Ed25519, Content-Digest): **RSA done 2026-10-05** (PKCS#1 v1.5 and PSS, Content-Digest, deliveries and signed fetches); Ed25519 waits for FEP-521a keys; - outbound double-knock, remembered per host; - - `publicKey` arrays and FEP-521a Multikey; - - FEP-8b32 proof verification; + - `publicKey` arrays and FEP-521a Multikey: Multikeys **done 2026-10-06** (each persona's Ed25519 key published, peers' + read); + - FEP-8b32 proofs: **done 2026-10-06** (`eddsa-jcs-2022` on a persona's activity going to a relay, and only there: + Mitra refuses a proof by a key it has not read and does not read the actor again; a forwarded activity with a + proof its actor's key verifies is taken without reading it again; Mastodon accepts PrivaPub's, so Activity-Relay's + forwards reach it); - `hs2019` with SHA-512. - **Discovery:** - a relay client for both relay styles: **done 2026-10-05/06** (`Federation:Relays`; forwarded posts read again from their origin, announces unwrapped; personas' public posts sent to them, owner decision; checked live against Activity-Relay and aode-relay). Mastodon drops what Activity-Relay forwards without an LD signature or FEP-8b32 - proof; - - instance actor discovery (FEP-d556, FEP-2677); - - `implements` (FEP-844e). + proof, which personas' activities now carry; + - instance actor discovery (FEP-d556, FEP-2677): **done 2026-10-06**; + - `implements` (FEP-844e): **done 2026-10-06** (RFC 9421, on the instance actor and as every actor's `generator`). - **Mastodon API:** ~~streaming WebSocket~~ (done 2026-10-05: `/api/v1/streaming` as a WebSocket and as server-sent events, user, notification, public, hashtag and list streams, each event mapped for its persona; a deletion reaches only the streams that showed the post), Web Push (gated: outbound traffic to push services), grouped notifications. diff --git a/tools/pasture/scenarios/relay.sh b/tools/pasture/scenarios/relay.sh index bff6c50..4d24c8c 100644 --- a/tools/pasture/scenarios/relay.sh +++ b/tools/pasture/scenarios/relay.sh @@ -1,7 +1,8 @@ # Relays as PrivaPub uses them (Federation:Relays in appsettings.Pasture.json): Activity-Relay, which forwards what its # subscribers send, and aode-relay, which announces it. For each, the instance actor subscribes, Mastodon subscribes too, # a public post of a Mastodon account nobody here follows reaches PrivaPub's federated timeline through the relay, and a -# persona's public post goes to the relay (owner decision 2026-10-06), nothing less public. Mastodon leaves each relay +# persona's public post goes to the relay (owner decision 2026-10-06), nothing less public, and reaches Mastodon through +# it: forwarded on its FEP-8b32 proof, or announced. Mastodon leaves each relay # after, so the town sees no relayed posts. Needs the relay, aoderelay and mastodon peers. M=https://mastodon.test:6443 mcurl() { curl -sk --resolve mastodon.test:6443:127.0.0.1 "$@"; } @@ -38,10 +39,16 @@ until_true 60 'p_public_has "$s_uri"' && ok "a public post of an account nobody && ok "and no one's home" || ko "the relayed post landed in alice's home" # relay_creates : the Creates PrivaPub has queued for relay.test naming the text relay_creates() { podman exec pasture-mongo mongosh --quiet PrivaPub --eval 'print(db.Job.countDocuments({Host:"relay.test", Payload:/Create/, Payload:/'"$1"'/}))'; } -curl -s -o /dev/null -X POST -H "$PH" "$P/api/v1/statuses" -d "status=a PrivaPub post for the relay $run&visibility=public" +# (Mastodon reads a known account's keys again at most daily: made to read alice's anew, with her Ed25519 key) +alice_uri="$(curl -s -H "$PH" "$P/api/v1/accounts/verify_credentials" | j "print(d['url'])" | sed 's|/@|/peasants/|')" +podman exec pasture-mastodon bin/rails runner "Account.where(uri: \"$alice_uri\").update_all(last_webfingered_at: nil)" >/dev/null 2>&1 +r_post=$(curl -s -X POST -H "$PH" "$P/api/v1/statuses" -d "status=a PrivaPub post for the relay $run&visibility=public" | j "print(d['uri'])") curl -s -o /dev/null -X POST -H "$PH" "$P/api/v1/statuses" -d "status=a quiet PrivaPub post $run&visibility=unlisted" until_true 30 '[ "$(relay_creates "a PrivaPub post for the relay $run")" = "1" ]' && ok "alice's public post goes to the relay" || ko "PrivaPub never sent the relay alice's public post" [ "$(relay_creates "a quiet PrivaPub post $run")" = "0" ] && ok "and nothing less public" || ko "PrivaPub sent the relay an unlisted post" +# Activity-Relay forwards her Create as she sent it, signed by the relay: Mastodon takes it on its FEP-8b32 proof +until_true 60 '[ "$(podman exec pasture-mastodon bin/rails runner "puts Status.exists?(uri: \"$r_post\")" 2>/dev/null | tail -1)" = "true" ]' \ + && ok "alice's public post reaches Mastodon through Activity-Relay, on its proof" || ko "Mastodon dropped alice's post forwarded by Activity-Relay" m_unrelay relay.test until_true 45 '! relay_subscribers | grep -qx mastodon.test' && ok "Mastodon leaves Activity-Relay" || ko "Mastodon is still subscribed to Activity-Relay"