From 6f1ab0073c0fa269245b641e40486162c4e91f02 Mon Sep 17 00:00:00 2001 From: thepra Date: Thu, 1 Oct 2026 18:53:28 +0200 Subject: [PATCH] P6: quote posts, received and sent (FEP-044f and the older keys) - Received quotes: read from `quote`, `quoteUrl`, `quoteUri`, `_misskey_quote` or a FEP-e232 Link tag; the quoted post is fetched once; a quoteAuthorization stamp is verified field by field on the quoted author's origin; a consent quote without a stamp is pending; an older-key quote of a public post is shown; Delete of a stamp revokes. Counts and a `quote` notification follow the accepted state. The `quote-inline` fallback survives sanitising and is removed from content when the real quote is shown. - Personas quote through `quoted_status_id`: posts that state a quote policy get a QuoteRequest and stay pending until an Accept brings a stamp we can verify, then an Update adds quoteAuthorization; posts that state none are quoted the older way, without `quote`; another persona's posts cannot be quoted yet (we issue no stamps). Quoting posts are delivered to the quoted author too. - Mastodon API: Status.quote (with the quoted status one level deep), quotes_count, quote_approval from the remote policy, GET /api/v1/statuses/:id/quotes, `quote` notifications, and api_versions.mastodon = 7. Checked live: GoToSocial's author-only quote policy is respected. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_012CzABvBkbcFqoHdmi8b9WB --- CLAUDE.md | 8 + FEDERATION.md | 15 ++ .../Federation/InboxScenarioTests.cs | 8 +- PrivaPub.Tests/Federation/QuoteTests.cs | 159 ++++++++++++++ PrivaPub.Tests/Support/Harness.cs | 16 +- .../Controllers/InstanceController.cs | 1 + .../Controllers/StatusesController.cs | 16 ++ .../Controllers/TimelinesController.cs | 3 +- PrivaPub/Api/Mastodon/Entities/Entities.cs | 17 ++ .../Api/Mastodon/Mappers/MastodonMapper.cs | 60 +++++- PrivaPub/Domain/Statuses/PollService.cs | 21 +- PrivaPub/Domain/Statuses/QuoteService.cs | 196 ++++++++++++++++++ PrivaPub/Domain/Statuses/StatusService.cs | 27 ++- .../Inbox/Handlers/AcceptHandler.cs | 9 +- .../Inbox/Handlers/CreateHandler.cs | 5 +- .../Inbox/Handlers/DeleteHandler.cs | 8 +- .../Inbox/Handlers/UpdateHandler.cs | 14 +- PrivaPub/Federation/Inbox/RemotePosts.cs | 1 + .../Federation/Objects/ContentSanitizer.cs | 2 +- PrivaPub/Federation/Objects/NoteParser.cs | 16 +- PrivaPub/Federation/Objects/ObjectShapes.cs | 18 ++ PrivaPub/Federation/Outbox/OutboxPublisher.cs | 10 + .../Rendering/ActivityPubRenderer.cs | 52 +++++ .../Middleware/SocialPubConfigurations.cs | 1 + PrivaPub/Models/Post/Post.cs | 18 ++ PrivaPub/Models/Post/PostDetails.cs | 6 + PrivaPub/Models/Social/Notification.cs | 3 +- docs/ROADMAP.md | 8 +- tools/pasture/interop.sh | 5 + 29 files changed, 678 insertions(+), 45 deletions(-) create mode 100644 PrivaPub.Tests/Federation/QuoteTests.cs create mode 100644 PrivaPub/Domain/Statuses/QuoteService.cs diff --git a/CLAUDE.md b/CLAUDE.md index 1c24720..6d4ffc8 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -261,6 +261,14 @@ cd /var/www/privapub.thepra.dev && sudo -u www-data ASPNETCORE_ENVIRONMENT=Produ - **Link previews follow owner decision 1** (`Domain/Content/LinkPreviews.cs`): only public posts, queued on arrival with 0–60 s of jitter, one cached `LinkPreview` per address for the whole server, never fetched when someone reads. `LinkPreviews.Wanted` is called where posts are saved (`CreateHandler`, `StoreContext`, `StatusService.Publish`). +- **Quote states come from `QuoteService.Resolve`:** + - with a stamp: accepted only if `Verified` (fetched, on the quoted author's origin, naming both posts exactly); + - a FEP-044f `quote` without a stamp: pending; + - older keys only: accepted when the quoted post is public. + + `QuotesCount` moves with the accepted state, never with the raw key. When a persona quotes, `QuotePermission` decides: + asking first for posts that state a policy, quoting at once for posts that state none, refusing otherwise. Only `quote` + of a post that asks for consent puts `quote` in our JSON; older-key quotes leave it out, as Sharkey learned they must. - **A server is described on arrival, never on read.** The first record from a host enqueues `DescribeInstance` (its NodeInfo, at most once a week, into `RemoteInstance`), so opening the details view tells nobody anything. - **Post ids are the timeline order, so they follow arrival, not `published`.** `PrivacyIds.Arrived` gives a remote post diff --git a/FEDERATION.md b/FEDERATION.md index ef66ac3..9ceaabe 100644 --- a/FEDERATION.md +++ b/FEDERATION.md @@ -117,6 +117,21 @@ content, one per choice, counted once per voter. Counts are refreshed with an `U minutes. Our own votes on other servers' polls are sent the same way to the poll's author only, without `published`. An incoming `Update` without a newer `updated` only refreshes counts and details; it is never recorded as an edit. +**Quotes** (FEP-044f, with the older keys): +- **Received.** A quote is read from `quote`, `quoteUrl`, `quoteUri`, `_misskey_quote` or a FEP-e232 `Link` tag. + - With a `quoteAuthorization`, the stamp is fetched and must be a `QuoteAuthorization` on the quoted author's origin, + attributed to them, naming exactly the two posts. + - A FEP-044f `quote` without a stamp is pending until an `Update` brings one. + - A quote made with the older keys only, by software that asks nobody, is shown when the quoted post is public. + - A `Delete` of a stamp revokes the quote. +- **Sent.** Our accounts quote with every key, plus the `RE:` fallback. + - When the quoted post states an `interactionPolicy.canQuote` that allows us, `quote` is included and a `QuoteRequest` + (with the quoting post as `instrument`) goes to the quoted author. The quote stays pending until their `Accept` brings a + stamp we can verify; the post is then updated with `quoteAuthorization`. + - Posts that state no policy are quoted the older way, without `quote`. + - Quoting posts are also delivered to the quoted author. +- **Not yet.** Our own posts cannot be quoted by others yet: we do not issue stamps. + **Custom emoji** (`Emoji` tags) are read on posts, display names, bios and profile fields, at most 64 per object. **Profiles** keep their header, profile fields, `manuallyApprovesFollowers`, `published`, `movedTo`, `indexable`, `memorial` and avatar and header descriptions. A post's title becomes `name` and is also the first, bold line of `content`, because Mastodon does not show diff --git a/PrivaPub.Tests/Federation/InboxScenarioTests.cs b/PrivaPub.Tests/Federation/InboxScenarioTests.cs index 5cd6c12..10227a8 100644 --- a/PrivaPub.Tests/Federation/InboxScenarioTests.cs +++ b/PrivaPub.Tests/Federation/InboxScenarioTests.cs @@ -52,13 +52,15 @@ namespace PrivaPub.Tests.Federation var db = new DbEntities(); _blocks = new DomainBlocks(NullLogger.Instance); _receiver = new InboxReceiver(_local, remote, queue, _blocks, NullLogger.Instance); + var remotePosts = new RemotePosts(db, _local, remote, _blocks, queue, new ObjectRecords(queue), new NoPreviews()); + var quotes = new QuoteService(db, remote, remotePosts, _local, delivery, new OutboxPublisher(db, _local, delivery)); _processor = new InboxProcessor(remote, new IActivityHandler[] { new FollowHandler(db, _local, remote, delivery), new UndoHandler(db, _local, new Reactions(db, delivery)), - new CreateHandler(db, _local, remote, delivery, _blocks, new Fanout(db), new RemotePosts(db, _local, remote, _blocks, queue, new ObjectRecords(queue), new NoPreviews()), new GroupDistributor(delivery), new ObjectRecords(queue), new PollService(db, _local, delivery, queue), new NoPreviews()), - new DeleteHandler(db, _local, remote, delivery, new GroupDistributor(delivery)), - new UpdateHandler(db, _local, remote, new GroupDistributor(delivery), new ObjectRecords(queue)) + new CreateHandler(db, _local, remote, delivery, _blocks, new Fanout(db), remotePosts, new GroupDistributor(delivery), new ObjectRecords(queue), new PollService(db, _local, delivery, queue), new NoPreviews(), quotes), + new DeleteHandler(db, _local, remote, delivery, new GroupDistributor(delivery), quotes), + new UpdateHandler(db, _local, remote, new GroupDistributor(delivery), new ObjectRecords(queue), quotes) }, NullLogger.Instance); } diff --git a/PrivaPub.Tests/Federation/QuoteTests.cs b/PrivaPub.Tests/Federation/QuoteTests.cs new file mode 100644 index 0000000..8508001 --- /dev/null +++ b/PrivaPub.Tests/Federation/QuoteTests.cs @@ -0,0 +1,159 @@ +using MongoDB.Entities; + +using PrivaPub.ClientModels.Post; +using PrivaPub.Domain.Statuses; +using PrivaPub.Federation.Objects; +using PrivaPub.Models.Post; +using PrivaPub.Tests.Support; + +using System.Text.Json.Nodes; + +namespace PrivaPub.Tests.Federation +{ + [Trait("Category", "Integration")] + public sealed class QuoteTests : IAsyncLifetime + { + Harness _harness; + + public async ValueTask InitializeAsync() + { + Assert.SkipUnless(MongoFixture.Enabled, MongoFixture.Skip); + _harness = await Harness.Start(); + } + + public async ValueTask DisposeAsync() + { + if (_harness != default) + await _harness.DisposeAsync(); + } + + static string Origin(RemoteActor actor) => new Uri(actor.Id).GetLeftPart(UriPartial.Authority); + + static string NewId(RemoteActor actor, string kind) => $"{Origin(actor)}/{kind}/{Guid.NewGuid():N}"; + + async Task Delivered(RemoteActor author, string mentioned, Action shape = default) + { + var note = new JsonObject + { + ["id"] = NewId(author, "notes"), ["type"] = "Note", ["attributedTo"] = author.Id, ["content"] = "

hello

", + ["to"] = new JsonArray(Addressing.Public), ["cc"] = new JsonArray(mentioned), + ["tag"] = new JsonArray(new JsonObject { ["type"] = "Mention", ["href"] = mentioned, ["name"] = "@someone" }) + }; + shape?.Invoke(note); + await _harness.Deliver(author, "/human-centipede", new JsonObject { ["id"] = NewId(author, "activities"), ["type"] = "Create", ["actor"] = author.Id, ["object"] = note }); + return await DB.Default.Find().Match(p => p.ObjectURI == note["id"]!.GetValue()).ExecuteFirstAsync(TestContext.Current.CancellationToken); + } + + string Stamp(RemoteActor quotedAuthor, string quoting, string quoted) + { + var path = $"/stamps/{Guid.NewGuid():N}"; + var id = Origin(quotedAuthor) + path; + _harness.Peer.Serve(path, new JsonObject + { + ["@context"] = "https://www.w3.org/ns/activitystreams", ["id"] = id, ["type"] = "QuoteAuthorization", + ["attributedTo"] = quotedAuthor.Id, ["interactingObject"] = quoting, ["interactionTarget"] = quoted + }.ToJsonString()); + return id; + } + + [Fact] + public async Task A_misskey_quote_of_a_public_post_is_shown_and_counted() + { + var token = TestContext.Current.CancellationToken; + var (_, alice) = await _harness.Persona("alice"); + var ann = new RemoteActor(_harness.Peer, "ann"); + var kitty = new RemoteActor(_harness.Peer, "kitty"); + var original = await Delivered(ann, alice.Uri); + + var quoting = await Delivered(kitty, alice.Uri, n => + { + n["_misskey_quote"] = original.ObjectURI; + n["content"] = $"

look
RE: {original.ObjectURI}

"; + }); + + Assert.Equal(QuoteState.Accepted, quoting.QuoteState); + Assert.Equal(original.ID, quoting.QuotedPostId); + Assert.Equal(1, (await DB.Default.Find().OneAsync(original.ID, token)).QuotesCount); + var status = await new PrivaPub.Api.Mastodon.Mappers.MastodonMapper(_harness.Db, _harness.Local).Status(quoting, alice.Id, token); + Assert.Equal("accepted", status.Quote.State); + Assert.Equal(original.ID, status.Quote.QuotedStatus.Id); + Assert.DoesNotContain("RE:", status.Content); + } + + [Fact] + public async Task A_stamped_quote_is_accepted_a_forged_one_is_not_and_revoking_takes_it_back() + { + var token = TestContext.Current.CancellationToken; + var (_, alice) = await _harness.Persona("alice"); + var ann = new RemoteActor(_harness.Peer, "ann"); + var bob = new RemoteActor(_harness.Peer, "bob"); + var original = await Delivered(ann, alice.Uri); + var quotingId = NewId(bob, "notes"); + var stamp = Stamp(ann, quotingId, original.ObjectURI); + var forged = Stamp(ann, NewId(bob, "notes"), original.ObjectURI); + + var quoting = await Delivered(bob, alice.Uri, n => { n["id"] = quotingId; n["quote"] = original.ObjectURI; n["quoteAuthorization"] = stamp; }); + var cheat = await Delivered(bob, alice.Uri, n => { n["quote"] = original.ObjectURI; n["quoteAuthorization"] = forged; }); + var waiting = await Delivered(bob, alice.Uri, n => n["quote"] = original.ObjectURI); + + Assert.Equal(QuoteState.Accepted, quoting.QuoteState); + Assert.Equal(QuoteState.Unauthorized, cheat.QuoteState); + Assert.Equal(QuoteState.Pending, waiting.QuoteState); + + await _harness.Deliver(ann, "/human-centipede", new JsonObject + { + ["id"] = NewId(ann, "activities"), ["type"] = "Delete", ["actor"] = ann.Id, ["object"] = stamp + }); + Assert.Equal(QuoteState.Revoked, (await DB.Default.Find().OneAsync(quoting.ID, token)).QuoteState); + Assert.Equal(0, (await DB.Default.Find().OneAsync(original.ID, token)).QuotesCount); + } + + [Fact] + public async Task A_persona_asks_before_quoting_and_is_stamped_when_accepted() + { + var token = TestContext.Current.CancellationToken; + var (_, alice) = await _harness.Persona("alice"); + var ann = new RemoteActor(_harness.Peer, "ann"); + var original = await Delivered(ann, alice.Uri, n => n["interactionPolicy"] = new JsonObject + { + ["canQuote"] = new JsonObject { ["automaticApproval"] = new JsonArray(Addressing.Public) } + }); + + var outcome = await _harness.Statuses.Publish(alice, new StatusDraft { Text = "this", QuotedStatusId = original.ID }, token); + Assert.Equal(QuoteState.Pending, outcome.Post.QuoteState); + var request = Assert.Single(await _harness.Outgoing(ann.Id + "/inbox"), a => a["type"]!.GetValue() == "QuoteRequest"); + Assert.Equal(original.ObjectURI, request["object"]!.GetValue()); + Assert.Equal(original.ObjectURI, request["instrument"]!["quote"]!.GetValue()); + + await _harness.Deliver(ann, "/human-centipede", new JsonObject + { + ["id"] = NewId(ann, "activities"), ["type"] = "Accept", ["actor"] = ann.Id, + ["object"] = request["id"]!.GetValue(), ["result"] = Stamp(ann, outcome.Post.ObjectURI, original.ObjectURI) + }); + + var quoting = await DB.Default.Find().OneAsync(outcome.Post.ID, token); + Assert.Equal(QuoteState.Accepted, quoting.QuoteState); + Assert.NotNull(quoting.QuoteAuthorizationURI); + Assert.Contains(await _harness.Outgoing(ann.Id + "/inbox"), a => a["type"]!.GetValue() == "Update" + && a["object"]!["quoteAuthorization"]?.GetValue() == quoting.QuoteAuthorizationURI); + } + + [Fact] + public async Task A_legacy_post_is_quoted_without_asking_and_another_persona_cannot_be_quoted_yet() + { + var token = TestContext.Current.CancellationToken; + var (aliceRoot, alice) = await _harness.Persona("alice"); + var (bobRoot, bob) = await _harness.Persona("bob"); + var kitty = new RemoteActor(_harness.Peer, "kitty"); + var legacy = await Delivered(kitty, alice.Uri); + await _harness.Posts.InsertPost(bobRoot, new InsertPostForm { AvatarId = bob.Id, Text = "mine" }, token); + var bobs = await DB.Default.Find().Match(p => p.GroupUserId == bob.Id).ExecuteFirstAsync(token); + + var outcome = await _harness.Statuses.Publish(alice, new StatusDraft { Text = "nice", QuotedStatusId = legacy.ID }, token); + Assert.Equal(QuoteState.Accepted, outcome.Post.QuoteState); + Assert.False(outcome.Post.QuoteByConsent); + Assert.False((await _harness.Statuses.Publish(alice, new StatusDraft { Text = "and this", QuotedStatusId = bobs.ID }, token)).Ok); + Assert.True((await _harness.Statuses.Publish(bob, new StatusDraft { Text = "me again", QuotedStatusId = bobs.ID }, token)).Ok); + } + } +} diff --git a/PrivaPub.Tests/Support/Harness.cs b/PrivaPub.Tests/Support/Harness.cs index 7a5c280..eb46002 100644 --- a/PrivaPub.Tests/Support/Harness.cs +++ b/PrivaPub.Tests/Support/Harness.cs @@ -47,29 +47,30 @@ namespace PrivaPub.Tests.Support Polls = new PollService(Db, Local, Delivery, Queue); Reactions = new Reactions(Db, Delivery); RemotePosts = new RemotePosts(Db, Local, Remote, new NoBlocks(), Queue, Records, new NoPreviews()); + Outbox = new OutboxPublisher(Db, Local, Delivery); + Quotes = new QuoteService(Db, Remote, RemotePosts, Local, Delivery, Outbox); Receiver = new InboxReceiver(Local, Remote, Queue, new NoBlocks(), NullLogger.Instance); Processor = new InboxProcessor(Remote, new IActivityHandler[] { new FollowHandler(Db, Local, Remote, Delivery), - new AcceptHandler(Db, Local), - new RejectHandler(Db, Local), + new AcceptHandler(Db, Local, Quotes), + new RejectHandler(Db, Local, Quotes), new UndoHandler(Db, Local, Reactions), new LikeHandler(Db, Reactions), new EmojiReactHandler(Db, Reactions), new DislikeHandler(Db), new JoinHandler(Db, Local, Delivery), new AnnounceHandler(Db, Local, RemotePosts, Fanout, Remote), - new CreateHandler(Db, Local, Remote, Delivery, new NoBlocks(), Fanout, RemotePosts, Groups, Records, Polls, new NoPreviews()), - new DeleteHandler(Db, Local, Remote, Delivery, Groups), - new UpdateHandler(Db, Local, Remote, Groups, Records), + new CreateHandler(Db, Local, Remote, Delivery, new NoBlocks(), Fanout, RemotePosts, Groups, Records, Polls, new NoPreviews(), Quotes), + new DeleteHandler(Db, Local, Remote, Delivery, Groups, Quotes), + new UpdateHandler(Db, Local, Remote, Groups, Records, Quotes), new FlagHandler(Db, Local) }, NullLogger.Instance); Follows = new FollowService(Db, Local, Remote, Delivery, new KeyLocalizer(), NullLogger.Instance); Content = new ContentRenderer(Local, Remote); - Outbox = new OutboxPublisher(Db, Local, Delivery); Media = new MediaService(new StaticOptions(new MediaOptions { Root = Path.Combine(Path.GetTempPath(), $"privapub-media-{Guid.NewGuid():N}") }), Local, default, NullLogger.Instance); - Statuses = new StatusService(Db, Local, Remote, Delivery, Content, Outbox, Fanout, Media, Groups, Polls, new NoPreviews()); + Statuses = new StatusService(Db, Local, Remote, Delivery, Content, Outbox, Fanout, Media, Groups, Polls, new NoPreviews(), Quotes); Posts = new PostsService(Db, Local, Statuses, new KeyLocalizer(), NullLogger.Instance); Timelines = new TimelineService(Db, new KeyLocalizer()); Relationships = new RelationshipService(Db, Follows, Delivery); @@ -96,6 +97,7 @@ namespace PrivaPub.Tests.Support public ObjectRecords Records { get; } public PollService Polls { get; } public Reactions Reactions { get; } + public QuoteService Quotes { get; } public TimelineService Timelines { get; } public RelationshipService Relationships { get; } public ReportService Reports { get; } diff --git a/PrivaPub/Api/Mastodon/Controllers/InstanceController.cs b/PrivaPub/Api/Mastodon/Controllers/InstanceController.cs index 4032f9c..f946d9f 100644 --- a/PrivaPub/Api/Mastodon/Controllers/InstanceController.cs +++ b/PrivaPub/Api/Mastodon/Controllers/InstanceController.cs @@ -73,6 +73,7 @@ namespace PrivaPub.Api.Mastodon.Controllers domain = Domain, title = "PrivaPub", version = Version, + api_versions = new { mastodon = 7 }, source_url = "https://git.thepra.dev/thepra/SocialPub", description = Description, usage = new { users = new { active_month = 0 } }, diff --git a/PrivaPub/Api/Mastodon/Controllers/StatusesController.cs b/PrivaPub/Api/Mastodon/Controllers/StatusesController.cs index e2fccbd..299c9da 100644 --- a/PrivaPub/Api/Mastodon/Controllers/StatusesController.cs +++ b/PrivaPub/Api/Mastodon/Controllers/StatusesController.cs @@ -54,6 +54,7 @@ namespace PrivaPub.Api.Mastodon.Controllers InReplyTo = Params.Get("in_reply_to_id"), Language = Params.Get("language") ?? Me.Settings.DefaultLanguage, MediaIds = Params.List("media_ids"), + QuotedStatusId = Params.Get("quoted_status_id"), Poll = Params.Has("poll[options]") ? new PollDraft(Params.List("poll[options]"), Params.Int("poll[expires_in]") ?? 0, Params.Bool("poll[multiple]") == true, Params.Bool("poll[hide_totals]") == true) @@ -226,6 +227,21 @@ namespace PrivaPub.Api.Mastodon.Controllers return Json(reblogs.Select(r => accounts.GetValueOrDefault(MastodonMapper.AuthorOf(r))).Where(a => a != default).ToList()); } + [HttpGet("/api/v1/statuses/{id}/quotes"), Scope("read:statuses", requiresUser: false), Microsoft.AspNetCore.Authorization.AllowAnonymous] + public async Task Quotes(string id, CancellationToken token) + { + if (await Visible(id, token) == default) + return NotFoundError(); + var query = _dbEntities.Posts.Match(p => p.QuotedPostId == id && p.QuoteState == QuoteState.Accepted && !p.DeletedAt.HasValue); + var quotes = await Page.From(Params, Limit()).Fetch(query, p => p.ID, token); + var visible = new List(); + foreach (var quote in quotes) + if (quote.Visibility != PostVisibility.LocalGeo && await VisibilityPolicy.CanSee(quote, MyId, token)) + visible.Add(quote); + Link($"/api/v1/statuses/{id}/quotes", quotes.LastOrDefault()?.ID, quotes.FirstOrDefault()?.ID); + return Json(await _mapper.Statuses(visible, MyId, token)); + } + [HttpPost("/api/v1/statuses/{id}/bookmark"), Scope("write:bookmarks")] public async Task Bookmark(string id, CancellationToken token) { diff --git a/PrivaPub/Api/Mastodon/Controllers/TimelinesController.cs b/PrivaPub/Api/Mastodon/Controllers/TimelinesController.cs index 860f706..e34649a 100644 --- a/PrivaPub/Api/Mastodon/Controllers/TimelinesController.cs +++ b/PrivaPub/Api/Mastodon/Controllers/TimelinesController.cs @@ -164,7 +164,8 @@ namespace PrivaPub.Api.Mastodon.Controllers [NotificationType.Reblog] = "reblog", [NotificationType.Update] = "update", [NotificationType.Poll] = "poll", - [NotificationType.Reaction] = "pleroma:emoji_reaction" + [NotificationType.Reaction] = "pleroma:emoji_reaction", + [NotificationType.Quote] = "quote" }; readonly MastodonMapper _mapper; diff --git a/PrivaPub/Api/Mastodon/Entities/Entities.cs b/PrivaPub/Api/Mastodon/Entities/Entities.cs index 14dfa54..5cd24b8 100644 --- a/PrivaPub/Api/Mastodon/Entities/Entities.cs +++ b/PrivaPub/Api/Mastodon/Entities/Entities.cs @@ -161,6 +161,20 @@ namespace PrivaPub.Api.Mastodon.Entities public int Down { get; set; } } + public class QuoteEntity + { + public string State { get; set; } + public Status QuotedStatus { get; set; } + public string QuotedStatusId { get; set; } + } + + public class QuoteApprovalEntity + { + public List Automatic { get; set; } = new(); + public List Manual { get; set; } = new(); + public string CurrentUser { get; set; } = "denied"; + } + public class EmojiReactionEntity { public string Name { get; set; } @@ -253,6 +267,9 @@ namespace PrivaPub.Api.Mastodon.Entities public PrivaPubStatus Privapub { get; set; } public List EmojiReactions { get; set; } = new(); public PleromaStatus Pleroma { get; set; } + public QuoteEntity Quote { get; set; } + public int QuotesCount { get; set; } + public QuoteApprovalEntity QuoteApproval { get; set; } } public class MediaAttachment diff --git a/PrivaPub/Api/Mastodon/Mappers/MastodonMapper.cs b/PrivaPub/Api/Mastodon/Mappers/MastodonMapper.cs index a7d9728..435d2c3 100644 --- a/PrivaPub/Api/Mastodon/Mappers/MastodonMapper.cs +++ b/PrivaPub/Api/Mastodon/Mappers/MastodonMapper.cs @@ -7,6 +7,8 @@ using PrivaPub.Federation.Actors; using PrivaPub.Federation.Rendering; using PrivaPub.Models.Federation; using PrivaPub.Models.Group; +using PrivaPub.Domain.Privacy; +using PrivaPub.Federation.Objects; using PrivaPub.Models.Post; using PrivaPub.Domain.Statuses; using PrivaPub.Models.Social; @@ -146,7 +148,10 @@ namespace PrivaPub.Api.Mastodon.Mappers public async Task Status(PostEntity post, string viewerId, CancellationToken token) => (await Statuses(new[] { post }, viewerId, token)).FirstOrDefault(); - public async Task> Statuses(IReadOnlyCollection posts, string viewerId, CancellationToken token) + public Task> Statuses(IReadOnlyCollection posts, string viewerId, CancellationToken token) => + Statuses(posts, viewerId, nested: false, token); + + async Task> Statuses(IReadOnlyCollection posts, string viewerId, bool nested, CancellationToken token) { var originalIds = posts.Where(p => p.ReblogOfPostId != default).Select(p => p.ReblogOfPostId).Distinct().ToList(); var originals = originalIds.Count == 0 @@ -233,6 +238,30 @@ namespace PrivaPub.Api.Mastodon.Mappers return status; } + var quotedIds = nested ? new List() : all.Where(p => p.QuoteState == QuoteState.Accepted && p.QuotedPostId != default).Select(p => p.QuotedPostId).Distinct().ToList(); + var quotedPosts = new List(); + foreach (var quotedPost in quotedIds.Count == 0 ? new List() : await _dbEntities.Posts.Match(p => quotedIds.Contains(p.ID) && !p.DeletedAt.HasValue).ExecuteAsync(token)) + if (quotedPost.Visibility != PostVisibility.LocalGeo && await VisibilityPolicy.CanSee(quotedPost, viewerId, token)) + quotedPosts.Add(quotedPost); + var quotedStatuses = quotedPosts.Count == 0 + ? new Dictionary() + : (await Statuses(quotedPosts, viewerId, nested: true, token)).ToDictionary(q => q.Id); + + void Quote(Status status, PostEntity post) + { + status.QuotesCount = post.QuotesCount; + status.QuoteApproval = Approval(post, viewerId); + if (post.QuoteState == QuoteState.None) + return; + var state = post.QuoteState.ToString().ToLowerInvariant(); + var quoted = post.QuotedPostId != default && quotedStatuses.TryGetValue(post.QuotedPostId, out var shown) ? shown : default; + status.Quote = nested + ? new QuoteEntity { State = state, QuotedStatusId = post.QuotedPostId } + : new QuoteEntity { State = state, QuotedStatus = post.QuoteState == QuoteState.Accepted ? quoted : default, QuotedStatusId = post.QuotedPostId }; + if (!nested && post.QuoteState == QuoteState.Accepted && quoted != default) + status.Content = WithoutQuoteFallback(status.Content); + } + var mapped = new List(); foreach (var post in posts) { @@ -242,10 +271,12 @@ namespace PrivaPub.Api.Mastodon.Mappers var status = Map(post); if (status == default) continue; + Quote(status, post); if (post.ReblogOfPostId != default) { if (!originals.TryGetValue(post.ReblogOfPostId, out var original) || Map(original) is not { } inner) continue; + Quote(inner, original); status.Reblog = inner; status.Content = string.Empty; status.Reblogged = reblogged.Contains(original.ID); @@ -257,6 +288,33 @@ namespace PrivaPub.Api.Mastodon.Mappers public static string AuthorOf(PostEntity post) => post.AuthorAccountId ?? post.GroupUserId; + static readonly AngleSharp.Html.Parser.HtmlParser Fragments = new(); + + public static string WithoutQuoteFallback(string html) + { + if (string.IsNullOrEmpty(html) || !html.Contains("quote-inline", StringComparison.Ordinal)) + return html; + var document = Fragments.ParseDocument($"{html}"); + foreach (var fallback in document.QuerySelectorAll(".quote-inline").ToList()) + fallback.Remove(); + return document.Body!.InnerHtml; + } + + static QuoteApprovalEntity Approval(PostEntity post, string viewerId) + { + if (!post.IsFederatedCopy || post.Visibility is not (PostVisibility.Public or PostVisibility.Unlisted)) + return new QuoteApprovalEntity(); + if (post.QuotePolicy is not { } policy) + return new QuoteApprovalEntity { Automatic = new List { "public" }, CurrentUser = viewerId == default ? "unknown" : "automatic" }; + static List Named(IEnumerable who) => who.Select(w => Addressing.IsPublic(w) ? "public" : w.EndsWith("/followers") ? "followers" : "unsupported_policy").Distinct().ToList(); + return new QuoteApprovalEntity + { + Automatic = Named(policy.Automatic), + Manual = Named(policy.Manual), + CurrentUser = viewerId == default ? "unknown" : policy.Automatic.Any(Addressing.IsPublic) ? "automatic" : policy.Manual.Any(Addressing.IsPublic) ? "manual" : "denied" + }; + } + public static string Content(PostEntity post) { var content = post.ContentHtml ?? ActivityPubRenderer.Html(post.Text); diff --git a/PrivaPub/Domain/Statuses/PollService.cs b/PrivaPub/Domain/Statuses/PollService.cs index a6dddda..0994478 100644 --- a/PrivaPub/Domain/Statuses/PollService.cs +++ b/PrivaPub/Domain/Statuses/PollService.cs @@ -233,30 +233,11 @@ namespace PrivaPub.Domain.Statuses return JobOutcome.Done; await Closing(post, author, token); if (!post.IsLocalOnly) - await _outbox.Publish(author, post, Update(post, author), token); + await _outbox.Publish(author, post, ActivityPubRenderer.UpdateOf(post, author, $"poll-{DateTime.UtcNow.Ticks}"), token); return JobOutcome.Done; } protected virtual Task Closing(PostEntity post, LocalActor author, CancellationToken token) => Task.CompletedTask; - - static JsonObject Update(PostEntity post, LocalActor author) - { - var question = post.Visibility == PostVisibility.Direct - ? ActivityPubRenderer.DirectNote(post, author, Array.Empty<(string, string)>(), post.ContextURI) - : ActivityPubRenderer.Note(post, author, default, post.InReplyToURI); - question["to"] = new JsonArray(post.To.Select(t => (JsonNode)t).ToArray()); - question["cc"] = new JsonArray(post.Cc.Select(c => (JsonNode)c).ToArray()); - return new JsonObject - { - ["@context"] = ActivityPubRenderer.Context(), - ["id"] = author.ActivityUri($"update-{post.ID}-poll-{DateTime.UtcNow.Ticks}"), - ["type"] = "Update", - ["actor"] = author.Uri, - ["to"] = question["to"]!.DeepClone(), - ["cc"] = question["cc"]!.DeepClone(), - ["object"] = question - }; - } } public class PollCloseJob : PollRefreshJob diff --git a/PrivaPub/Domain/Statuses/QuoteService.cs b/PrivaPub/Domain/Statuses/QuoteService.cs new file mode 100644 index 0000000..9d82953 --- /dev/null +++ b/PrivaPub/Domain/Statuses/QuoteService.cs @@ -0,0 +1,196 @@ +using MongoDB.Entities; + +using PrivaPub.Domain.Social; +using PrivaPub.Federation.Actors; +using PrivaPub.Federation.Inbox; +using PrivaPub.Federation.Objects; +using PrivaPub.Federation.Outbox; +using PrivaPub.Federation.Rendering; +using PrivaPub.Models.Federation; +using PrivaPub.Models.User; +using PrivaPub.Models.Post; +using PrivaPub.Models.Social; +using PrivaPub.StaticServices; + +using System.Text.Json.Nodes; + +using static PrivaPub.Federation.Objects.ActivityJson; + +using PostEntity = PrivaPub.Models.Post.Post; + +namespace PrivaPub.Domain.Statuses +{ + public interface IQuoteService + { + Task Resolve(PostEntity post, NoteDocument note, CancellationToken token); + Task Revoke(string stampUri, CancellationToken token); + Task Verified(string stampUri, string quotingUri, PostEntity quoted, CancellationToken token); + QuotePermission Permission(PostEntity quoted, LocalActor author); + Task Request(LocalActor author, PostEntity post, PostEntity quoted, JsonObject note, CancellationToken token); + Task Answered(JsonNode answer, ForeignAvatar actor, bool accepted, CancellationToken token); + } + + public enum QuotePermission + { + Denied, + Granted, + AskFirst + } + + public class QuoteService : IQuoteService + { + readonly DbEntities _dbEntities; + readonly IRemoteActorService _remoteActors; + readonly IRemotePosts _remotePosts; + readonly ILocalActorService _localActors; + readonly IDeliveryService _delivery; + readonly IOutboxPublisher _outbox; + + public QuoteService(DbEntities dbEntities, IRemoteActorService remoteActors, IRemotePosts remotePosts, ILocalActorService localActors, + IDeliveryService delivery, IOutboxPublisher outbox) + { + _dbEntities = dbEntities; + _remoteActors = remoteActors; + _remotePosts = remotePosts; + _localActors = localActors; + _delivery = delivery; + _outbox = outbox; + } + + const string RequestPrefix = "quote-request-"; + + public QuotePermission Permission(PostEntity quoted, LocalActor author) + { + if (!quoted.IsFederatedCopy) + return quoted.GroupUserId == author.Id && quoted.Visibility is PostVisibility.Public or PostVisibility.Unlisted + ? QuotePermission.Granted + : QuotePermission.Denied; + if (quoted.Visibility is not (PostVisibility.Public or PostVisibility.Unlisted)) + return QuotePermission.Denied; + if (quoted.QuotePolicy is not { } policy) + return QuotePermission.Granted; + return policy.Automatic.Concat(policy.Manual).Any(Addressing.IsPublic) || policy.Automatic.Concat(policy.Manual).Contains(author.Uri) + ? QuotePermission.AskFirst + : QuotePermission.Denied; + } + + public async Task Request(LocalActor author, PostEntity post, PostEntity quoted, JsonObject note, CancellationToken token) + { + var owner = await _dbEntities.ForeignAvatars.Match(f => f.ActorURI == quoted.ActorURI).ExecuteFirstAsync(token); + if (string.IsNullOrEmpty(owner?.InboxURL)) + return; + await _delivery.Enqueue(author, new[] { owner.InboxURL }, new JsonObject + { + ["@context"] = ActivityPubRenderer.Context(), + ["id"] = author.ActivityUri(RequestPrefix + post.ID), + ["type"] = "QuoteRequest", + ["actor"] = author.Uri, + ["object"] = quoted.ObjectURI, + ["instrument"] = note.DeepClone(), + ["to"] = new JsonArray(quoted.ActorURI) + }, token); + } + + public async Task Answered(JsonNode answer, ForeignAvatar actor, bool accepted, CancellationToken token) + { + var request = answer["object"]; + var requestId = Id(request); + var marker = requestId?.LastIndexOf("/grunts/" + RequestPrefix, StringComparison.Ordinal) ?? -1; + if (marker < 0 && !(request is JsonObject && Value(request, "type") == "QuoteRequest")) + return false; + if (marker < 0) + return true; + var postId = requestId[(marker + "/grunts/".Length + RequestPrefix.Length)..]; + var post = await _dbEntities.Posts.Match(p => p.ID == postId && !p.IsFederatedCopy && !p.DeletedAt.HasValue).ExecuteFirstAsync(token); + var author = post == default ? default : await _localActors.FindById(LocalActorKind.Person, post.GroupUserId, token); + if (author == default || author.ActivityUri(RequestPrefix + post.ID) != requestId || post.QuoteState != QuoteState.Pending) + return true; + var quoted = await _dbEntities.Posts.MatchID(post.QuotedPostId).ExecuteFirstAsync(token); + if (quoted?.ActorURI != actor.ActorURI) + return true; + if (!accepted) + { + await DB.Default.Update().MatchID(post.ID).Modify(p => p.QuoteState, QuoteState.Rejected).ExecuteAsync(token); + return true; + } + var stamp = Id(answer["result"]); + if (stamp == default || !await Verified(stamp, post.ObjectURI, quoted, token)) + return true; + post.QuoteAuthorizationURI = stamp; + post.QuoteState = QuoteState.Accepted; + await DB.Default.Update().MatchID(post.ID) + .Modify(p => p.QuoteAuthorizationURI, stamp) + .Modify(p => p.QuoteState, QuoteState.Accepted) + .ExecuteAsync(token); + await DB.Default.Update().MatchID(quoted.ID).Modify(b => b.Inc(p => p.QuotesCount, 1)).ExecuteAsync(token); + if (!post.IsLocalOnly) + await _outbox.Publish(author, post, ActivityPubRenderer.UpdateOf(post, author, "quote-approved"), token); + return true; + } + + public async Task Resolve(PostEntity post, NoteDocument note, CancellationToken token) + { + if (note.QuoteUri == default && !note.QuoteDeleted) + return; + var quoted = note.QuoteUri == default + ? default + : await _dbEntities.Posts.Match(p => p.ObjectURI == note.QuoteUri && !p.DeletedAt.HasValue).ExecuteFirstAsync(token) + ?? await _remotePosts.StoreContext(note.QuoteUri, RemotePosts.MaxDepth, token); + + var state = note.QuoteDeleted ? QuoteState.Deleted + : quoted == default ? QuoteState.Pending + : note.QuoteAuthorization != default + ? !quoted.IsFederatedCopy || !await Verified(note.QuoteAuthorization, post.ObjectURI, quoted, token) ? QuoteState.Unauthorized : QuoteState.Accepted + : note.QuotesByConsent ? QuoteState.Pending + : quoted.Visibility is PostVisibility.Public or PostVisibility.Unlisted ? QuoteState.Accepted + : QuoteState.Unauthorized; + + var wasCounted = post.QuoteState == QuoteState.Accepted ? post.QuotedPostId : default; + await DB.Default.Update().MatchID(post.ID) + .Modify(p => p.QuoteURI, note.QuoteUri) + .Modify(p => p.QuotedPostId, quoted?.ID) + .Modify(p => p.QuoteState, state) + .Modify(p => p.QuoteAuthorizationURI, state == QuoteState.Accepted ? note.QuoteAuthorization : default) + .ExecuteAsync(token); + post.QuotedPostId = quoted?.ID; + post.QuoteState = state; + + var nowCounted = state == QuoteState.Accepted ? quoted?.ID : default; + if (wasCounted == nowCounted) + return; + if (wasCounted != default) + await DB.Default.Update().MatchID(wasCounted).Modify(b => b.Inc(p => p.QuotesCount, -1)).ExecuteAsync(token); + if (nowCounted != default) + { + await DB.Default.Update().MatchID(nowCounted).Modify(b => b.Inc(p => p.QuotesCount, 1)).ExecuteAsync(token); + if (!quoted.IsFederatedCopy) + await Notifications.Add(quoted.GroupUserId, NotificationType.Quote, post.AuthorAccountId, post.ActorURI, post.ID, token); + } + } + + public async Task Revoke(string stampUri, CancellationToken token) + { + var revoked = await _dbEntities.Posts.Match(p => p.QuoteAuthorizationURI == stampUri && p.QuoteState == QuoteState.Accepted).ExecuteAsync(token); + foreach (var post in revoked) + { + await DB.Default.Update().MatchID(post.ID).Modify(p => p.QuoteState, QuoteState.Revoked).ExecuteAsync(token); + if (post.QuotedPostId != default) + await DB.Default.Update().MatchID(post.QuotedPostId).Modify(b => b.Inc(p => p.QuotesCount, -1)).ExecuteAsync(token); + } + } + + public async Task Verified(string stampUri, string quotingUri, PostEntity quoted, CancellationToken token) + { + if (!Origin.Same(stampUri, quoted.ActorURI)) + return false; + using var fetched = await _remoteActors.FetchObject(stampUri, token); + if (fetched == default) + return false; + var stamp = JsonNode.Parse(fetched.Root.GetRawText()); + return Value(stamp, "type") == "QuoteAuthorization" + && Id(stamp["attributedTo"]) == quoted.ActorURI + && Id(stamp["interactingObject"]) == quotingUri + && Id(stamp["interactionTarget"]) == quoted.ObjectURI; + } + } +} diff --git a/PrivaPub/Domain/Statuses/StatusService.cs b/PrivaPub/Domain/Statuses/StatusService.cs index c251712..08acfb2 100644 --- a/PrivaPub/Domain/Statuses/StatusService.cs +++ b/PrivaPub/Domain/Statuses/StatusService.cs @@ -40,6 +40,7 @@ namespace PrivaPub.Domain.Statuses public double? Longitude { get; init; } public double? RangeKm { get; init; } public PollDraft Poll { get; init; } + public string QuotedStatusId { get; init; } } public sealed record StatusOutcome(PostEntity Post, int Status = StatusCodes.Status200OK, string Error = default) @@ -72,12 +73,14 @@ namespace PrivaPub.Domain.Statuses readonly IGroupDistributor _groups; readonly IPollService _polls; readonly ILinkPreviews _previews; + readonly IQuoteService _quotes; public StatusService(DbEntities dbEntities, ILocalActorService localActors, IRemoteActorService remoteActors, IDeliveryService delivery, IContentRenderer content, IOutboxPublisher outbox, IFanout fanout, IMediaService media, IGroupDistributor groups, IPollService polls, - ILinkPreviews previews) + ILinkPreviews previews, IQuoteService quotes) { _previews = previews; + _quotes = quotes; _polls = polls; _media = media; _groups = groups; @@ -115,6 +118,18 @@ namespace PrivaPub.Domain.Statuses if (parent != default && !await VisibilityPolicy.CanSee(parent, author.Id, token)) return StatusOutcome.Fail(StatusCodes.Status404NotFound, "Record not found"); + PostEntity quoted = default; + var quotePermission = QuotePermission.Denied; + if (!string.IsNullOrEmpty(draft.QuotedStatusId)) + { + quoted = await _dbEntities.Posts.Match(p => p.ID == draft.QuotedStatusId && !p.DeletedAt.HasValue && p.ReblogOfPostId == null).ExecuteFirstAsync(token); + if (quoted == default || !await VisibilityPolicy.CanSee(quoted, author.Id, token)) + return StatusOutcome.Fail(StatusCodes.Status404NotFound, "Record not found"); + quotePermission = _quotes.Permission(quoted, author); + if (quotePermission == QuotePermission.Denied) + return StatusOutcome.Fail(StatusCodes.Status422UnprocessableEntity, "Validation failed: This post cannot be quoted"); + } + var located = draft.Latitude.HasValue || draft.Longitude.HasValue; if (located && (draft.Latitude is not (>= -90 and <= 90) || draft.Longitude is not (>= -180 and <= 180) || group != default || draft.Visibility == PostVisibility.Direct)) @@ -166,7 +181,11 @@ namespace PrivaPub.Domain.Statuses InReplyToAccountId = parent?.AuthorAccountId ?? parent?.GroupUserId, IsLocalOnly = isLocalOnly, ActorURI = author.Uri, - Poll = draft.Poll == default ? default : _polls.Create(draft.Poll) + Poll = draft.Poll == default ? default : _polls.Create(draft.Poll), + QuoteURI = quoted?.ObjectURI, + QuotedPostId = quoted?.ID, + QuoteByConsent = quoted != default && (quoted.QuotePolicy != default || !quoted.IsFederatedCopy), + QuoteState = quoted == default ? QuoteState.None : quotePermission == QuotePermission.Granted ? QuoteState.Accepted : QuoteState.Pending }; post.ID = (string)post.GenerateNewID(); post.ObjectURI = author.PostUri(post.ID); @@ -210,6 +229,10 @@ namespace PrivaPub.Domain.Statuses await DB.Default.Update().MatchID(parent.ID).Modify(b => b.Inc(p => p.RepliesCount, 1)).ExecuteAsync(token); await _fanout.Distribute(post, token); await _polls.Scheduled(post, token); + if (post.QuoteState == QuoteState.Accepted) + await DB.Default.Update().MatchID(quoted.ID).Modify(b => b.Inc(p => p.QuotesCount, 1)).ExecuteAsync(token); + if (post.QuoteState == QuoteState.Pending && create?["object"] is JsonObject quotingNote) + await _quotes.Request(author, post, quoted, quotingNote, token); await _previews.Wanted(post, token); if (create != default) await _outbox.Publish(author, post, create, token); diff --git a/PrivaPub/Federation/Inbox/Handlers/AcceptHandler.cs b/PrivaPub/Federation/Inbox/Handlers/AcceptHandler.cs index 7a971b0..6f71b86 100644 --- a/PrivaPub/Federation/Inbox/Handlers/AcceptHandler.cs +++ b/PrivaPub/Federation/Inbox/Handlers/AcceptHandler.cs @@ -1,5 +1,6 @@ using MongoDB.Entities; +using PrivaPub.Domain.Statuses; using PrivaPub.Federation.Actors; using PrivaPub.Models.Social; using PrivaPub.Models.User; @@ -15,17 +16,21 @@ namespace PrivaPub.Federation.Inbox.Handlers { readonly DbEntities _dbEntities; readonly ILocalActorService _localActors; + readonly IQuoteService _quotes; - public AcceptHandler(DbEntities dbEntities, ILocalActorService localActors) + public AcceptHandler(DbEntities dbEntities, ILocalActorService localActors, IQuoteService quotes) { _dbEntities = dbEntities; _localActors = localActors; + _quotes = quotes; } public virtual string Type => "Accept"; public async Task Handle(JsonNode activity, ForeignAvatar actor, CancellationToken token) { + if (await _quotes.Answered(activity, actor, accepted: Type == "Accept", token)) + return; var following = await FindFollowing(activity["object"], actor, _dbEntities, _localActors, token); if (following == default) return; @@ -56,7 +61,7 @@ namespace PrivaPub.Federation.Inbox.Handlers public class RejectHandler : AcceptHandler { - public RejectHandler(DbEntities dbEntities, ILocalActorService localActors) : base(dbEntities, localActors) + public RejectHandler(DbEntities dbEntities, ILocalActorService localActors, IQuoteService quotes) : base(dbEntities, localActors, quotes) { } diff --git a/PrivaPub/Federation/Inbox/Handlers/CreateHandler.cs b/PrivaPub/Federation/Inbox/Handlers/CreateHandler.cs index d03fc29..1ee978c 100644 --- a/PrivaPub/Federation/Inbox/Handlers/CreateHandler.cs +++ b/PrivaPub/Federation/Inbox/Handlers/CreateHandler.cs @@ -39,10 +39,12 @@ namespace PrivaPub.Federation.Inbox.Handlers readonly IObjectRecords _records; readonly IPollService _polls; readonly ILinkPreviews _previews; + readonly IQuoteService _quotes; public CreateHandler(DbEntities dbEntities, ILocalActorService localActors, IRemoteActorService remoteActors, IDeliveryService delivery, - IDomainBlocks domainBlocks, IFanout fanout, IRemotePosts remotePosts, IGroupDistributor groups, IObjectRecords records, IPollService polls, ILinkPreviews previews) + IDomainBlocks domainBlocks, IFanout fanout, IRemotePosts remotePosts, IGroupDistributor groups, IObjectRecords records, IPollService polls, ILinkPreviews previews, IQuoteService quotes) { + _quotes = quotes; _previews = previews; _records = records; _polls = polls; @@ -152,6 +154,7 @@ namespace PrivaPub.Federation.Inbox.Handlers } await _records.Record(note, post, ObjectPath.Delivered, refetched, token); await _previews.Wanted(post, token); + await _quotes.Resolve(post, note, token); if (parent != default) await DB.Default.Update().MatchID(parent.ID).Modify(b => b.Inc(p => p.RepliesCount, 1)).ExecuteAsync(token); diff --git a/PrivaPub/Federation/Inbox/Handlers/DeleteHandler.cs b/PrivaPub/Federation/Inbox/Handlers/DeleteHandler.cs index 46e8bcc..b2f2d98 100644 --- a/PrivaPub/Federation/Inbox/Handlers/DeleteHandler.cs +++ b/PrivaPub/Federation/Inbox/Handlers/DeleteHandler.cs @@ -1,5 +1,6 @@ using MongoDB.Entities; +using PrivaPub.Domain.Statuses; using PrivaPub.Federation.Actors; using PrivaPub.Federation.Objects; using PrivaPub.Federation.Outbox; @@ -27,10 +28,12 @@ namespace PrivaPub.Federation.Inbox.Handlers readonly IRemoteActorService _remoteActors; readonly IDeliveryService _delivery; readonly IGroupDistributor _groups; + readonly IQuoteService _quotes; public DeleteHandler(DbEntities dbEntities, ILocalActorService localActors, IRemoteActorService remoteActors, IDeliveryService delivery, - IGroupDistributor groups) + IGroupDistributor groups, IQuoteService quotes) { + _quotes = quotes; _groups = groups; _dbEntities = dbEntities; _localActors = localActors; @@ -72,7 +75,10 @@ namespace PrivaPub.Federation.Inbox.Handlers .ExecuteAsync(token); var post = await _dbEntities.Posts.Match(p => p.ObjectURI == objectUri && p.ActorURI == actor.ActorURI).ExecuteFirstAsync(token); if (post == default) + { + await _quotes.Revoke(objectUri, token); return; + } await DB.Default.DeleteAsync(post.ID); await DB.Default.DeleteAsync(r => r.PostId == post.ID || r.ObjectURI == objectUri); await DB.Default.DeleteAsync(e => e.PostId == post.ID || e.ReblogOfPostId == post.ID); diff --git a/PrivaPub/Federation/Inbox/Handlers/UpdateHandler.cs b/PrivaPub/Federation/Inbox/Handlers/UpdateHandler.cs index 76737db..a248392 100644 --- a/PrivaPub/Federation/Inbox/Handlers/UpdateHandler.cs +++ b/PrivaPub/Federation/Inbox/Handlers/UpdateHandler.cs @@ -1,6 +1,7 @@ using MongoDB.Entities; using PrivaPub.Federation.Actors; +using PrivaPub.Domain.Statuses; using PrivaPub.Federation.Objects; using PrivaPub.Federation.Outbox; using PrivaPub.Models.Post; @@ -24,10 +25,12 @@ namespace PrivaPub.Federation.Inbox.Handlers readonly IRemoteActorService _remoteActors; readonly IGroupDistributor _groups; readonly IObjectRecords _records; + readonly IQuoteService _quotes; public UpdateHandler(DbEntities dbEntities, ILocalActorService localActors, IRemoteActorService remoteActors, IGroupDistributor groups, - IObjectRecords records) + IObjectRecords records, IQuoteService quotes) { + _quotes = quotes; _records = records; _groups = groups; _dbEntities = dbEntities; @@ -56,6 +59,7 @@ namespace PrivaPub.Federation.Inbox.Handlers return; post.Poll = note.Poll ?? post.Poll; + post.QuotePolicy = note.QuotePolicy; post.Video = note.Video ?? post.Video; post.Audio = note.Audio ?? post.Audio; post.Event = note.Event ?? post.Event; @@ -63,6 +67,7 @@ namespace PrivaPub.Federation.Inbox.Handlers { await DB.Default.SaveAsync(post, token); await _records.Revise(note, Id(activity), token); + await Requote(post, note, token); return; } @@ -102,11 +107,18 @@ namespace PrivaPub.Federation.Inbox.Handlers post.UpdateDate = DateTime.UtcNow; await DB.Default.SaveAsync(post, token); await _records.Revise(note, Id(activity), token); + await Requote(post, note, token); if (!string.IsNullOrEmpty(post.GroupId) && await _localActors.FindById(Models.Federation.LocalActorKind.Group, post.GroupId, token) is { IsCircle: false } community) await _groups.Announce(community, activity.AsObject(), post.ObjectURI, isNewPost: false, token); } static bool IsEdit(NoteDocument note, PostEntity post) => note.Updated is { } updated && updated > (post.EditedAt ?? post.CreationDate); + + async Task Requote(PostEntity post, NoteDocument note, CancellationToken token) + { + if (note.QuoteUri != post.QuoteURI || note.QuoteAuthorization != post.QuoteAuthorizationURI || post.QuoteState != QuoteState.Accepted) + await _quotes.Resolve(post, note, token); + } } } diff --git a/PrivaPub/Federation/Inbox/RemotePosts.cs b/PrivaPub/Federation/Inbox/RemotePosts.cs index ff2c184..db67008 100644 --- a/PrivaPub/Federation/Inbox/RemotePosts.cs +++ b/PrivaPub/Federation/Inbox/RemotePosts.cs @@ -82,6 +82,7 @@ namespace PrivaPub.Federation.Inbox Excerpt = note.Excerpt, Source = note.Source, Poll = note.Poll, + QuotePolicy = note.QuotePolicy, Emojis = note.Emojis.ToList(), CoverURL = note.CoverURL, Link = note.Link, diff --git a/PrivaPub/Federation/Objects/ContentSanitizer.cs b/PrivaPub/Federation/Objects/ContentSanitizer.cs index df53ec0..e713bca 100644 --- a/PrivaPub/Federation/Objects/ContentSanitizer.cs +++ b/PrivaPub/Federation/Objects/ContentSanitizer.cs @@ -38,7 +38,7 @@ namespace PrivaPub.Federation.Objects { KeepChildNodes = true }; - foreach (var allowed in new[] { "mention", "hashtag", "ellipsis", "invisible" }) + foreach (var allowed in new[] { "mention", "hashtag", "ellipsis", "invisible", "quote-inline" }) sanitizer.AllowedClasses.Add(allowed); sanitizer.RemovingCssClass += (_, e) => e.Cancel = MicroformatClass().IsMatch(e.CssClass); sanitizer.RemovingTag += (_, e) => diff --git a/PrivaPub/Federation/Objects/NoteParser.cs b/PrivaPub/Federation/Objects/NoteParser.cs index 2590933..04f6b11 100644 --- a/PrivaPub/Federation/Objects/NoteParser.cs +++ b/PrivaPub/Federation/Objects/NoteParser.cs @@ -25,6 +25,9 @@ namespace PrivaPub.Federation.Objects public string Context { get; init; } public string Audience { get; init; } public string QuoteUri { get; init; } + public string QuoteAuthorization { get; init; } + public bool QuotesByConsent { get; init; }//FEP-044f `quote`, as opposed to the legacy keys that ask nobody + public bool QuoteDeleted { get; init; } public DateTime Published { get; init; } public DateTime? Updated { get; init; } public IReadOnlyList To { get; init; } = Array.Empty(); @@ -34,6 +37,7 @@ namespace PrivaPub.Federation.Objects public IReadOnlyList Attachments { get; init; } = Array.Empty(); public PostSource Source { get; init; } public PostPoll Poll { get; init; } + public InteractionRule QuotePolicy { get; init; } public IReadOnlyList Emojis { get; init; } = Array.Empty(); public string CoverURL { get; init; } public PostLink Link { get; init; } @@ -79,7 +83,10 @@ namespace PrivaPub.Federation.Objects InReplyTo = Id(note["inReplyTo"]), Context = Id(note["context"]) ?? Value(note, "conversation"), Audience = Id(note["audience"]), - QuoteUri = Value(note, "quote") ?? Value(note, "quoteUrl") ?? Value(note, "quoteUri") ?? Value(note, "_misskey_quote"), + QuoteUri = Id(note["quote"]) ?? Value(note, "quoteUrl") ?? Value(note, "quoteUri") ?? Value(note, "_misskey_quote") ?? QuoteLink(note), + QuoteAuthorization = Id(note["quoteAuthorization"]), + QuotesByConsent = note.ContainsKey("quote"), + QuoteDeleted = note["quote"] is JsonObject quoted && Value(quoted, "type") == "Tombstone", Published = Time(Value(note, "published")) ?? DateTime.UtcNow, Updated = Time(Value(note, "updated")), To = List(note["to"]), @@ -97,6 +104,7 @@ namespace PrivaPub.Federation.Objects Attachments = Attachments(note), Source = ObjectShapes.Source(note), Poll = ObjectShapes.Poll(note), + QuotePolicy = ObjectShapes.QuotePolicy(note), Emojis = ObjectShapes.Emojis(note["tag"]), CoverURL = ObjectShapes.Cover(note), Link = ObjectShapes.Link(note), @@ -169,6 +177,12 @@ namespace PrivaPub.Federation.Objects }; } + static string QuoteLink(JsonObject note) => + Tags(note, "Link") + .Where(t => Value(t, "mediaType") is { } type && (type.StartsWith("application/activity+json") || type.StartsWith("application/ld+json"))) + .Select(t => Value(t, "href")) + .FirstOrDefault(href => Origin.Of(href) != default); + static string FirstOf(JsonNode map) => map is JsonObject languages ? languages.Select(l => l.Value is JsonValue v && v.TryGetValue(out var text) ? text : default).FirstOrDefault(t => t != default) : default; diff --git a/PrivaPub/Federation/Objects/ObjectShapes.cs b/PrivaPub/Federation/Objects/ObjectShapes.cs index de478f6..feec14c 100644 --- a/PrivaPub/Federation/Objects/ObjectShapes.cs +++ b/PrivaPub/Federation/Objects/ObjectShapes.cs @@ -18,6 +18,7 @@ namespace PrivaPub.Federation.Objects const int CoverMaxWidth = 1280; const int MaxEmojis = 64; const int MaxPollOptions = 20; + const int MaxAudience = 32; static readonly Dictionary MediaTypesByExtension = new(StringComparer.OrdinalIgnoreCase) { @@ -97,6 +98,23 @@ namespace PrivaPub.Federation.Objects .Take(MaxEmojis) .ToList(); + public static InteractionRule QuotePolicy(JsonObject note) + { + if (note["interactionPolicy"] is not JsonObject policy || policy["canQuote"] is not JsonObject canQuote) + return default; + static List Who(JsonNode node) => node switch + { + JsonArray array => array.Select(Id).Where(id => id != default).Take(MaxAudience).ToList(), + JsonNode single when Id(single) is { } id => new List { id }, + _ => new List() + }; + return new InteractionRule + { + Automatic = Who(canQuote["automaticApproval"] ?? canQuote["always"]), + Manual = Who(canQuote["manualApproval"] ?? canQuote["approvalRequired"]) + }; + } + public static PostPoll Poll(JsonObject note) { if (Value(note, "type") != "Question") diff --git a/PrivaPub/Federation/Outbox/OutboxPublisher.cs b/PrivaPub/Federation/Outbox/OutboxPublisher.cs index e22c917..2354882 100644 --- a/PrivaPub/Federation/Outbox/OutboxPublisher.cs +++ b/PrivaPub/Federation/Outbox/OutboxPublisher.cs @@ -60,6 +60,16 @@ namespace PrivaPub.Federation.Outbox inboxes.Add(parentAuthor.InboxURL); } + if (post.Visibility is PostVisibility.Public or PostVisibility.Unlisted && !string.IsNullOrEmpty(post.QuotedPostId)) + { + var quoted = await _dbEntities.Posts.MatchID(post.QuotedPostId).ExecuteFirstAsync(token); + var quotedAuthor = quoted is { IsFederatedCopy: true } + ? await _dbEntities.ForeignAvatars.Match(a => a.ActorURI == quoted.ActorURI).ExecuteFirstAsync(token) + : default; + if (!string.IsNullOrEmpty(quotedAuthor?.InboxURL)) + inboxes.Add(quotedAuthor.InboxURL); + } + return inboxes.Where(i => !string.IsNullOrEmpty(i)).Distinct(StringComparer.Ordinal).ToList(); } diff --git a/PrivaPub/Federation/Rendering/ActivityPubRenderer.cs b/PrivaPub/Federation/Rendering/ActivityPubRenderer.cs index 821c427..86b12a9 100644 --- a/PrivaPub/Federation/Rendering/ActivityPubRenderer.cs +++ b/PrivaPub/Federation/Rendering/ActivityPubRenderer.cs @@ -39,6 +39,15 @@ namespace PrivaPub.Federation.Rendering ["blurhash"] = "toot:blurhash", ["votersCount"] = "toot:votersCount", ["Emoji"] = "toot:Emoji", + ["quote"] = new JsonObject { ["@id"] = "https://w3id.org/fep/044f#quote", ["@type"] = "@id" }, + ["quoteAuthorization"] = new JsonObject { ["@id"] = "https://w3id.org/fep/044f#quoteAuthorization", ["@type"] = "@id" }, + ["QuoteRequest"] = "https://w3id.org/fep/044f#QuoteRequest", + ["QuoteAuthorization"] = "https://w3id.org/fep/044f#QuoteAuthorization", + ["misskey"] = "https://misskey-hub.net/ns#", + ["_misskey_quote"] = "misskey:_misskey_quote", + ["fedibird"] = "http://fedibird.com/ns#", + ["quoteUri"] = "fedibird:quoteUri", + ["quoteUrl"] = "as:quoteUrl", ["litepub"] = "http://litepub.social/ns#", ["EmojiReact"] = "litepub:EmojiReact", ["focalPoint"] = new JsonObject { ["@container"] = "@list", ["@id"] = "toot:focalPoint" }, @@ -160,6 +169,47 @@ namespace PrivaPub.Federation.Rendering return note; } + public static JsonObject UpdateOf(PostEntity post, LocalActor author, string reason) + { + var note = post.Visibility == PostVisibility.Direct + ? DirectNote(post, author, Array.Empty<(string, string)>(), post.ContextURI) + : Note(post, author, default, post.InReplyToURI); + note["to"] = new JsonArray(post.To.Select(t => (JsonNode)t).ToArray()); + note["cc"] = new JsonArray(post.Cc.Select(c => (JsonNode)c).ToArray()); + return new JsonObject + { + ["@context"] = Context(), + ["id"] = author.ActivityUri($"update-{post.ID}-{reason}"), + ["type"] = "Update", + ["actor"] = author.Uri, + ["to"] = note["to"]!.DeepClone(), + ["cc"] = note["cc"]!.DeepClone(), + ["object"] = note + }; + } + + static void AddQuote(JsonObject note, PostEntity post, ref string content) + { + if (string.IsNullOrEmpty(post.QuoteURI)) + return; + var link = WebUtility.HtmlEncode(post.QuoteURI); + content += $"

RE: {link}

"; + if (post.QuoteByConsent) + note["quote"] = post.QuoteURI; + if (!string.IsNullOrEmpty(post.QuoteAuthorizationURI)) + note["quoteAuthorization"] = post.QuoteAuthorizationURI; + note["_misskey_quote"] = post.QuoteURI; + note["quoteUri"] = post.QuoteURI; + note["quoteUrl"] = post.QuoteURI; + (note["tag"] as JsonArray)?.Add(new JsonObject + { + ["type"] = "Link", + ["mediaType"] = "application/ld+json; profile=\"https://www.w3.org/ns/activitystreams\"", + ["href"] = post.QuoteURI, + ["name"] = $"RE: {post.QuoteURI}" + }); + } + static void AddPoll(JsonObject note, PostPoll poll) { note["type"] = "Question"; @@ -211,6 +261,8 @@ namespace PrivaPub.Federation.Rendering })) .ToArray()) }; + AddQuote(note, post, ref content); + note["content"] = content; if (!string.IsNullOrEmpty(post.Language)) note["contentMap"] = new JsonObject { [post.Language] = content }; if (post.Poll is { } poll) diff --git a/PrivaPub/Middleware/SocialPubConfigurations.cs b/PrivaPub/Middleware/SocialPubConfigurations.cs index 0d81599..95fa444 100644 --- a/PrivaPub/Middleware/SocialPubConfigurations.cs +++ b/PrivaPub/Middleware/SocialPubConfigurations.cs @@ -92,6 +92,7 @@ namespace PrivaPub.Middleware .AddSingleton() .AddSingleton() .AddSingleton() + .AddSingleton() .AddSingleton() .AddSingleton() .AddSingleton() diff --git a/PrivaPub/Models/Post/Post.cs b/PrivaPub/Models/Post/Post.cs index e321184..8fa2b35 100644 --- a/PrivaPub/Models/Post/Post.cs +++ b/PrivaPub/Models/Post/Post.cs @@ -57,6 +57,13 @@ namespace PrivaPub.Models.Post public string Url { get; set; } public string ContextURI { get; set; } public string QuoteURI { get; set; } + public string QuotedPostId { get; set; }//our copy of the quoted post, once fetched + public QuoteState QuoteState { get; set; } + public string QuoteAuthorizationURI { get; set; }//FEP-044f: the quoted author's stamp + public bool QuoteByConsent { get; set; }//our quote of a post whose server asks for consent (FEP-044f), so `quote` is sent + public int QuotesCount { get; set; } + [BsonIgnoreIfNull] + public InteractionRule QuotePolicy { get; set; }//a remote post's interactionPolicy.canQuote; null when it states none public List To { get; set; } = new(); public List Cc { get; set; } = new(); @@ -73,6 +80,17 @@ namespace PrivaPub.Models.Post public string UpdateReason { get; set; } } + public enum QuoteState + { + None, + Pending, + Accepted, + Rejected, + Revoked, + Deleted, + Unauthorized + } + public enum PostVisibility { Public, diff --git a/PrivaPub/Models/Post/PostDetails.cs b/PrivaPub/Models/Post/PostDetails.cs index 5fc8809..e0b2cab 100644 --- a/PrivaPub/Models/Post/PostDetails.cs +++ b/PrivaPub/Models/Post/PostDetails.cs @@ -23,6 +23,12 @@ namespace PrivaPub.Models.Post public int Votes { get; set; } } + public class InteractionRule + { + public List Automatic { get; set; } = new();//actor or collection URIs; Public for anyone + public List Manual { get; set; } = new(); + } + public class PostSource { public string Content { get; set; } diff --git a/PrivaPub/Models/Social/Notification.cs b/PrivaPub/Models/Social/Notification.cs index 0665b59..deedb6e 100644 --- a/PrivaPub/Models/Social/Notification.cs +++ b/PrivaPub/Models/Social/Notification.cs @@ -25,6 +25,7 @@ namespace PrivaPub.Models.Social Reblog, Update, Poll, - Reaction + Reaction, + Quote } } diff --git a/docs/ROADMAP.md b/docs/ROADMAP.md index 2637e19..e91384d 100644 --- a/docs/ROADMAP.md +++ b/docs/ROADMAP.md @@ -453,9 +453,11 @@ it, raw where it doesn't. - **Polls** in and out, with the Misskey, Pleroma and PieFed vote shapes. A count refresh is never an edit (done in v1.10.0; checked live both ways against GoToSocial). - **Quotes (FEP-044f):** - - read every key; verify `QuoteAuthorization` on all of its fields; handle revocation; - - be quotable: `canQuote`, answer `QuoteRequest`, serve and revoke stamps; - - then advertise `api_versions.mastodon ≥ 7` (4.5.0). + - read every key; verify `QuoteAuthorization` on all of its fields; handle revocation (done in v1.13.0); + - personas quote others: `quoted_status_id`, `QuoteRequest`, `Accept{result}` verified, `Update` with + `quoteAuthorization`; legacy quotes for posts that state no policy (done in v1.13.0); `api_versions.mastodon = 7`; + - still open: be quotable (`canQuote` on our posts, answer `QuoteRequest`, serve and revoke stamps). It needs a themed + route name for stamps, agreed with the owner, and a default quote policy. - **Emoji reactions** in all three inbound forms, plus outbound `EmojiReact`, exposed as `emoji_reactions` (done in v1.11.0; Pleroma's `/api/v1/pleroma/statuses/:id/reactions` endpoints and `pleroma:emoji_reaction` notifications). - **Link cards** (done in v1.12.0): diff --git a/tools/pasture/interop.sh b/tools/pasture/interop.sh index af9e5d1..a90d08f 100755 --- a/tools/pasture/interop.sh +++ b/tools/pasture/interop.sh @@ -114,6 +114,11 @@ gts_poll_on_pp=$(curl -s -H "$PH" "$P/api/v1/timelines/home" | j "print(next(s[' curl -s -o /dev/null -X POST -H "$PH" "$P/api/v1/polls/$gts_poll_on_pp/votes" -d 'choices[]=0' until_true 20 '[ "$(gcurl -s -H "$GH" "$G/api/v1/statuses/$gts_poll" | j "print(d[\"poll\"][\"options\"][0][\"votes_count\"])")" = "1" ]' && ok "alice's vote counts on GoToSocial" || ko "vote not counted on GoToSocial" +echo "quotes" +quote_target=$(curl -s -H "$PH" "$P/api/v1/timelines/home" | j "print(next(s['id'] for s in d if 'red or blue' in s['content']))") +[ "$(curl -s -o /dev/null -w '%{http_code}' -X POST -H "$PH" $P/api/v1/statuses -d "status=quoting"ed_status_id=$quote_target")" = "422" ] \ + && ok "GoToSocial's author-only quote policy is respected" || ko "quoted a post whose author forbids it" + echo "link previews" linked=$(curl -s -X POST -H "$PH" $P/api/v1/statuses --data-urlencode 'status=have a look https://gts.test/@gtsuser' -d 'visibility=public' | j "print(d['id'])") until_true 45 '[ -n "$(curl -s -H "$PH" "$P/api/v1/statuses/$linked" | j "print((d[\"card\"] or {}).get(\"title\") or \"\")")" ]' && ok "the server builds a card for a linked page" || ko "no card for the linked page"