diff --git a/CLAUDE.md b/CLAUDE.md index 439f9ac..d284e40 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -22,8 +22,12 @@ Its defining idea: **one private login owns several public personas.** - **`DmGroup` is a direct-message conversation.** - **`privapub` is the instance actor** (type Application). It signs fetches no persona should be tied to. - **`privapub_reports` is the reporter** (type Service, `LocalActorKind.Reporter`). It carries reports to Lemmy, which - takes none from an Application, and to PieFed and Mbin, and names nobody. Both are server actors - (`LocalActor.IsServerActor`): never followed, mentioned or shown as accounts. + takes none from an Application, and to PieFed and Mbin, and names nobody. +- **`privapub_feeds` is the reader of feeds** (type Service, `LocalActorKind.Reader`). When a persona follows a feed + (Lemmy's multi-community, PieFed's feed), it follows the feed's communities in the server's name + (`Domain/Social/FeedFollows.cs`, owner decision 2026-10-07); its `Following` rows carry `FollowerKind`, and nobody's + home gets what it brings in. All three are server actors (`LocalActor.IsServerActor`): never followed, mentioned or + shown as accounts. Privacy features in the model: - location-ranged posts (`Post.Location`, `RangeKm`), to become local-only and never federated; @@ -288,7 +292,9 @@ group www-data and reaches the private mongod; `sudo -u www-data` works too. (`NotificationPermission`). 7. **What a Mastodon `Status` cannot say goes in `Status.privapub`** (`PrivaPubStatus`): object type, title, excerpt, cover, the author's source, link, video, audio and event details, a remote post's own place (Pixelfed's `location`, - never federated again), up/down votes, and its community's flairs (`ObjectShapes.Flairs`, both dialects). Every media URL in it goes + never federated again), up/down votes and the viewer's own downvote (`votes.downvoted`, set through PrivaPub's + `POST /api/v1/statuses/:id/downvote` and `/undownvote`; a favourite is the upvote), and its community's flairs + (`ObjectShapes.Flairs`, both dialects). Every media URL in it goes through the proxy; only page links (`link.url`, an event's online link) point at the remote site, because following one is the reader's choice. diff --git a/FEDERATION.md b/FEDERATION.md index 17886c3..789ce2f 100644 --- a/FEDERATION.md +++ b/FEDERATION.md @@ -136,6 +136,9 @@ The names are the project's own and are stable; resolve actors through WebFinger - The reporter is `/peasants/privapub_reports` (type `Service`, "Reports from "), one for the whole server with its own key. It sends the reports Lemmy, PieFed and Mbin take in Lemmy's shape (see **Reports** below) and does nothing else: nobody follows or mentions it, the Mastodon API has no account for it, and it has no page. +- The reader of feeds is `/peasants/privapub_feeds` (type `Service`, "Feeds read on "), one for the whole server + with its own key. When a persona here follows a feed, the reader follows each community in it, and stops following one + no feed read here holds any more; no community and no feed learns which persona reads it. It does nothing else. ## Groups @@ -172,7 +175,9 @@ A group is either a **community** or a **circle**. there with its thread; a move into a community this server hosts is not taken. - **Feeds** (Lemmy 1.0's multi-communities and PieFed's feeds, `type: Feed`) are read as accounts whose following is their communities: the feed's `following` collection, read once a day with its counts, and each community in it read in - turn. A feed cannot be followed from here yet; its communities can. + turn. A persona following a feed is followed through the server (owner decision 2026-10-07): the reader of feeds + follows its communities, the feed itself is sent nothing, and the persona reads the feed's threads in a timeline of + its own (`GET /api/v1/timelines/feed/:id`; the feeds it reads are `GET /api/v1/feeds`), not in its home. ## Activities @@ -201,8 +206,14 @@ Received: | `Block` of a persona | the follows between them end, the blocker's posts and notifications are hidden from the persona, nothing of the persona's is addressed to the blocker, and the relationship says `blocked_by`; `Undo{Block}` lifts it | Sent: `Follow`, `Undo{Follow}`, `Create{Note}`, `Create{Question}` and poll votes, `EmojiReact` and its `Undo`, `Update{Note}`, `Update{Person}`, `Delete{Tombstone}`, `Accept{Follow}`, -`Reject{Follow}`, `Like`, `Announce` and their `Undo`, `Flag`, `Join` and `Leave` of a remote event, and the replies to a -persona's posts passed on to its followers. A deleted post answers 410 with a `Tombstone`. +`Reject{Follow}`, `Like`, `Dislike`, `Announce` and their `Undo`, `Flag`, `Join` and `Leave` of a remote event, and the +replies to a persona's posts passed on to its followers. A deleted post answers 410 with a `Tombstone`. + +- **Votes.** A favourite is a `Like`, a downvote a `Dislike`; a persona has one vote a post, and a changed vote is sent + as the new vote alone, as Lemmy sends one (an `Undo` of the old one beside it could arrive after it and take the new + one away). `Undo` is sent only when the vote is taken back. A vote on a post in a remote community goes to the + community, which counts it and passes it on, as Lemmy sends one, and to the post's author only when it lives on + another server: one copy a server, since PieFed drops a second copy of an activity it has just seen. - **Attachments** are `Document`s with `mediaType`, `name` (alt text), `blurhash`, `focalPoint`, `width` and `height`. Uploaded files have all metadata removed. @@ -405,8 +416,9 @@ Posts with a location (shown to nearby users of this server) never leave the ser unlisted replies are kept, each fetched from its own origin. A thread collection read whole, which counts its posts or holds them all, is asked again with `If-None-Match` and its last ETag; a 304 ends the read. - **Reading our documents (SecureMode).** privapub.thepra.dev answers ActivityPub GETs only when they are signed, like - Mastodon's authorized fetch; the server's own actors (the instance actor `/peasants/privapub` and the reporter - `/peasants/privapub_reports`) are the exception, since their keys are needed first. + Mastodon's authorized fetch; the server's own actors (the instance actor `/peasants/privapub`, the reporter + `/peasants/privapub_reports` and the reader of feeds `/peasants/privapub_feeds`) are the exception, since their keys + are needed first. A browser asking for HTML is redirected to the public page instead. - **Posts that are not public** (followers-only, direct, circle) are served to a signed request from someone they were for, or from the instance actor of a server where someone they were for lives, and to nobody else (404). Once deleted diff --git a/PrivaPub.Tests/Domain/ContentRendererTests.cs b/PrivaPub.Tests/Domain/ContentRendererTests.cs index 8357b58..6ecaf6b 100644 --- a/PrivaPub.Tests/Domain/ContentRendererTests.cs +++ b/PrivaPub.Tests/Domain/ContentRendererTests.cs @@ -71,6 +71,7 @@ namespace PrivaPub.Tests.Domain public Task FindByAddress(string address, CancellationToken token) => throw new NotSupportedException(); public Task GetInstanceActor(CancellationToken token) => throw new NotSupportedException(); public Task GetReporterActor(CancellationToken token) => throw new NotSupportedException(); + public Task GetReaderActor(CancellationToken token) => throw new NotSupportedException(); public Task IsUserNameTaken(string userName, CancellationToken token) => throw new NotSupportedException(); public Task TryReserveUserName(string userName, LocalActorKind kind, string ownerId, CancellationToken token) => throw new NotSupportedException(); public LocalActor FromAvatar(Avatar avatar) => throw new NotSupportedException(); diff --git a/PrivaPub.Tests/Http/MastodonFeedTests.cs b/PrivaPub.Tests/Http/MastodonFeedTests.cs index 3ea369e..f70f1c0 100644 --- a/PrivaPub.Tests/Http/MastodonFeedTests.cs +++ b/PrivaPub.Tests/Http/MastodonFeedTests.cs @@ -1,5 +1,10 @@ +using MongoDB.Entities; + using PrivaPub.Federation.Actors; +using PrivaPub.Models.Federation; using PrivaPub.Models.Jobs; +using PrivaPub.Models.Post; +using PrivaPub.Models.Social; using PrivaPub.Tests.Support; using PrivaPub.Tests.Support.Host; @@ -61,7 +66,75 @@ namespace PrivaPub.Tests.Http var communities = (await reader.Client.Get($"/api/v1/accounts/{account.ID}/following")).Ok().Array; Assert.Equal(new[] { cats.Id, dogs.Id }, communities.Select(c => c.Text("uri"))); Assert.All(communities, c => Assert.True(c!["group"]!.GetValue())); - Assert.Equal(HttpStatusCode.UnprocessableEntity, (await reader.Client.Post($"/api/v1/accounts/{account.ID}/follow")).Status); + } + + // owner decision 2026-10-07: a feed is read through the server, whose reader follows its communities; the persona + // follows nothing, and reads the feed in a timeline of its own + [Fact] + public async Task A_feed_is_followed_by_the_servers_reader_and_read_in_its_own_timeline() + { + var token = TestContext.Current.CancellationToken; + var since = DateTime.UtcNow.AddSeconds(-1); + var persona = await _host.Mastodon("feedreader"); + var cats = new RemoteActor(_peer, "cats", type: "Group"); + var dogs = new RemoteActor(_peer, "dogs", type: "Group"); + var poster = new RemoteActor(_peer, "poster"); + var name = $"feed{Guid.NewGuid():N}"[..12]; + var feed = $"{_peer.A}/m/{name}"; + _peer.Serve($"/m/{name}", new JsonObject + { + ["type"] = "Feed", ["id"] = feed, ["inbox"] = $"{_peer.A}/inbox", ["following"] = $"{feed}/following", ["preferredUsername"] = name, ["name"] = "Pets" + }.ToJsonString()); + _peer.Serve($"/m/{name}/following", new JsonObject + { + ["type"] = "Collection", ["id"] = $"{feed}/following", ["totalItems"] = 2, ["items"] = new JsonArray(cats.Id, dogs.Id) + }.ToJsonString()); + var account = await _host.Get().GetActor(feed, refresh: true, token); + await _host.Run(j => j.Kind == JobKind.CountAccount && j.Payload == feed, token); + + var followed = (await persona.Client.Post($"/api/v1/accounts/{account.ID}/follow")).Ok().Body; + Assert.True(followed.Flag("following")); + Assert.Equal(new[] { account.ID }, (await persona.Client.Get("/api/v1/feeds")).Ok().Ids); + await _host.Run(j => j.Kind == JobKind.SyncFeeds, token); + + var readerActor = await _host.Get().GetReaderActor(token); + var follows = await DB.Default.Find().Match(f => f.AvatarId == readerActor.Id).ExecuteAsync(token); + Assert.Equal(new[] { cats.Id, dogs.Id }.Order(), follows.Select(f => f.TargetActorURI).Order()); + Assert.All(follows, f => Assert.Equal(LocalActorKind.Reader, f.FollowerKind)); + Assert.False(await DB.Default.Find().Match(f => f.AvatarId == persona.Id).ExecuteAnyAsync(token)); + var asked = Assert.Single(await cats.Delivered(since), d => d.Type() == "Follow"); + Assert.Equal(readerActor.Uri, asked.Text("actor")); + var document = (await _host.Client().Fetch(new Uri(readerActor.Uri).AbsolutePath)).Json; + Assert.Equal(("Service", LocalActorService.ReaderUserName), (document.Text("type"), document.Text("preferredUsername"))); + + await _host.Deliver(cats, "/human-centipede", new JsonObject + { + ["id"] = $"{cats.Id}/accepts/{Guid.NewGuid():N}", ["type"] = "Accept", ["actor"] = cats.Id, ["object"] = asked.Text("id") + }); + var create = poster.Create("

a thread among cats

", new[] { "https://www.w3.org/ns/activitystreams#Public", cats.Id }, shape: note => + { + note["type"] = "Page"; + note["name"] = "Cats"; + note["audience"] = cats.Id; + }); + _peer.Serve(new Uri(create["object"]!.Text("id")).AbsolutePath, create["object"]!.ToJsonString()); + await _host.Deliver(cats, "/human-centipede", new JsonObject + { + ["id"] = $"{cats.Id}/announces/{Guid.NewGuid():N}", ["type"] = "Announce", ["actor"] = cats.Id, ["object"] = create, + ["to"] = new JsonArray("https://www.w3.org/ns/activitystreams#Public") + }); + var thread = await DB.Default.Find().Match(p => p.ObjectURI == create["object"]!.Text("id")).ExecuteSingleAsync(token); + + Assert.Equal(new[] { thread.ID }, (await persona.Client.Get($"/api/v1/timelines/feed/{account.ID}")).Ok().Ids); + Assert.DoesNotContain(thread.ID, (await persona.Client.Get("/api/v1/timelines/home")).Ok().Ids); + Assert.False(await DB.Default.Find().Match(e => e.PostId == thread.ID).ExecuteAnyAsync(token)); + var stranger = await _host.Mastodon("notreading"); + Assert.Equal(HttpStatusCode.NotFound, (await stranger.Client.Get($"/api/v1/timelines/feed/{account.ID}")).Status); + + Assert.False((await persona.Client.Post($"/api/v1/accounts/{account.ID}/unfollow")).Ok().Body.Flag("following")); + await _host.Run(j => j.Kind == JobKind.SyncFeeds, token); + Assert.False(await DB.Default.Find().Match(f => f.AvatarId == readerActor.Id).ExecuteAnyAsync(token)); + Assert.Contains(await cats.Delivered(since), d => d.Type() == "Undo" && d["object"].Text("type") == "Follow"); } } } diff --git a/PrivaPub.Tests/Http/MastodonStatusesTests.cs b/PrivaPub.Tests/Http/MastodonStatusesTests.cs index bca0dc6..97a54ee 100644 --- a/PrivaPub.Tests/Http/MastodonStatusesTests.cs +++ b/PrivaPub.Tests/Http/MastodonStatusesTests.cs @@ -388,6 +388,47 @@ namespace PrivaPub.Tests.Http Assert.True((await alice.Client.Post($"/api/v1/statuses/{quiet.Text("id")}/favourite")).Ok().Body.Flag("favourited")); } + // a downvote is a Dislike, one vote a post: a downvote replaces the favourite and a favourite the downvote, each by its + // own activity alone; a vote on a post in a community goes to the community, which counts it, and to its author only + // when on another server + [Fact] + public async Task A_downvote_replaces_the_favourite_and_votes_reach_the_posts_community() + { + var since = DateTime.UtcNow.AddSeconds(-1); + var alice = await _host.Mastodon("alice"); + var bob = new RemoteActor(_peer, "bob", origin: _peer.B); + var community = new RemoteActor(_peer, "pies", type: "Group"); + await _host.Get().GetActor(community.Id, refresh: true, TestContext.Current.CancellationToken); + var remote = await _host.PublicPostFrom(bob, alice, shape: note => note["audience"] = community.Id); + var neighbour = new RemoteActor(_peer, "neighbour"); + var nearby = await _host.PublicPostFrom(neighbour, alice, shape: note => note["audience"] = community.Id); + (await alice.Client.Post($"/api/v1/statuses/{nearby.ID}/favourite")).Ok(); + Assert.Empty(await neighbour.Delivered(since)); + var path = $"/api/v1/statuses/{remote.ID}"; + + (await alice.Client.Post(path + "/favourite")).Ok(); + var down = (await alice.Client.Post(path + "/downvote")).Ok().Body; + Assert.False(down.Flag("favourited")); + Assert.Equal((0, 1, true), (down["privapub"]!["votes"].Number("up"), down["privapub"]!["votes"].Number("down"), down["privapub"]!["votes"].Flag("downvoted"))); + Assert.Equal(1, (await alice.Client.Post(path + "/downvote")).Ok().Body["privapub"]!["votes"].Number("down")); + var up = (await alice.Client.Post(path + "/favourite")).Ok().Body; + Assert.Equal((true, 0, false), (up.Flag("favourited"), up["privapub"]!["votes"].Number("down"), up["privapub"]!["votes"].Flag("downvoted"))); + Assert.Null((await _host.Client().Get(path)).Ok().Body["privapub"]!["votes"]!["downvoted"]); + (await alice.Client.Post(path + "/downvote")).Ok(); + var none = (await alice.Client.Post(path + "/undownvote")).Ok().Body; + Assert.Equal((false, 0, 0), (none.Flag("favourited"), none.Number("favourites_count"), none["privapub"]!["votes"].Number("down"))); + + foreach (var inbox in new[] { bob, community }) + { + var sent = (await inbox.Delivered(since)).Where(d => (d.Type() == "Undo" ? d["object"].Text("object") : d.Text("object")) == remote.ObjectURI).ToList(); + Assert.Equal(new[] { "Like", "Dislike", "Like", "Dislike", "Undo:Dislike" }, + sent.Select(d => d.Type() == "Undo" ? "Undo:" + d["object"].Text("type") : d.Type())); + var dislike = sent.Last(d => d.Type() == "Dislike"); + Assert.Equal((alice.Uri, remote.ObjectURI), (dislike.Text("actor"), dislike.Text("object"))); + Assert.Equal(dislike.Text("id"), sent.Single(d => d.Type() == "Undo")["object"].Text("id")); + } + } + [Fact] public async Task A_favourite_or_a_boost_given_again_after_its_undo_is_a_new_activity_that_is_delivered() { diff --git a/PrivaPub/Api/Mastodon/Controllers/AccountsController.cs b/PrivaPub/Api/Mastodon/Controllers/AccountsController.cs index f1e565d..aa8f58c 100644 --- a/PrivaPub/Api/Mastodon/Controllers/AccountsController.cs +++ b/PrivaPub/Api/Mastodon/Controllers/AccountsController.cs @@ -32,10 +32,12 @@ namespace PrivaPub.Api.Mastodon.Controllers readonly IFollowService _follows; readonly IOutboxPublisher _outbox; readonly IRelationshipService _relationships; + readonly IFeedFollows _feeds; public AccountsController(MastodonMapper mapper, DbEntities dbEntities, ILocalActorService localActors, IRemoteActorService remoteActors, - IFollowService follows, IOutboxPublisher outbox, IRelationshipService relationships) + IFollowService follows, IOutboxPublisher outbox, IRelationshipService relationships, IFeedFollows feeds) { + _feeds = feeds; _relationships = relationships; _mapper = mapper; _dbEntities = dbEntities; @@ -266,14 +268,27 @@ namespace PrivaPub.Api.Mastodon.Controllers return Json(following.Select(f => accounts.GetValueOrDefault(f.TargetAccountId)).Where(a => a != default).ToList()); } + // PrivaPub's own: the feeds the persona reads (a feed's timeline is /api/v1/timelines/feed/:id) + [HttpGet("/api/v1/feeds"), Scope("read:follows")] + public async Task Feeds(CancellationToken token) + { + var read = await DB.Default.Find().Match(s => s.AvatarId == MyId).Sort(s => s.ID, MongoDB.Entities.Order.Descending).ExecuteAsync(token); + var accounts = await _mapper.Accounts(read.Select(s => s.FeedAccountId), token); + return Json(read.Select(s => accounts.GetValueOrDefault(s.FeedAccountId)).Where(a => a != default).ToList()); + } + [HttpPost("/api/v1/accounts/{id}/follow"), Scope("write:follows")] public async Task Follow(string id, CancellationToken token) { var (local, remote) = await Find(id, token); if (local == default && remote == default) return NotFoundError(); + // a feed is read through the server, whose reader follows its communities; the persona follows nothing if (remote is { AvatarType: AvatarType.Feed }) - return Error(StatusCodes.Status422UnprocessableEntity, "A feed is followed through its communities"); + { + await _feeds.Subscribe(Me, remote, true, token); + return Json(await Relationship(id, token)); + } var following = await _follows.FollowAs(Me, local?.Uri ?? remote.ActorURI, Params.Bool("reblogs") != false, token); return following == default ? Error(StatusCodes.Status403Forbidden, "This action is not allowed") : Json(await Relationship(id, token)); } @@ -284,7 +299,10 @@ namespace PrivaPub.Api.Mastodon.Controllers var (local, remote) = await Find(id, token); if (local == default && remote == default) return NotFoundError(); - await _follows.UnfollowAs(Me, local?.Uri ?? remote.ActorURI, token); + if (remote is { AvatarType: AvatarType.Feed }) + await _feeds.Subscribe(Me, remote, false, token); + else + await _follows.UnfollowAs(Me, local?.Uri ?? remote.ActorURI, token); return Json(await Relationship(id, token)); } @@ -432,6 +450,7 @@ namespace PrivaPub.Api.Mastodon.Controllers var (local, remote) = await Find(id, token); var uri = local?.Uri ?? remote?.ActorURI; var following = uri == default ? default : await _dbEntities.Followings.Match(f => f.AvatarId == MyId && f.TargetActorURI == uri).ExecuteFirstAsync(token); + var reading = remote is { AvatarType: AvatarType.Feed } && await _feeds.IsSubscribed(MyId, remote.ActorURI, token); var followedBy = uri == default ? default : await _dbEntities.Followers.Match(f => f.LocalActorId == MyId && f.ActorURI == uri).ExecuteFirstAsync(token); var blocking = uri != default && await DB.Default.Find().Match(b => b.AvatarId == MyId && b.TargetActorURI == uri).ExecuteAnyAsync(token); var mute = uri == default ? default : await DB.Default.Find().Match(m => m.AvatarId == MyId && m.TargetActorURI == uri).ExecuteFirstAsync(token); @@ -448,7 +467,7 @@ namespace PrivaPub.Api.Mastodon.Controllers Muting = mute != default && (mute.ExpiresAt == default || mute.ExpiresAt > DateTime.UtcNow), MutingNotifications = mute?.HideNotifications == true, DomainBlocking = domainBlocked, - Following = following?.State == FollowState.Accepted, + Following = following?.State == FollowState.Accepted || reading, Requested = following?.State == FollowState.Requested, ShowingReblogs = following?.ShowReblogs ?? false, FollowedBy = followedBy?.IsAccepted == true, diff --git a/PrivaPub/Api/Mastodon/Controllers/StatusesController.cs b/PrivaPub/Api/Mastodon/Controllers/StatusesController.cs index 729e1c0..c843ba7 100644 --- a/PrivaPub/Api/Mastodon/Controllers/StatusesController.cs +++ b/PrivaPub/Api/Mastodon/Controllers/StatusesController.cs @@ -333,6 +333,13 @@ namespace PrivaPub.Api.Mastodon.Controllers [HttpPost("/api/v1/statuses/{id}/unfavourite"), Scope("write:favourites")] public Task Unfavourite(string id, CancellationToken token) => Toggle(_statuses.Favourite(Me, id, false, token), id, token); + // PrivaPub's own, for the threadiverse's downvotes (a favourite is the upvote) + [HttpPost("/api/v1/statuses/{id}/downvote"), Scope("write:favourites")] + public Task Downvote(string id, CancellationToken token) => Toggle(_statuses.Downvote(Me, id, true, token), id, token); + + [HttpPost("/api/v1/statuses/{id}/undownvote"), Scope("write:favourites")] + public Task Undownvote(string id, CancellationToken token) => Toggle(_statuses.Downvote(Me, id, false, token), id, token); + [HttpPost("/api/v1/statuses/{id}/reblog"), Scope("write:statuses")] public async Task Reblog(string id, CancellationToken token) { diff --git a/PrivaPub/Api/Mastodon/Controllers/TimelinesController.cs b/PrivaPub/Api/Mastodon/Controllers/TimelinesController.cs index 1ba7311..0dc8b02 100644 --- a/PrivaPub/Api/Mastodon/Controllers/TimelinesController.cs +++ b/PrivaPub/Api/Mastodon/Controllers/TimelinesController.cs @@ -64,6 +64,26 @@ namespace PrivaPub.Api.Mastodon.Controllers return await Respond(query, "/api/v1/timelines/public", token); } + // PrivaPub's own: a feed the persona reads (Lemmy's multi-community, PieFed's feed), its communities' threads, + // newest first, as the server's reader of feeds brings them in (owner decision 2026-10-07) + [HttpGet("/api/v1/timelines/feed/{id}"), Scope("read:statuses")] + public async Task Feed(string id, CancellationToken token) + { + var feed = await _dbEntities.ForeignAvatars.MatchID(id).ExecuteFirstAsync(token); + if (feed is not { AvatarType: Models.User.AvatarType.Feed } + || !await DB.Default.Find().Match(s => s.AvatarId == MyId && s.FeedActorURI == feed.ActorURI).ExecuteAnyAsync(token)) + return NotFoundError(); + var communities = feed.FeedCommunities; + var ours = Me.BaseAddress + "/peasants/"; + var names = communities.Where(c => c.StartsWith(ours, StringComparison.Ordinal)).Select(c => c[ours.Length..]).ToList(); + var groups = names.Count == 0 + ? new List() + : (await _dbEntities.Groups.Match(g => names.Contains(g.UserName) && !g.DeletionAt.HasValue).ExecuteAsync(token)).Select(g => g.ID).ToList(); + var query = _dbEntities.Posts.Match(p => (communities.Contains(p.AudienceURI) || groups.Contains(p.GroupId)) + && p.ReblogOfPostId == null && p.InReplyToURI == null).Match(VisibilityPolicy.IsPublic); + return await Respond(query, $"/api/v1/timelines/feed/{id}", token); + } + [HttpGet("/api/v1/timelines/tag/{hashtag}"), Scope("read:statuses", requiresUser: false), Microsoft.AspNetCore.Authorization.AllowAnonymous] public async Task Tag(string hashtag, CancellationToken token) { diff --git a/PrivaPub/Api/Mastodon/Entities/Entities.cs b/PrivaPub/Api/Mastodon/Entities/Entities.cs index d0bc5ae..694f57a 100644 --- a/PrivaPub/Api/Mastodon/Entities/Entities.cs +++ b/PrivaPub/Api/Mastodon/Entities/Entities.cs @@ -180,6 +180,7 @@ { public int Up { get; set; } public int Down { get; set; } + public bool? Downvoted { get; set; }//the viewer's own downvote; null with no viewer } public class QuoteEntity diff --git a/PrivaPub/Api/Mastodon/Mappers/MastodonMapper.cs b/PrivaPub/Api/Mastodon/Mappers/MastodonMapper.cs index 30b5129..075731f 100644 --- a/PrivaPub/Api/Mastodon/Mappers/MastodonMapper.cs +++ b/PrivaPub/Api/Mastodon/Mappers/MastodonMapper.cs @@ -218,6 +218,9 @@ namespace PrivaPub.Api.Mastodon.Mappers var favourited = viewerId == default ? new HashSet() : (await _dbEntities.Favourites.Match(f => f.AccountId == viewerId && ids.Contains(f.PostId)).ExecuteAsync(token)).Select(f => f.PostId).ToHashSet(); + var downvoted = viewerId == default + ? new HashSet() + : (await DB.Default.Find().Match(d => d.AccountId == viewerId && ids.Contains(d.PostId)).ExecuteAsync(token)).Select(d => d.PostId).ToHashSet(); var bookmarked = viewerId == default ? new HashSet() : (await DB.Default.Find().Match(b => b.AvatarId == viewerId && ids.Contains(b.PostId)).ExecuteAsync(token)).Select(b => b.PostId).ToHashSet(); @@ -286,7 +289,7 @@ namespace PrivaPub.Api.Mastodon.Mappers Card = Card(post), Poll = Poll(post, viewerId, ownVotes.GetValueOrDefault(post.ID)), Emojis = Emojis(post.Emojis), - Privapub = Extension(post, accounts), + Privapub = Extension(post, accounts, viewerId == default ? default(bool?) : downvoted.Contains(post.ID)), EmojiReactions = reactions.GetValueOrDefault(post.ID) ?? new(), Pleroma = new PleromaStatus { EmojiReactions = reactions.GetValueOrDefault(post.ID) ?? new() }, Mentions = post.Mentions.Where(m => !m.Silent).Select(m => Mention(m, mentionAccounts)).Where(m => m != default).ToList(), @@ -489,7 +492,7 @@ namespace PrivaPub.Api.Mastodon.Mappers List Emojis(IEnumerable emojis) => emojis?.Select(e => new CustomEmojiEntity { Shortcode = e.Shortcode, Url = Proxied(e.URL), StaticUrl = Proxied(e.URL) }).ToList() ?? new(); - PrivaPubStatus Extension(PostEntity post, Dictionary accounts) => new() + PrivaPubStatus Extension(PostEntity post, Dictionary accounts, bool? downvoted) => new() { Wall = post.WallOwnerAccountId == default || post.WallOwnerAccountId == AuthorOf(post) ? default : accounts.GetValueOrDefault(post.WallOwnerAccountId), ObjectType = post.ObjectType, @@ -543,7 +546,7 @@ namespace PrivaPub.Api.Mastodon.Mappers { Name = post.Place.Name, Latitude = post.Place.Latitude, Longitude = post.Place.Longitude, Country = post.Place.Country }, - Votes = new PrivaPubVotes { Up = post.FavouritesCount, Down = post.DownvotesCount }, + Votes = new PrivaPubVotes { Up = post.FavouritesCount, Down = post.DownvotesCount, Downvoted = downvoted }, Flairs = post.Flairs?.Select(f => new PrivaPubFlair { Id = f.Id, Name = f.Name, Slug = f.Slug, Description = f.Description, Color = f.Color, TextColor = f.TextColor, diff --git a/PrivaPub/Domain/Social/FeedFollows.cs b/PrivaPub/Domain/Social/FeedFollows.cs new file mode 100644 index 0000000..92b2ba2 --- /dev/null +++ b/PrivaPub/Domain/Social/FeedFollows.cs @@ -0,0 +1,134 @@ +using MongoDB.Driver; +using MongoDB.Entities; + +using PrivaPub.Federation.Actors; +using PrivaPub.Infrastructure.Jobs; +using PrivaPub.Models.Jobs; +using PrivaPub.Models.Social; +using PrivaPub.Models.User; +using PrivaPub.StaticServices; + +namespace PrivaPub.Domain.Social +{ + public interface IFeedFollows + { + Task Subscribe(LocalActor me, ForeignAvatar feed, bool on, CancellationToken token); + Task IsSubscribed(string avatarId, string feedActorUri, CancellationToken token); + Task Reconcile(CancellationToken token); + } + + // Feeds read through the server (owner decision 2026-10-07): a persona following a feed is kept here alone, and the + // server's reader (LocalActorService.ReaderUserName) follows every community in the feeds anyone reads, as Lemmy's own + // follower of multi-communities does, and stops following one no feed holds any more. What it follows arrives as any + // community's posts do; nobody's home shows them, the feed's timeline does. + public class FeedFollows : IFeedFollows + { + public const int MaxCommunities = 500; + static readonly TimeSpan Stale = TimeSpan.FromDays(1); + + readonly DbEntities _dbEntities; + readonly ILocalActorService _localActors; + readonly IRemoteActorService _remoteActors; + readonly IFollowService _follows; + readonly IJobQueue _jobs; + + public FeedFollows(DbEntities dbEntities, ILocalActorService localActors, IRemoteActorService remoteActors, IFollowService follows, IJobQueue jobs) + { + _dbEntities = dbEntities; + _localActors = localActors; + _remoteActors = remoteActors; + _follows = follows; + _jobs = jobs; + } + + public static Task Sync(IJobQueue jobs, CancellationToken token) => + jobs.Enqueue(JobKind.SyncFeeds, string.Empty, default, $"feeds|{Guid.NewGuid():N}", token); + + public async Task Subscribe(LocalActor me, ForeignAvatar feed, bool on, CancellationToken token) + { + if (feed is not { AvatarType: AvatarType.Feed }) + return false; + if (on) + try + { + await DB.Default.SaveAsync(new FeedSubscription { AvatarId = me.Id, FeedActorURI = feed.ActorURI, FeedAccountId = feed.ID }, token); + } + catch (MongoWriteException ex) when (ex.WriteError?.Category == ServerErrorCategory.DuplicateKey) + { + return true; + } + else if ((await DB.Default.DeleteAsync(s => s.AvatarId == me.Id && s.FeedActorURI == feed.ActorURI)).DeletedCount == 0) + return true; + await Sync(_jobs, token); + return true; + } + + public Task IsSubscribed(string avatarId, string feedActorUri, CancellationToken token) => + DB.Default.Find().Match(s => s.AvatarId == avatarId && s.FeedActorURI == feedActorUri).ExecuteAnyAsync(token); + + public async Task Reconcile(CancellationToken token) + { + var read = (await DB.Default.Find().Project(s => s.FeedActorURI).ExecuteAsync(token)).Distinct().ToList(); + var feeds = await _dbEntities.ForeignAvatars.Match(a => read.Contains(a.ActorURI) && a.AvatarType == AvatarType.Feed && !a.DeletionAt.HasValue) + .ExecuteAsync(token); + // a feed's communities are as its server listed them a day ago at most (AccountCountsJob reads them, then asks + // for this again) + foreach (var stale in feeds.Where(f => !(DateTime.UtcNow - f.CountedAt < Stale))) + if (Uri.TryCreate(stale.ActorURI, UriKind.Absolute, out var uri)) + await _jobs.Enqueue(JobKind.CountAccount, stale.ActorURI, uri.Host.ToLowerInvariant(), AccountCountsJob.DedupeKey(stale.ActorURI, DateTime.UtcNow), token); + + var ours = _localActors.BaseAddress + "/"; + var wanted = feeds.SelectMany(f => f.FeedCommunities).Where(c => !c.StartsWith(ours, StringComparison.Ordinal)) + .Distinct(StringComparer.Ordinal).Take(MaxCommunities).ToHashSet(StringComparer.Ordinal); + var reader = await _localActors.GetReaderActor(token); + var following = await _dbEntities.Followings.Match(f => f.AvatarId == reader.Id).ExecuteAsync(token); + foreach (var gone in following.Where(f => !wanted.Contains(f.TargetActorURI))) + await _follows.UnfollowAs(reader, gone.TargetActorURI, token); + var followed = following.Select(f => f.TargetActorURI).ToHashSet(StringComparer.Ordinal); + foreach (var community in wanted.Where(c => !followed.Contains(c))) + // only a community: a feed lists nothing else, and the reader follows nothing else + if (await _remoteActors.GetActor(community, refresh: false, token) is { AvatarType: AvatarType.Group }) + await _follows.FollowAs(reader, community, showReblogs: false, token); + } + } + + public class FeedSyncJob(IServiceScopeFactory scopes) : IJobHandler + { + public JobKind Kind => JobKind.SyncFeeds; + public int Concurrency => 1; + public int MaxAttempts => 2; + public int PerHostLimit => 1; + + public async Task Handle(Job job, CancellationToken token) + { + using var scope = scopes.CreateScope(); + await scope.ServiceProvider.GetRequiredService().Reconcile(token); + return JobOutcome.Done; + } + } + + // brings the reader's follows up to date with the feeds read here, a minute after start and then every six hours + public sealed class FeedReader(IServiceScopeFactory scopes, ILogger logger) : BackgroundService + { + static readonly TimeSpan Every = TimeSpan.FromHours(6); + + protected override async Task ExecuteAsync(CancellationToken token) + { + await Task.Delay(TimeSpan.FromMinutes(1), token); + using var timer = new PeriodicTimer(Every); + do + { + try + { + using var scope = scopes.CreateScope(); + await FeedFollows.Sync(scope.ServiceProvider.GetRequiredService(), token); + } + catch (Exception ex) when (ex is not OperationCanceledException) + { + logger.LogWarning(ex, "Feeds could not be brought up to date"); + } + } + while (await timer.WaitForNextTickAsync(token)); + } + } +} diff --git a/PrivaPub/Domain/Social/FollowService.cs b/PrivaPub/Domain/Social/FollowService.cs index 4820f06..05eba12 100644 --- a/PrivaPub/Domain/Social/FollowService.cs +++ b/PrivaPub/Domain/Social/FollowService.cs @@ -119,6 +119,7 @@ namespace PrivaPub.Domain.Social var following = new Following { AvatarId = follower.Id, + FollowerKind = follower.Kind == LocalActorKind.Person ? default(LocalActorKind?) : follower.Kind, TargetActorURI = targetUri, TargetAccountId = local?.Id ?? remote.ID, TargetIsLocal = local != default, @@ -298,7 +299,7 @@ namespace PrivaPub.Domain.Social .ExecuteAsync(token); foreach (var following in pending.Where(f => (f.ResentAt ?? f.CreatedAt) + ResendAfter[f.Resent] <= now)) { - var follower = await _localActors.FindById(LocalActorKind.Person, following.AvatarId, token); + var follower = await _localActors.FindById(following.FollowerKind ?? LocalActorKind.Person, following.AvatarId, token); var inbox = following.TargetInboxURL ?? (await _remoteActors.GetActor(following.TargetActorURI, refresh: false, token))?.InboxURL; if (follower != default && !string.IsNullOrEmpty(inbox)) { diff --git a/PrivaPub/Domain/Statuses/StatusService.cs b/PrivaPub/Domain/Statuses/StatusService.cs index 4306c47..2b63a32 100644 --- a/PrivaPub/Domain/Statuses/StatusService.cs +++ b/PrivaPub/Domain/Statuses/StatusService.cs @@ -64,6 +64,7 @@ namespace PrivaPub.Domain.Statuses Task Edit(LocalActor author, string postId, StatusDraft draft, CancellationToken token); Task Remove(LocalActor author, string postId, CancellationToken token); Task Favourite(LocalActor me, string postId, bool on, CancellationToken token); + Task Downvote(LocalActor me, string postId, bool on, CancellationToken token); Task Reblog(LocalActor me, string postId, bool on, PostVisibility visibility, CancellationToken token); } @@ -438,6 +439,9 @@ namespace PrivaPub.Domain.Statuses var likePermission = on ? await Permission(post, me, InteractionKind.Like, token) : QuotePermission.Granted; if (likePermission == QuotePermission.Denied) return StatusOutcome.Fail(StatusCodes.Status422UnprocessableEntity, "Validation failed: Its author does not take likes from you"); + // one vote a post: an upvote replaces the downvote, here and, by its Like alone, there + if (on) + await Forget(d => d.PostId == post.ID && d.ActorURI == me.Uri, post, p => p.DownvotesCount, token); // each favourite is a Like of its own, as Mastodon's are: a favourite after an unfavourite is a new Like, which no // server (and no delivery queue) takes for the one already undone; an unfavourite undoes the Like it ends @@ -472,7 +476,7 @@ namespace PrivaPub.Domain.Statuses post.FavouritesCount--; } - if (post.IsFederatedCopy && await AuthorInbox(post, token) is { } inbox) + if (post.IsFederatedCopy && await VoteInboxes(post, token) is { Count: > 0 } inboxes) { var like = new JsonObject { @@ -485,11 +489,95 @@ namespace PrivaPub.Domain.Statuses if (on && favourite.Approval == ApprovalState.Pending) await _approvals.Ask(me, post, InteractionKind.Like, like, favourite.ID, token); else - await _delivery.Enqueue(me, new[] { inbox }, on ? like : Undo(me, like, $"undo-like-{favourite.ID}"), token); + await _delivery.Enqueue(me, inboxes, on ? like : Undo(me, like, $"undo-like-{favourite.ID}"), token); } return new StatusOutcome(post); } + // A downvote, as the threadiverse counts them (Lemmy, PieFed): a `Dislike` to the post's author and its community, + // `Undo{Dislike}` to take it back. One vote a post: a downvote replaces the upvote by its Dislike alone, as Lemmy + // sends a changed vote, since an Undo{Like} sent beside it could arrive after it and take the downvote away (Mbin, + // which keeps a favourite apart, keeps it). Public, as a favourite is (owner decision: the client tells each + // persona once). + public async Task Downvote(LocalActor me, string postId, bool on, CancellationToken token) + { + var post = await Visible(me, postId, token); + if (post?.ReblogOfPostId != default) + post = await Visible(me, post.ReblogOfPostId, token); + if (post == default) + return StatusOutcome.Fail(StatusCodes.Status404NotFound, "Record not found"); + + Downvote downvote; + if (on) + { + await Forget(f => f.AccountId == me.Id && f.PostId == post.ID, post, p => p.FavouritesCount, token); + downvote = new Downvote { ID = ObjectId.GenerateNewId().ToString(), AccountId = me.Id, ActorURI = me.Uri, PostId = post.ID }; + downvote.ActivityURI = me.ActivityUri($"dislike-{downvote.ID}"); + try + { + await DB.Default.SaveAsync(downvote, token); + } + catch (MongoWriteException ex) when (ex.WriteError?.Category == ServerErrorCategory.DuplicateKey) + { + return new StatusOutcome(post); + } + await DB.Default.Update().MatchID(post.ID).Modify(b => b.Inc(p => p.DownvotesCount, 1)).ExecuteAsync(token); + post.DownvotesCount++; + } + else + { + downvote = await DB.Default.Find().Match(d => d.PostId == post.ID && d.ActorURI == me.Uri).ExecuteFirstAsync(token); + if (downvote == default || (await DB.Default.DeleteAsync(downvote.ID)).DeletedCount == 0) + return new StatusOutcome(post); + await DB.Default.Update().MatchID(post.ID).Modify(b => b.Inc(p => p.DownvotesCount, -1)).ExecuteAsync(token); + post.DownvotesCount--; + } + + if (post.IsFederatedCopy && await VoteInboxes(post, token) is { Count: > 0 } inboxes) + { + var dislike = new JsonObject + { + ["@context"] = ActivityPubRenderer.ActivityStreams, + ["id"] = downvote.ActivityURI, + ["type"] = "Dislike", + ["actor"] = me.Uri, + ["object"] = post.ObjectURI + }; + await _delivery.Enqueue(me, inboxes, on ? dislike : Undo(me, dislike, $"undo-dislike-{downvote.ID}"), token); + } + return new StatusOutcome(post); + } + + // the persona's other vote on the post, dropped here alone: the vote that replaces it says so there + async Task Forget(System.Linq.Expressions.Expression> mine, PostEntity post, + System.Linq.Expressions.Expression> count, CancellationToken token) where TVote : Entity + { + var vote = await DB.Default.Find().Match(mine).ExecuteFirstAsync(token); + if (vote == default || (await DB.Default.DeleteAsync(vote.ID)).DeletedCount == 0) + return; + await DB.Default.Update().MatchID(post.ID).Modify(b => b.Inc(count, -1)).ExecuteAsync(token); + if (typeof(TVote) == typeof(Favourite)) + post.FavouritesCount--; + else + post.DownvotesCount--; + } + + // where a vote on a remote post goes: the community it was made in, which counts the votes and passes them on, as + // Lemmy sends one; and its author when on another server (a Mastodon account's post in a Lemmy community). One copy a + // server: PieFed drops the second it is sent within a minute, though its first may yet fail. + async Task> VoteInboxes(PostEntity post, CancellationToken token) + { + var inboxes = new List(); + if (await Federation.Inbox.Communities.Of(post, _dbEntities, token) is { } communityUri + && await _dbEntities.ForeignAvatars.Match(f => f.ActorURI == communityUri && f.AvatarType == AvatarType.Group).ExecuteFirstAsync(token) is { } community + && !string.IsNullOrEmpty(community.InboxURL)) + inboxes.Add(community.InboxURL); + if (await AuthorInbox(post, token) is { } author + && !inboxes.Any(i => string.Equals(new Uri(i).Authority, new Uri(author).Authority, StringComparison.OrdinalIgnoreCase))) + inboxes.Add(author); + return inboxes; + } + // how a remote post's interaction policy takes an interaction of ours: at once, once asked, or not at all async Task Permission(PostEntity target, LocalActor actor, InteractionKind kind, CancellationToken token) => target is { IsFederatedCopy: true } && _approvals != default ? await _approvals.Judge(target, actor, kind, token) : QuotePermission.Granted; diff --git a/PrivaPub/Domain/Timelines/Fanout.cs b/PrivaPub/Domain/Timelines/Fanout.cs index 807b5e5..cd0c9e5 100644 --- a/PrivaPub/Domain/Timelines/Fanout.cs +++ b/PrivaPub/Domain/Timelines/Fanout.cs @@ -40,13 +40,14 @@ namespace PrivaPub.Domain.Timelines { case PostVisibility.Public or PostVisibility.Unlisted or PostVisibility.FollowersOnly: var followers = post.IsFederatedCopy - ? await _dbEntities.Followings.Match(f => f.TargetActorURI == post.ActorURI && f.State == FollowState.Accepted).ExecuteAsync(token) + ? await _dbEntities.Followings.Match(f => f.TargetActorURI == post.ActorURI && f.State == FollowState.Accepted && f.FollowerKind == null).ExecuteAsync(token) : await _dbEntities.Followings.Match(f => f.TargetAccountId == post.GroupUserId && f.TargetIsLocal && f.State == FollowState.Accepted).ExecuteAsync(token); foreach (var following in followers) if (await Shows(following, post, token)) recipients.Add(following.AvatarId); if (!string.IsNullOrEmpty(post.AudienceURI)) - foreach (var member in await _dbEntities.Followings.Match(f => f.TargetActorURI == post.AudienceURI && f.State == FollowState.Accepted).ExecuteAsync(token)) + // (the server's reader of feeds follows communities for nobody's home) + foreach (var member in await _dbEntities.Followings.Match(f => f.TargetActorURI == post.AudienceURI && f.State == FollowState.Accepted && f.FollowerKind == null).ExecuteAsync(token)) recipients.Add(member.AvatarId); // a post on a wall (FEP-400e): its owner, when a persona, and those following the owner if (!string.IsNullOrEmpty(post.WallOwnerURI)) diff --git a/PrivaPub/Federation/Actors/AccountCountsJob.cs b/PrivaPub/Federation/Actors/AccountCountsJob.cs index 0d001a1..7471dec 100644 --- a/PrivaPub/Federation/Actors/AccountCountsJob.cs +++ b/PrivaPub/Federation/Actors/AccountCountsJob.cs @@ -19,16 +19,18 @@ namespace PrivaPub.Federation.Actors readonly DbEntities _dbEntities; readonly IFeaturedPosts _featured; readonly ILocalActorService _localActors; + readonly IJobQueue _jobs; public const int MaxFeedCommunities = 50; public AccountCountsJob(IRemoteActorService remoteActors, DbEntities dbEntities, IFeaturedPosts featured = default, - ILocalActorService localActors = default) + ILocalActorService localActors = default, IJobQueue jobs = default) { _remoteActors = remoteActors; _dbEntities = dbEntities; _featured = featured; _localActors = localActors; + _jobs = jobs; } public JobKind Kind => JobKind.CountAccount; @@ -44,19 +46,24 @@ namespace PrivaPub.Federation.Actors if (actor == default) return JobOutcome.Done; var feed = actor.AvatarType == AvatarType.Feed; - var (following, communities) = await Read(actor, actor.FollowingURL, feed, token); + var (following, communities, listed) = await Read(actor, actor.FollowingURL, feed, token); var update = DB.Default.Update().MatchID(actor.ID) .Modify(a => a.FollowersCount, (await Read(actor, actor.FollowersURL, false, token)).Total) .Modify(a => a.FollowingCount, following) .Modify(a => a.StatusesCount, (await Read(actor, actor.OutboxURL, false, token)).Total) .Modify(a => a.CountedAt, DateTime.UtcNow); - if (feed) + // (a feed whose list could not be read keeps the one it had: the reader of feeds would unfollow them all) + if (feed && listed) update = update.Modify(a => a.FeedCommunities, communities); await update.ExecuteAsync(token); - if (feed) + if (feed && listed) + { foreach (var community in communities) if (_localActors == default || !community.StartsWith(_localActors.BaseAddress + "/", StringComparison.Ordinal)) await _remoteActors.GetActor(community, refresh: false, token); + if (_jobs != default && await DB.Default.Find().Match(s => s.FeedActorURI == actor.ActorURI).ExecuteAnyAsync(token)) + await Domain.Social.FeedFollows.Sync(_jobs, token); + } if (_featured != default) await _featured.Sync(actor, token); return JobOutcome.Done; @@ -64,31 +71,33 @@ namespace PrivaPub.Federation.Actors // only from the actor's own server, as everything we believe about it: its total, and the ids it lists when asked // (inline, or on its first page) - async Task<(int? Total, List Items)> Read(ForeignAvatar actor, string collection, bool items, CancellationToken token) + async Task<(int? Total, List Items, bool Listed)> Read(ForeignAvatar actor, string collection, bool items, CancellationToken token) { var listed = new List(); if (string.IsNullOrEmpty(collection) || !Origin.Same(collection, actor.ActorURI)) - return (default, listed); + return (default, listed, false); using var scope = HttpScope.For("collection"); using var fetched = await _remoteActors.FetchObject(collection, token); if (fetched == default || fetched.Root.ValueKind != System.Text.Json.JsonValueKind.Object) - return (default, listed); + return (default, listed, false); int? total = fetched.Root.TryGetProperty("totalItems", out var counted) && counted.TryGetInt32(out var count) && count >= 0 ? count : default; if (!items) - return (total, listed); + return (total, listed, true); var document = System.Text.Json.Nodes.JsonNode.Parse(fetched.Root.GetRawText()); var page = document["orderedItems"] ?? document["items"] ?? document["first"]?["orderedItems"] ?? document["first"]?["items"]; if (page == default && ActivityJson.Id(document["first"]) is { } first && Origin.Same(first, actor.ActorURI)) { using var next = await _remoteActors.FetchObject(first, token); - var nextPage = next == default ? default : System.Text.Json.Nodes.JsonNode.Parse(next.Root.GetRawText()); + if (next == default) + return (total, listed, false); + var nextPage = System.Text.Json.Nodes.JsonNode.Parse(next.Root.GetRawText()); page = nextPage?["orderedItems"] ?? nextPage?["items"]; } if (page is System.Text.Json.Nodes.JsonArray array) listed = array.Select(ActivityJson.Id) .Where(id => Uri.TryCreate(id, UriKind.Absolute, out var uri) && uri.Scheme is "https" or "http") .Distinct(StringComparer.Ordinal).Take(MaxFeedCommunities).ToList(); - return (total, listed); + return (total, listed, true); } } } diff --git a/PrivaPub/Federation/Actors/LocalActorService.cs b/PrivaPub/Federation/Actors/LocalActorService.cs index 8b5dc63..7e2a963 100644 --- a/PrivaPub/Federation/Actors/LocalActorService.cs +++ b/PrivaPub/Federation/Actors/LocalActorService.cs @@ -51,7 +51,7 @@ namespace PrivaPub.Federation.Actors public string Wall => $"{Uri}/graffiti"; public bool HasWall => Kind == LocalActorKind.Person && !IsCircle && IsFederated; // the server's own actors (the instance actor, the reporter): nobody follows, mentions or looks them up as accounts - public bool IsServerActor => Kind is LocalActorKind.Application or LocalActorKind.Reporter; + public bool IsServerActor => Kind is LocalActorKind.Application or LocalActorKind.Reporter or LocalActorKind.Reader; public string Flock => $"{Uri}/flock"; public string Wardens => $"{Uri}/wardens"; public string SharedInbox => $"{BaseAddress}/human-centipede"; @@ -74,6 +74,7 @@ namespace PrivaPub.Federation.Actors Task FindByAddress(string address, CancellationToken token); Task GetInstanceActor(CancellationToken token); Task GetReporterActor(CancellationToken token); + Task GetReaderActor(CancellationToken token); Task IsUserNameTaken(string userName, CancellationToken token); Task TryReserveUserName(string userName, LocalActorKind kind, string ownerId, CancellationToken token); LocalActor FromAvatar(Avatar avatar); @@ -89,13 +90,17 @@ namespace PrivaPub.Federation.Actors // the anonymous Service that carries this server's reports to Lemmy, which takes no report from an Application // (owner decision 2026-10-06): one actor for the server, never one per persona public const string ReporterUserName = "privapub_reports"; + // the anonymous Service that follows the communities of the feeds personas read, so no community learns which persona + // reads it (owner decision 2026-10-07); a Service, since Lemmy takes a follow from no Application + public const string ReaderUserName = "privapub_feeds"; public static bool IsServerActorName(string userName) => - string.Equals(userName, InstanceUserName, StringComparison.OrdinalIgnoreCase) || string.Equals(userName, ReporterUserName, StringComparison.OrdinalIgnoreCase); + string.Equals(userName, InstanceUserName, StringComparison.OrdinalIgnoreCase) || string.Equals(userName, ReporterUserName, StringComparison.OrdinalIgnoreCase) + || string.Equals(userName, ReaderUserName, StringComparison.OrdinalIgnoreCase); static readonly HashSet ReservedByInstance = new(StringComparer.Ordinal) { - InstanceUserName, ReporterUserName, "admin", "administrator", "root", "system", "support", "help", "moderator", "mod", + InstanceUserName, ReporterUserName, ReaderUserName, "admin", "administrator", "root", "system", "support", "help", "moderator", "mod", "abuse", "postmaster", "webmaster", "hostmaster", "security", "noreply", "no_reply", "null", "undefined" }; @@ -103,6 +108,7 @@ namespace PrivaPub.Federation.Actors readonly IOptionsMonitor _appConfiguration; InstanceActor _instanceActor; ReporterActor _reporterActor; + ReaderActor _readerActor; public LocalActorService(DbEntities dbEntities, IOptionsMonitor appConfiguration) { @@ -121,6 +127,8 @@ namespace PrivaPub.Federation.Actors return await GetInstanceActor(token); if (userName == ReporterUserName) return await GetReporterActor(token); + if (userName == ReaderUserName) + return await GetReaderActor(token); var avatar = await _dbEntities.Avatars .Match(a => a.UserName == userName && !a.DeletionAt.HasValue) @@ -158,6 +166,8 @@ namespace PrivaPub.Federation.Actors return group == default ? default : FromGroup(group); case LocalActorKind.Reporter: return await GetReporterActor(token); + case LocalActorKind.Reader: + return await GetReaderActor(token); default: return await GetInstanceActor(token); } @@ -221,6 +231,36 @@ namespace PrivaPub.Federation.Actors }; } + public async Task GetReaderActor(CancellationToken token) + { + var reader = _readerActor ??= await LoadReaderActor(token); + + return new LocalActor + { + Id = reader.ID, + Kind = LocalActorKind.Reader, + UserName = ReaderUserName, + Name = $"Feeds read on {new Uri(BaseAddress).Host}", + Summary = "It follows the communities in the feeds this PrivaPub server's people read; it never names who reads them.", + PrivateKeyPem = reader.PrivateKey, + PublicKeyPem = reader.PublicKey, + Discoverable = false, + Published = reader.CreationDate, + BaseAddress = BaseAddress + }; + } + + async Task LoadReaderActor(CancellationToken token) + { + var reader = await _dbEntities.ReaderActors.Sort(r => r.CreationDate, Order.Ascending).ExecuteFirstAsync(token); + if (reader != default) + return reader; + var (privateKey, publicKey) = Keys.NewKeyPair(); + reader = new ReaderActor { PrivateKey = privateKey, PublicKey = publicKey }; + await DB.Default.SaveAsync(reader, token); + return reader; + } + async Task LoadReporterActor(CancellationToken token) { var reporter = await _dbEntities.ReporterActors.Sort(r => r.CreationDate, Order.Ascending).ExecuteFirstAsync(token); diff --git a/PrivaPub/Federation/Controllers/PeasantsController.cs b/PrivaPub/Federation/Controllers/PeasantsController.cs index 2e48cbf..57bdf49 100644 --- a/PrivaPub/Federation/Controllers/PeasantsController.cs +++ b/PrivaPub/Federation/Controllers/PeasantsController.cs @@ -532,8 +532,8 @@ namespace PrivaPub.Federation.Controllers if (HttpMethods.IsGet(Request.Method)) Response.Headers.Vary = "Accept"; // SecureMode asks every reader of ActivityPub documents for a signature, except for the server's own actors (the - // instance actor, the reporter), whose keys peers need first, and except for browsers, which only get redirected to - // the public pages + // instance actor, the reporter, the reader of feeds), whose keys peers need first, and except for browsers, which + // only get redirected to the public pages if (_federation.CurrentValue.SecureMode && HttpMethods.IsGet(Request.Method) && !WantsHtml() && Request.Path.Value != "/" && !LocalActorService.IsServerActorName(context.RouteData.Values["actor"] as string) && await _fetches.Requester(Request, HttpContext.RequestAborted) == default) diff --git a/PrivaPub/Federation/Inbox/Arrival.cs b/PrivaPub/Federation/Inbox/Arrival.cs index 30d18c4..d824109 100644 --- a/PrivaPub/Federation/Inbox/Arrival.cs +++ b/PrivaPub/Federation/Inbox/Arrival.cs @@ -67,6 +67,7 @@ namespace PrivaPub.Federation.Inbox LocalActorKind.Group when !local.IsCircle => "group", LocalActorKind.Application => "application", LocalActorKind.Reporter => "reporter", + LocalActorKind.Reader => "reader", _ => default }; } diff --git a/PrivaPub/Federation/Outbox/DeliveryService.cs b/PrivaPub/Federation/Outbox/DeliveryService.cs index 9ff3a95..dbeccf8 100644 --- a/PrivaPub/Federation/Outbox/DeliveryService.cs +++ b/PrivaPub/Federation/Outbox/DeliveryService.cs @@ -182,6 +182,7 @@ namespace PrivaPub.Federation.Outbox { Kind: LocalActorKind.Person } => "person", { Kind: LocalActorKind.Group } => "group", { Kind: LocalActorKind.Reporter } => "reporter", + { Kind: LocalActorKind.Reader } => "reader", _ => "application" }, Signature = "cavage:rsa-sha256" diff --git a/PrivaPub/Federation/Rendering/ActivityPubRenderer.cs b/PrivaPub/Federation/Rendering/ActivityPubRenderer.cs index 95ed4e5..59f3164 100644 --- a/PrivaPub/Federation/Rendering/ActivityPubRenderer.cs +++ b/PrivaPub/Federation/Rendering/ActivityPubRenderer.cs @@ -126,7 +126,7 @@ namespace PrivaPub.Federation.Rendering { LocalActorKind.Group => "Group", LocalActorKind.Application => "Application", - LocalActorKind.Reporter => "Service", + LocalActorKind.Reporter or LocalActorKind.Reader => "Service", _ when actor.IsBot => "Service", _ => "Person" }, diff --git a/PrivaPub/Infrastructure/Data/Indexes.cs b/PrivaPub/Infrastructure/Data/Indexes.cs index 226901a..cb615c4 100644 --- a/PrivaPub/Infrastructure/Data/Indexes.cs +++ b/PrivaPub/Infrastructure/Data/Indexes.cs @@ -26,6 +26,7 @@ namespace PrivaPub.Infrastructure.Data await Plain(token, p => p.WallURI, p => p.ID);//a persona's wall (FEP-400e) await Plain(token, p => p.ConversationId, p => p.ID); await Plain(token, p => p.InReplyToURI); + await Plain(token, p => p.AudienceURI, p => p.ID);//a feed's timeline: the posts of its communities // who boosted what: a persona's boosts of the posts a page shows were a scan of every post under load // (tools/pasture/load.sh), and a post's boosts and replies are read for its counts and context await Plain(token, p => p.AuthorAccountId, p => p.ReblogOfPostId); @@ -167,6 +168,9 @@ namespace PrivaPub.Infrastructure.Data await Plain(token, r => r.PostId); await Unique(d => d.ObjectURI, Builders.Filter.Gt(d => d.ObjectURI, ""), token); await DB.Default.Index().Key(d => d.DeletedAt, KeyType.Ascending).Option(o => o.ExpireAfter = TombstoneLifetime).CreateAsync(token); + await DB.Default.Index().Key(s => s.AvatarId, KeyType.Ascending).Key(s => s.FeedActorURI, KeyType.Ascending) + .Option(o => o.Unique = true).CreateAsync(token); + await Plain(token, s => s.FeedActorURI); await Unique(c => c.URI, Builders.Filter.Gt(c => c.URI, ""), token); await DB.Default.Index().Key(c => c.ReadAt, KeyType.Ascending).Option(o => o.ExpireAfter = ThreadCollectionLifetime).CreateAsync(token); await DB.Default.Index().Key(d => d.PostId, KeyType.Ascending).Key(d => d.ActorURI, KeyType.Ascending).Option(o => o.Unique = true).CreateAsync(token); diff --git a/PrivaPub/Infrastructure/Data/Migrations/_017_the_reader_has_its_name.cs b/PrivaPub/Infrastructure/Data/Migrations/_017_the_reader_has_its_name.cs new file mode 100644 index 0000000..c15cca9 --- /dev/null +++ b/PrivaPub/Infrastructure/Data/Migrations/_017_the_reader_has_its_name.cs @@ -0,0 +1,24 @@ +using MongoDB.Entities; + +using PrivaPub.Federation.Actors; +using PrivaPub.Models.Federation; +using PrivaPub.Models.User; + +namespace PrivaPub.Infrastructure.Data.Migrations +{ + // the feeds reader's name (LocalActorService.ReaderUserName) is the server's, as the reporter's is: a persona or group + // already holding it would be hidden behind the reader, so the upgrade stops and says so + public class _017_the_reader_has_its_name : IMigration + { + public async Task UpgradeAsync() + { + var held = await DB.Default.Find().Match(r => r.Name == LocalActorService.ReaderUserName).ExecuteFirstAsync(); + if (held is { OwnerKind: LocalActorKind.Reader }) + return; + if (held != default) + throw new InvalidOperationException( + $"The name {LocalActorService.ReaderUserName} belongs to a {held.OwnerKind} ({held.OwnerId}); rename it before upgrading"); + await DB.Default.SaveAsync(new ReservedName { Name = LocalActorService.ReaderUserName, OwnerKind = LocalActorKind.Reader }); + } + } +} diff --git a/PrivaPub/Middleware/SocialPubConfigurations.cs b/PrivaPub/Middleware/SocialPubConfigurations.cs index cdb298d..9ce248e 100644 --- a/PrivaPub/Middleware/SocialPubConfigurations.cs +++ b/PrivaPub/Middleware/SocialPubConfigurations.cs @@ -120,6 +120,7 @@ namespace PrivaPub.Middleware .AddSingleton() .AddSingleton() .AddSingleton() + .AddSingleton() .AddSingleton() .AddSingleton() .AddSingleton() @@ -136,6 +137,7 @@ namespace PrivaPub.Middleware .AddSingleton() .AddHostedService() .AddHostedService() + .AddHostedService() .AddSingleton() .AddHostedService(); } @@ -234,6 +236,7 @@ namespace PrivaPub.Middleware .AddTransient() .AddTransient() .AddTransient() + .AddTransient() .AddTransient() .AddSingleton() .AddHttpContextAccessor() diff --git a/PrivaPub/Models/Federation/Follower.cs b/PrivaPub/Models/Federation/Follower.cs index d8c3f82..d00a11b 100644 --- a/PrivaPub/Models/Federation/Follower.cs +++ b/PrivaPub/Models/Federation/Follower.cs @@ -19,6 +19,7 @@ namespace PrivaPub.Models.Federation Person, Group, Application, - Reporter//the anonymous Service that carries reports to Lemmy (LocalActorService.ReporterUserName) + Reporter,//the anonymous Service that carries reports to Lemmy (LocalActorService.ReporterUserName) + Reader//the anonymous Service that follows the communities of the feeds personas read (LocalActorService.ReaderUserName) } } diff --git a/PrivaPub/Models/Federation/ReaderActor.cs b/PrivaPub/Models/Federation/ReaderActor.cs new file mode 100644 index 0000000..36ccbac --- /dev/null +++ b/PrivaPub/Models/Federation/ReaderActor.cs @@ -0,0 +1,12 @@ +using MongoDB.Entities; + +namespace PrivaPub.Models.Federation +{ + // the server's anonymous reader of feeds (LocalActorService.ReaderUserName), its keys + public class ReaderActor : Entity + { + public string PrivateKey { get; set; } + public string PublicKey { get; set; } + public DateTime CreationDate { get; set; } = DateTime.UtcNow; + } +} diff --git a/PrivaPub/Models/Jobs/Job.cs b/PrivaPub/Models/Jobs/Job.cs index c300619..661122d 100644 --- a/PrivaPub/Models/Jobs/Job.cs +++ b/PrivaPub/Models/Jobs/Job.cs @@ -38,7 +38,8 @@ namespace PrivaPub.Models.Jobs SynchronizeFollowing, ProcessMedia, ExportArchive, - ImportArchive + ImportArchive, + SyncFeeds } public enum JobState diff --git a/PrivaPub/Models/Social/FeedSubscription.cs b/PrivaPub/Models/Social/FeedSubscription.cs new file mode 100644 index 0000000..8b0ba33 --- /dev/null +++ b/PrivaPub/Models/Social/FeedSubscription.cs @@ -0,0 +1,14 @@ +using MongoDB.Entities; + +namespace PrivaPub.Models.Social +{ + // a persona reading a feed elsewhere (Lemmy's multi-community, PieFed's feed): the server's reader follows the feed's + // communities, never the persona (owner decision 2026-10-07) + public class FeedSubscription : Entity + { + public string AvatarId { get; set; } + public string FeedActorURI { get; set; } + public string FeedAccountId { get; set; }//ForeignAvatar.ID + public DateTime CreatedAt { get; set; } = DateTime.UtcNow; + } +} diff --git a/PrivaPub/Models/Social/Following.cs b/PrivaPub/Models/Social/Following.cs index 80274b7..582e2a7 100644 --- a/PrivaPub/Models/Social/Following.cs +++ b/PrivaPub/Models/Social/Following.cs @@ -4,7 +4,8 @@ namespace PrivaPub.Models.Social { public class Following : Entity { - public string AvatarId { get; set; }//the local follower + public string AvatarId { get; set; }//the local follower: a persona, or the server's reader of feeds (FollowerKind) + public PrivaPub.Models.Federation.LocalActorKind? FollowerKind { get; set; }//null for a persona public string TargetActorURI { get; set; } public string TargetAccountId { get; set; }//Avatar.ID, Group.ID or ForeignAvatar.ID public bool TargetIsLocal { get; set; } diff --git a/PrivaPub/Services/RootRemoval.cs b/PrivaPub/Services/RootRemoval.cs index e35657f..b7113c7 100644 --- a/PrivaPub/Services/RootRemoval.cs +++ b/PrivaPub/Services/RootRemoval.cs @@ -69,6 +69,8 @@ namespace PrivaPub.Services await DB.Default.DeleteAsync(l => l.AvatarId == avatar.ID); await DB.Default.DeleteAsync(f => f.AvatarId == avatar.ID); await DB.Default.DeleteAsync(t => t.AvatarId == avatar.ID); + // (the reader of feeds stops following what nobody reads any more at its next round) + await DB.Default.DeleteAsync(s => s.AvatarId == avatar.ID); // nothing of it publishes later, and none of its files stays served: its posts' media, its pictures and // what its scheduled posts held await DB.Default.DeleteAsync(s => s.AvatarId == avatar.ID); diff --git a/PrivaPub/StaticServices/DbEntities.cs b/PrivaPub/StaticServices/DbEntities.cs index 71c7418..47dfec3 100644 --- a/PrivaPub/StaticServices/DbEntities.cs +++ b/PrivaPub/StaticServices/DbEntities.cs @@ -34,6 +34,7 @@ namespace PrivaPub.StaticServices public Find Deliveries { get { return DB.Default.Find(); } } public Find InstanceActors { get { return DB.Default.Find(); } } public Find ReporterActors { get { return DB.Default.Find(); } } + public Find ReaderActors { get { return DB.Default.Find(); } } public Find Followings { get { return DB.Default.Find(); } } public Find TimelineEntries { get { return DB.Default.Find(); } } diff --git a/docs/INTEROP.md b/docs/INTEROP.md index 1988d72..1d97adc 100644 --- a/docs/INTEROP.md +++ b/docs/INTEROP.md @@ -496,7 +496,10 @@ on a Page: pins live in `featured`, locks in `Lock`. shared inbox and the **instance's key** (`owner` is the instance, not the feed), text in `description` (not `summary`), the creator in `attributedTo`, and `following`, an unpaged `Collection` of the community ids with `totalItems`. Following one stays on the follower's server, which subscribes to each community itself (checked 2026-10-07 on - 1.0.0-beta.2). + 1.0.0-beta.2). PrivaPub does the same through its reader of feeds, the `Service` `privapub_feeds`: Lemmy 1.0 takes its + `Follow` and answers `Accept` about 30 s later, and announces the community's threads to it; the feed's thread and the + persona's own posts in a community of ours the feed holds show in the persona's feed timeline (checked live + 2026-10-08). **Expects** - **Fetched documents:** Content-Type exactly one of `activity+json`, `activity+json; charset=utf-8`, or `ld+json` with @@ -604,7 +607,10 @@ What it showed: - `Feed` actors at `/f/` with their own key, inbox, `followers`, `moderators` (the owner) and `following` (the communities, a `Collection` with `totalItems`); PieFed's handle is `~name@host`. Following one is a `Follow` to the feed, after which its owner's server sends `Add`/`Remove` of communities, signed by the feed; each member also - subscribes to every community as themselves (`feed_auto_follow`). Its API creates feeds (`POST /api/alpha/feed`); + subscribes to every community as themselves (`feed_auto_follow`). Its API creates feeds (`POST /api/alpha/feed`). + PrivaPub follows a feed through its reader of feeds instead (owner decision 2026-10-07), sending the feed nothing: + PieFed takes the reader's `Follow` of each community at once and its threads reach the feed timeline, a moved + thread included (checked live 2026-10-08); - emoji Likes and `EmojiReact` count as upvotes. - **How PieFed formats what it sends us:** - It **chooses the format by our NodeInfo software name**, so keep that accurate. diff --git a/docs/ROADMAP.md b/docs/ROADMAP.md index 3e679de..c112ba2 100644 --- a/docs/ROADMAP.md +++ b/docs/ROADMAP.md @@ -221,6 +221,7 @@ and circles (see Owner decisions). | Reactions and votes | **Public, with a one-time notice.** The client tells each persona once, before its first reaction or vote, that these are public and visible as that persona. | | Who may quote a persona (default) | **Anyone, automatically**, as on Mastodon, for public and unlisted posts only. Each persona can change its default (followers only, or nobody) and each post can be changed; a granted quote can be revoked. Followers-only posts and DMs can never be quoted. | | Quote permission address | `/peasants/{name}/parrot-licences/{id}` | +| Following a feed (2026-10-07) | **Through the server.** When a persona follows a feed (Lemmy's multi-community, PieFed's feed), the server's own account subscribes to each community in it, as Lemmy does, so the communities and the feed's owner learn only that someone on this server reads them, never which persona. The persona reads the feed's posts in a feed timeline of its own. A persona that comments in one of those communities is still seen there as itself. | ### Owner decisions on statistics (2026-10-03) @@ -694,11 +695,15 @@ it, raw where it doesn't. ban of a persona refuses its posts there and shows as `blocked_by`), featured; `Warn` and `Resolve` (**done 2026-10-07**: a warning becomes the persona's `moderation_warning` notification, believed from the community's server or its moderators collection, kept as `ForeignAvatar.ModeratorsURL`; a resolution is kept on our report); - - votes in and out; link posts; flairs (**done 2026-10-07**: Lemmy 1.0's `CommunityPostTag` and PieFed's + - votes in and out (**done 2026-10-07**: a persona's downvote is a `Dislike`, one vote a post, a changed vote sent as + the new one alone; a vote on a community's post goes to the community, and to its author only on another server; Mbin keeps a favourite apart from a vote, so + there a favourite stays after a switch to a downvote); link posts; flairs (**done 2026-10-07**: Lemmy 1.0's `CommunityPostTag` and PieFed's `lemmy:CommunityTag`, on posts and communities, described from the community's list, as `privapub.flairs`); `Feed` actors (**read 2026-10-07**: Lemmy 1.0's and PieFed's feeds resolve as accounts whose `/following` is their - communities, kept from the feed's `following` once a day; following one is refused for now, **owner decision - pending** on how); community polls; post `Move` (**done 2026-10-07**: PieFed's, relayed by the community the post + communities, kept from the feed's `following` once a day; **followed through the server since 2026-10-07** (owner + decision, see "Owner decisions on what PrivaPub reveals"): the Service `privapub_feeds` follows the communities of + every feed read here, reconciled when a persona follows or leaves one, when a feed's list is read again and every six + hours, and the persona reads `GET /api/v1/timelines/feed/:id`); community polls; post `Move` (**done 2026-10-07**: PieFed's, relayed by the community the post leaves, moves the post and its thread; checked live); - the outbound shape Lemmy requires; - communities we host announce to the author's own instance too: **done 2026-10-06** (its server's shared inbox,