T8: inbox gaps, jobs, migrations and pages; a deleted remote account's posts are hidden

Owner decision (2026-10-03, "A remote account deletes itself"): its posts are kept but
hidden everywhere.
- Post.AuthorGone (additive bool). DeleteHandler's actor-delete branch sets it on every
  post whose ActorURI is the actor (one update-many), besides dropping its follows and
  timeline rows as before. RemotePosts.Build sets it on a post stored later for an
  account already marked Deleted.
- One rule in VisibilityPolicy: IsShown (not deleted, author not gone), IsPublic and
  CanSee exclude AuthorGone, plus Shown(post) for loaded posts.
- Lookups by id answer 404 through CanSee (statuses/:id and every sub-route, context,
  bookmarks, favourites, polls, reactions, search); provenance, account statuses,
  home/public/tag timelines, notifications, conversations, reblogged_by, the clientapi
  home and post/DM lists, a community's outbox and our Announces filter on IsShown or
  IsPublic; the Mastodon mapper never renders a hidden post or a boost of one.

Tests (30 new):
- AuthorGoneTests: the rule, the handler (posts kept, boosts included, follows and rows
  gone), a post fetched after the delete, and 20 Mastodon/ActivityPub lookups over HTTP
  seen before and hidden after.
- InboxGapTests: actor Update refresh (name, sanitised summary, key rotation in place
  and to a new key id) even with an older `updated`; Undo{Follow} by activity id and by
  object; Reject of our QuoteRequest (and a stranger's ignored); group-wrapped
  Announce{Like} and Announce{Undo{Like}}; a locked persona's pending follow,
  FollowRequest notification, and Decide accepting and rejecting with the original Follow.
- JobHandlerTests: AncestorsJobHandler up to its depth limit; PollRefreshJob and
  PollCloseJob (local and remote polls); InstanceDescriber from a peer's NodeInfo and
  the weekly dedupe through ObjectRecords; LinkPreviews for public posts only;
  DeliveryJobHandler outcomes (2xx, 404/410, 429/503 with Retry-After in seconds and as
  a date, 5xx) and a signature and Digest the peer can verify; MediaJanitor.Sweep;
  OAuthPruner.Prune.
- MigrationTests: _003, _004, _006 and _007 on seeded rows.
- PublicPagesTests: /@user and /@user/{id} (visibility, junk ids, exact CSP,
  Referrer-Policy and nosniff), circle 404, community page, the instance actor,
  ActivityPub redirects, and markup escaped in posts, titles and bios.

Production changes besides the rule:
- LinkPreviews.Handle re-checks that a post is still shown and public (the rule
  Wanted applies) before fetching anything; before, only enqueueing checked it.
- The legacy /clientapi post and DM lists no longer return soft-deleted posts.
- MediaJanitor.Sweep and OAuthPruner.Prune are the loop bodies, now public and tested.
- InstanceDescriber.Address: a protected virtual identity seam so a test can point
  the https NodeInfo addresses at a plain-http peer; production behaviour unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ELjqpznMFMNrJoJUj6K5p2

PrivaPub/Api/Mastodon/Auth/OAuthPruner.cs

@@ -23,10 +23,7 @@ namespace PrivaPub.Api.Mastodon.Auth
 				try
 				{
 					await Task.Delay(Interval, stoppingToken);
-					using var scope = _services.CreateScope();
-					var threshold = DateTimeOffset.UtcNow - Threshold;
-					var tokens = await scope.ServiceProvider.GetRequiredService<IOpenIddictTokenManager>().PruneAsync(threshold, stoppingToken);
-					var authorizations = await scope.ServiceProvider.GetRequiredService<IOpenIddictAuthorizationManager>().PruneAsync(threshold, stoppingToken);
+					var (tokens, authorizations) = await Prune(DateTimeOffset.UtcNow - Threshold, stoppingToken);
 					if (tokens + authorizations > 0)
 						_logger.LogInformation("{Service} removed {Tokens} tokens and {Authorizations} authorizations", nameof(OAuthPruner), tokens, authorizations);
 				}
@@ -40,5 +37,14 @@ namespace PrivaPub.Api.Mastodon.Auth
 				}
 			}
 		}
+
+		//one pass: tokens and authorizations created before the threshold that are no longer valid
+		public async Task<(long Tokens, long Authorizations)> Prune(DateTimeOffset threshold, CancellationToken token)
+		{
+			using var scope = _services.CreateScope();
+			var tokens = await scope.ServiceProvider.GetRequiredService<IOpenIddictTokenManager>().PruneAsync(threshold, token);
+			var authorizations = await scope.ServiceProvider.GetRequiredService<IOpenIddictAuthorizationManager>().PruneAsync(threshold, token);
+			return (tokens, authorizations);
+		}
 	}
 }

PrivaPub/Api/Mastodon/Controllers/AccountsController.cs

@@ -155,13 +155,13 @@ namespace PrivaPub.Api.Mastodon.Controllers
 				if (local == default)
 					return Json(Array.Empty<Status>());
 				var pinIds = (await DB.Default.Find<Pin>().Match(p => p.AvatarId == local.Id).Sort(p => p.ID, Order.Descending).ExecuteAsync(token)).Select(p => p.PostId).ToList();
-				var pinnedPosts = await _dbEntities.Posts.Match(p => pinIds.Contains(p.ID) && !p.DeletedAt.HasValue).Match(VisibilityPolicy.IsPublic).ExecuteAsync(token);
+				var pinnedPosts = await _dbEntities.Posts.Match(p => pinIds.Contains(p.ID)).Match(VisibilityPolicy.IsPublic).ExecuteAsync(token);
 				return Json(await _mapper.Statuses(pinIds.Select(id => pinnedPosts.FirstOrDefault(p => p.ID == id)).Where(p => p != default).ToList(), MyId, token));
 			}
 
 			var query = local != default
-				? _dbEntities.Posts.Match(p => p.GroupUserId == local.Id && !p.IsFederatedCopy && !p.DeletedAt.HasValue)
-				: _dbEntities.Posts.Match(p => p.ActorURI == remote.ActorURI && p.IsFederatedCopy && !p.DeletedAt.HasValue);
+				? _dbEntities.Posts.Match(p => p.GroupUserId == local.Id && !p.IsFederatedCopy).Match(VisibilityPolicy.IsShown)
+				: _dbEntities.Posts.Match(p => p.ActorURI == remote.ActorURI && p.IsFederatedCopy).Match(VisibilityPolicy.IsShown);
 			var viewerFollows = MyId != default && local != default
 				&& await _dbEntities.Followings.Match(f => f.AvatarId == MyId && f.TargetAccountId == local.Id && f.State == FollowState.Accepted).ExecuteAnyAsync(token);
 			if (MyId != id)

PrivaPub/Api/Mastodon/Controllers/ProvenanceController.cs

@@ -26,9 +26,9 @@ namespace PrivaPub.Api.Mastodon.Controllers
 		[HttpGet("/api/privapub/v1/statuses/{id}/provenance"), Scope("read:statuses", requiresUser: false), AllowAnonymous]
 		public async Task<IActionResult> Status(string id, CancellationToken token)
 		{
-			var post = await _dbEntities.Posts.Match(p => p.ID == id && !p.DeletedAt.HasValue).ExecuteFirstAsync(token);
+			var post = await _dbEntities.Posts.Match(p => p.ID == id).Match(VisibilityPolicy.IsShown).ExecuteFirstAsync(token);
 			if (post?.ReblogOfPostId != default)
-				post = await _dbEntities.Posts.Match(p => p.ID == post.ReblogOfPostId && !p.DeletedAt.HasValue).ExecuteFirstAsync(token);
+				post = await _dbEntities.Posts.Match(p => p.ID == post.ReblogOfPostId).Match(VisibilityPolicy.IsShown).ExecuteFirstAsync(token);
 			if (post == default || post.Visibility == PostVisibility.LocalGeo || !await VisibilityPolicy.CanSee(post, MyId, token))
 				return NotFoundError();
 			if (!post.IsFederatedCopy)

PrivaPub/Api/Mastodon/Controllers/StatusesController.cs

@@ -231,7 +231,7 @@ namespace PrivaPub.Api.Mastodon.Controllers
 		{
 			if (await Visible(id, token) == default)
 				return NotFoundError();
-			var reblogs = await _dbEntities.Posts.Match(p => p.ReblogOfPostId == id && !p.DeletedAt.HasValue).Sort(p => p.ID, Order.Descending).Limit(Limit(40, 80)).ExecuteAsync(token);
+			var reblogs = await _dbEntities.Posts.Match(p => p.ReblogOfPostId == id).Match(VisibilityPolicy.IsShown).Sort(p => p.ID, Order.Descending).Limit(Limit(40, 80)).ExecuteAsync(token);
 			var accounts = await _mapper.Accounts(reblogs.Select(MastodonMapper.AuthorOf), token);
 			return Json(reblogs.Select(r => accounts.GetValueOrDefault(MastodonMapper.AuthorOf(r))).Where(a => a != default).ToList());
 		}

PrivaPub/Api/Mastodon/Controllers/TimelinesController.cs

@@ -32,7 +32,7 @@ namespace PrivaPub.Api.Mastodon.Controllers
 		{
 			var entries = await Page.From(Params, Limit()).Fetch(_dbEntities.TimelineEntries.Match(e => e.AvatarId == MyId), e => e.PostId, token);
 			var ids = entries.Select(e => e.PostId).ToList();
-			var posts = (await _dbEntities.Posts.Match(p => ids.Contains(p.ID) && !p.DeletedAt.HasValue).ExecuteAsync(token)).ToDictionary(p => p.ID);
+			var posts = (await _dbEntities.Posts.Match(p => ids.Contains(p.ID)).Match(VisibilityPolicy.IsShown).ExecuteAsync(token)).ToDictionary(p => p.ID);
 			var statuses = await _mapper.Statuses(ids.Where(posts.ContainsKey).Select(id => posts[id]).ToList(), MyId, token);
 			Link("/api/v1/timelines/home", entries.LastOrDefault()?.PostId, entries.FirstOrDefault()?.PostId);
 			return Json(statuses);
@@ -41,7 +41,7 @@ namespace PrivaPub.Api.Mastodon.Controllers
 		[HttpGet("/api/v1/timelines/public"), Scope("read:statuses", requiresUser: false), Microsoft.AspNetCore.Authorization.AllowAnonymous]
 		public async Task<IActionResult> Public(CancellationToken token)
 		{
-			var query = _dbEntities.Posts.Match(p => p.Visibility == PostVisibility.Public && !p.DeletedAt.HasValue && p.ReblogOfPostId == null);
+			var query = _dbEntities.Posts.Match(p => p.Visibility == PostVisibility.Public && p.ReblogOfPostId == null).Match(VisibilityPolicy.IsPublic);
 			if (Params.Bool("local") == true)
 				query.Match(p => !p.IsFederatedCopy);
 			if (Params.Bool("remote") == true)
@@ -55,7 +55,7 @@ namespace PrivaPub.Api.Mastodon.Controllers
 		public async Task<IActionResult> Tag(string hashtag, CancellationToken token)
 		{
 			var tag = hashtag.TrimStart('#').ToLowerInvariant();
-			var query = _dbEntities.Posts.Match(p => p.Tags.Contains(tag) && p.Visibility == PostVisibility.Public && !p.DeletedAt.HasValue && p.ReblogOfPostId == null);
+			var query = _dbEntities.Posts.Match(p => p.Tags.Contains(tag) && p.Visibility == PostVisibility.Public && p.ReblogOfPostId == null).Match(VisibilityPolicy.IsPublic);
 			if (Params.Bool("local") == true)
 				query.Match(p => !p.IsFederatedCopy);
 			return await Respond(query, $"/api/v1/timelines/tag/{hashtag}", token);
@@ -86,7 +86,7 @@ namespace PrivaPub.Api.Mastodon.Controllers
 			var conversations = new List<Conversation>();
 			foreach (var group in groups)
 			{
-				var last = await _dbEntities.Posts.Match(p => p.ConversationId == group.ID && !p.DeletedAt.HasValue).Sort(p => p.ID, Order.Descending).ExecuteFirstAsync(token);
+				var last = await _dbEntities.Posts.Match(p => p.ConversationId == group.ID).Match(VisibilityPolicy.IsShown).Sort(p => p.ID, Order.Descending).ExecuteFirstAsync(token);
 				if (last == default)
 					continue;
 				var others = new List<string>();
@@ -111,7 +111,7 @@ namespace PrivaPub.Api.Mastodon.Controllers
 			var group = await _dbEntities.DmGroups.Match(g => g.ID == id && g.Members.Any(m => !m.IsForeign && m.AvatarId == MyId)).ExecuteFirstAsync(token);
 			if (group == default)
 				return NotFoundError();
-			var last = await _dbEntities.Posts.Match(p => p.ConversationId == group.ID && !p.DeletedAt.HasValue).Sort(p => p.ID, Order.Descending).ExecuteFirstAsync(token);
+			var last = await _dbEntities.Posts.Match(p => p.ConversationId == group.ID).Match(VisibilityPolicy.IsShown).Sort(p => p.ID, Order.Descending).ExecuteFirstAsync(token);
 			return Json(new Conversation { Id = group.ID, LastStatus = last == default ? default : await _mapper.Status(last, MyId, token) });
 		}
 
@@ -225,7 +225,7 @@ namespace PrivaPub.Api.Mastodon.Controllers
 			var postIds = notifications.Where(n => n.PostId != default).Select(n => n.PostId).Distinct().ToList();
 			var posts = postIds.Count == 0
 				? new List<PostEntity>()
-				: await _dbEntities.Posts.Match(p => postIds.Contains(p.ID) && !p.DeletedAt.HasValue).ExecuteAsync(token);
+				: await _dbEntities.Posts.Match(p => postIds.Contains(p.ID)).Match(VisibilityPolicy.IsShown).ExecuteAsync(token);
 			var statuses = (await _mapper.Statuses(posts, MyId, token)).ToDictionary(s => s.Id);
 			return notifications
 				.Where(n => accounts.ContainsKey(n.FromAccountId ?? string.Empty) && (n.PostId == default || statuses.ContainsKey(n.PostId)))

PrivaPub/Api/Mastodon/Mappers/MastodonMapper.cs

@@ -157,7 +157,7 @@ namespace PrivaPub.Api.Mastodon.Mappers
 			var originalIds = posts.Where(p => p.ReblogOfPostId != default).Select(p => p.ReblogOfPostId).Distinct().ToList();
 			var originals = originalIds.Count == 0
 				? new Dictionary<string, PostEntity>()
-				: (await _dbEntities.Posts.Match(p => originalIds.Contains(p.ID) && !p.DeletedAt.HasValue).ExecuteAsync(token)).ToDictionary(p => p.ID);
+				: (await _dbEntities.Posts.Match(p => originalIds.Contains(p.ID)).Match(VisibilityPolicy.IsShown).ExecuteAsync(token)).ToDictionary(p => p.ID);
 			var all = posts.Concat(originals.Values).ToList();
 
 			var mentionUris = all.SelectMany(p => p.Mentions).Where(m => !m.IsLocal).Select(m => m.ActorURI).Distinct().ToList();
@@ -245,7 +245,7 @@ namespace PrivaPub.Api.Mastodon.Mappers
 				: await FollowedLocalAuthors(viewerId, localAuthors, token);
 			var quotedIds = nested ? new List<string>() : all.Where(p => p.QuoteState == QuoteState.Accepted && p.QuotedPostId != default).Select(p => p.QuotedPostId).Distinct().ToList();
 			var quotedPosts = new List<PostEntity>();
-			foreach (var quotedPost in quotedIds.Count == 0 ? new List<PostEntity>() : await _dbEntities.Posts.Match(p => quotedIds.Contains(p.ID) && !p.DeletedAt.HasValue).ExecuteAsync(token))
+			foreach (var quotedPost in quotedIds.Count == 0 ? new List<PostEntity>() : await _dbEntities.Posts.Match(p => quotedIds.Contains(p.ID)).Match(VisibilityPolicy.IsShown).ExecuteAsync(token))
 				if (quotedPost.Visibility != PostVisibility.LocalGeo && await VisibilityPolicy.CanSee(quotedPost, viewerId, token))
 					quotedPosts.Add(quotedPost);
 			var quotedStatuses = quotedPosts.Count == 0
@@ -270,7 +270,7 @@ namespace PrivaPub.Api.Mastodon.Mappers
 			var mapped = new List<Status>();
 			foreach (var post in posts)
 			{
-				if (hidden.Contains(post.ActorURI) || post.ReblogOfPostId != default && originals.TryGetValue(post.ReblogOfPostId, out var hiddenOriginal)
+				if (!VisibilityPolicy.Shown(post) || hidden.Contains(post.ActorURI) || post.ReblogOfPostId != default && originals.TryGetValue(post.ReblogOfPostId, out var hiddenOriginal)
 					&& hidden.Contains(hiddenOriginal.ActorURI))
 					continue;
 				var status = Map(post);