Load: a flood of fake servers; unique-key lookups no longer scan
tools/pasture/flood/flood.cs answers as twenty fake servers (flood1..20.test) and sends signed Creates, Likes and Follows at a set rate; load.sh measures the answers, the queue's wait and processing times, its drain and a persona's home timeline meanwhile (docs/LOAD.md has the method and the runs). What the runs found: - Every unique index was partial on $type: "string", which MongoDB never uses for an equality lookup, so every post by ObjectURI, actor by ActorURI, deleted object, domain block, remote instance and the rest was a collection scan (280 ms a post lookup at 30 000 posts). They are partial on $gt: "" now, which an equality on a string implies; MongoDB.Entities rebuilds them in place at the next start. - Two inbox workers capped intake near 110 activities a second: Federation:InboxConcurrency and DeliveryConcurrency (default 8) set them. - The indexes the plan listed as missing: a post's boosts and replies, a persona's boosts, who follows an actor, timeline rows by author, a post's likes and pins. At 300 activities a second (200 let through, the rest 429 by the per-origin limit) the queue wait went from 29 s to 6 ms at p50; with the limits lifted PrivaPub processes about 900 a second, each in under 10 ms, and the home timeline stays under 20 ms. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
This commit is contained in:
1 parent
f1e743c331
commit
eeb873b816
12 files changed
+497
-17
No files matched your search
@@ -28,9 +28,13 @@ namespace PrivaPub.Federation.Inbox
|
||||
readonly IInteractionLedger _ledger;
|
||||
readonly ILocalActorService _localActors;
|
||||
|
||||
readonly int _concurrency;
|
||||
|
||||
public InboxProcessor(IRemoteActorService remoteActors, IEnumerable<IActivityHandler> handlers, ILogger<InboxProcessor> logger,
|
||||
IInteractionLedger ledger = default, ILocalActorService localActors = default)
|
||||
IInteractionLedger ledger = default, ILocalActorService localActors = default,
|
||||
Microsoft.Extensions.Options.IOptions<Infrastructure.Http.FederationOptions> federation = default)
|
||||
{
|
||||
_concurrency = Math.Max(1, federation?.Value.InboxConcurrency ?? 8);
|
||||
_localActors = localActors;
|
||||
_remoteActors = remoteActors;
|
||||
_handlers = handlers.ToDictionary(h => h.Type, StringComparer.Ordinal);
|
||||
@@ -39,7 +43,7 @@ namespace PrivaPub.Federation.Inbox
|
||||
}
|
||||
|
||||
public JobKind Kind => JobKind.ProcessInbox;
|
||||
public int Concurrency => 2;
|
||||
public int Concurrency => _concurrency;
|
||||
public int MaxAttempts => 8;
|
||||
public int PerHostLimit => 2;
|
||||
|
||||
|
||||
Reference in new issue
Block a user