Load: a flood of fake servers; unique-key lookups no longer scan

tools/pasture/flood/flood.cs answers as twenty fake servers (flood1..20.test)
and sends signed Creates, Likes and Follows at a set rate; load.sh measures
the answers, the queue's wait and processing times, its drain and a persona's
home timeline meanwhile (docs/LOAD.md has the method and the runs).

What the runs found:
- Every unique index was partial on $type: "string", which MongoDB never uses
  for an equality lookup, so every post by ObjectURI, actor by ActorURI,
  deleted object, domain block, remote instance and the rest was a
  collection scan (280 ms a post lookup at 30 000 posts). They are partial on
  $gt: "" now, which an equality on a string implies; MongoDB.Entities
  rebuilds them in place at the next start.
- Two inbox workers capped intake near 110 activities a second:
  Federation:InboxConcurrency and DeliveryConcurrency (default 8) set them.
- The indexes the plan listed as missing: a post's boosts and replies, a
  persona's boosts, who follows an actor, timeline rows by author, a post's
  likes and pins.

At 300 activities a second (200 let through, the rest 429 by the per-origin
limit) the queue wait went from 29 s to 6 ms at p50; with the limits lifted
PrivaPub processes about 900 a second, each in under 10 ms, and the home
timeline stays under 20 ms.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
This commit is contained in:
thepraandClaude Opus 5.5 committed 2026-10-05 19:16:30 +02:00
1 parent f1e743c331
commit eeb873b816
12 files changed
+497 -17

No files matched your search

+16
View File
@@ -0,0 +1,16 @@
# flood: twenty fake servers (flood1..20.test, ten actors each) for load runs (load.sh), from tools/pasture/flood/flood.cs,
# a .NET file-based app published once into .flood and run on the runtime image PrivaPub uses. Its keys live in the
# pasture-flood volume, so `flood run` (podman exec) speaks as the actors `flood serve` answers for.
flood_up() {
local dotnet=${DOTNET:-$(command -v dotnet || echo ~/.dotnet/dotnet)}
[ -x "$here/.flood/flood" ] || "$dotnet" publish "$here/flood/flood.cs" -r linux-x64 --self-contained true -o "$here/.flood" -v quiet
podman volume exists pasture-flood || podman volume create --label pasture=1 pasture-flood >/dev/null
podman run -d --replace --name pasture-flood --network $net --label pasture=1 -w /app -v "$here/.flood:/app:Z,ro" \
-v pasture-flood:/state -v "$ca/bundle.pem:/etc/ssl/certs/ca-certificates.crt:z,ro" -e SSL_CERT_FILE=/etc/ssl/certs/ca-certificates.crt \
mcr.microsoft.com/dotnet/runtime-deps:10.0 /app/flood serve >/dev/null
for _ in $(seq 1 60); do
site flood1.test -s -o /dev/null -w '%{http_code}' https://flood1.test:6443/users/u0 2>/dev/null | grep -q 200 && break
sleep 2
done
echo "flood: https://flood1.test:6443 .. flood20.test"
}