M8: every server we touch is described, located and snapshotted weekly
- Touches: the ledger marks a server as touched when it sends us a verified activity, when
we exchange activities with it, or when we read its actors, keys, objects or WebFinger.
It upserts RemoteInstance.Seen, FirstSeenAt and LastSeenAt at most hourly per server, and
queues one DescribeInstance a week with the same dedupe key ObjectRecords uses. Suspended
servers and pages behind link previews are never described. Migration _010 marks the
servers already known as touched, with their dates.
- InstanceDescriber.Describe(host, crawled, allowed) reads:
- NodeInfo 2.2/2.1/2.0, now with its published user counts, posts, comments,
description, languages and schema version;
- for software with a Mastodon API, /api/v2/instance falling back to v1: title,
languages, registration mode, character limit, API version, source URL.
It never keeps a contact as a field; the raw document is kept for the admin only. It
locates the server from the address our connection reached (DB-IP Lite city and ASN, the
CDN named when fronted) and writes a RemoteInstanceSnapshot per ISO week, unreachable
weeks included. A crawled server is upserted as crawled only on insert, so it never
downgrades a touched one, and robots.txt can deny any path.
- PublicGeo.Project is the only public form of a location: a CDN-fronted server shows its
CDN only, a server reporting at least ten users shows its city, coordinates and network,
any other only its country.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ELjqpznMFMNrJoJUj6K5p2
This commit is contained in:
1 parent
fc5bb9511f
commit
c8a305ae92
10 files changed
+757
-72
No files matched your search
@@ -188,18 +188,15 @@ namespace PrivaPub.Tests.Http
|
||||
return response.StatusCode;
|
||||
}
|
||||
|
||||
var clock = System.Diagnostics.Stopwatch.StartNew();
|
||||
var first = new List<HttpStatusCode>();
|
||||
for (var i = 0; i < 300; i++)
|
||||
first.Add(await Post());
|
||||
var limited = await Post();
|
||||
// The bucket refills 50 every ten seconds: a slow machine may have earned a few more before the limit bites.
|
||||
for (var refilled = (int)((clock.Elapsed.TotalSeconds + 1) / 10) * 50; limited != HttpStatusCode.TooManyRequests && refilled > 0; refilled--)
|
||||
limited = await Post();
|
||||
// The bucket holds 300 and refills 50 every ten seconds on its own timer, so a refill can land at any moment:
|
||||
// the first refusal comes after at least 300 posts and within a few refills.
|
||||
var answers = new List<HttpStatusCode>();
|
||||
while (answers.Count < 600 && (answers.Count == 0 || answers[^1] != HttpStatusCode.TooManyRequests))
|
||||
answers.Add(await Post());
|
||||
|
||||
Assert.DoesNotContain(HttpStatusCode.TooManyRequests, first);
|
||||
Assert.All(first, status => Assert.Equal(HttpStatusCode.Unauthorized, status));
|
||||
Assert.Equal(HttpStatusCode.TooManyRequests, limited);
|
||||
Assert.Equal(HttpStatusCode.TooManyRequests, answers[^1]);
|
||||
Assert.True(answers.Count > 300, $"limited after only {answers.Count - 1} posts");
|
||||
Assert.All(answers.SkipLast(1), status => Assert.Equal(HttpStatusCode.Unauthorized, status));
|
||||
var polite = new RemoteActor(_peer, "polite");
|
||||
using var signed = await client.SendAsync(polite.SignedPost("/human-centipede", Direct(polite)), token);
|
||||
Assert.Equal(HttpStatusCode.Accepted, signed.StatusCode);
|
||||
|
||||
Reference in new issue
Block a user