M5: outbound requests, previews, the media proxy and served traffic

- HttpScope (AsyncLocal) tags each outbound request with a purpose and a trigger:
  - purpose is set by the caller: actor, key, object, webfinger, context, nodeinfo;
  - trigger is set by the job kind, by "verify" during inbox verification, or defaults
    to "request".
- FederationHttp records every JSON, media and stream fetch: status, time, bytes, hops,
  and an outcome of ok, refused or failed, with a reason: disallowed, remembered,
  bad-redirect, too-many-redirects, content-type, too-large, bad-json, private-address,
  timeout, network, or the status. A fetch a reader caused (trigger "request") is only
  counted per server per day.
- Link previews record a 'preview' event: card, no-card or failed.
- The media proxy counts cache hits.
- TrafficMeter counts the client API per endpoint group, method and status class. It
  counts our served documents (actor, outbox, collection, object, activity, licence,
  webfinger, nodeinfo) by kind, status and whether signed, per day and never per server,
  and never names a circle's collections.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ELjqpznMFMNrJoJUj6K5p2
This commit is contained in:
thepraandClaude Opus 5.5 committed 2026-10-03 11:10:42 +02:00
1 parent e247001bdb
commit d37999970c
14 files changed
+487 -19

No files matched your search

+17 -1
View File
@@ -4,6 +4,8 @@ using Microsoft.Extensions.Options;
using MongoDB.Entities;
using PrivaPub.Infrastructure.Statistics;
using PrivaPub.Models.Statistics;
using PrivaPub.Federation.Actors;
using PrivaPub.Federation.Objects;
using PrivaPub.Infrastructure.Http;
@@ -46,14 +48,17 @@ namespace PrivaPub.Domain.Content
readonly IJobQueue _queue;
readonly IOptionsMonitor<FederationOptions> _options;
readonly IInteractionLedger _ledger;
public LinkPreviews(DbEntities dbEntities, ILocalActorService localActors, IFederationHttp http, IJobQueue queue,
IOptionsMonitor<FederationOptions> options)
IOptionsMonitor<FederationOptions> options, IInteractionLedger ledger = default)
{
_dbEntities = dbEntities;
_localActors = localActors;
_http = http;
_queue = queue;
_options = options;
_ledger = ledger;
}
public JobKind Kind => JobKind.FetchPreview;
@@ -104,8 +109,19 @@ namespace PrivaPub.Domain.Content
async Task<LinkPreview> Fetch(string url, CancellationToken token)
{
var started = System.Diagnostics.Stopwatch.GetTimestamp();
var (finalUri, html) = await _http.GetPage(url, token);
var preview = html == default ? new LinkPreview { Url = url, Failed = true } : Read(url, finalUri, html);
_ledger?.Record(new InteractionEvent
{
Channel = Interactions.Preview,
Host = Interactions.HostOf(url),
Outcome = html == default ? Interactions.Failed : Interactions.Ok,
Reason = html == default ? default : preview.Title == default && preview.ImageURL == default ? "no-card" : "card",
LatencyMs = (int)System.Diagnostics.Stopwatch.GetElapsedTime(started).TotalMilliseconds,
Bytes = html == default ? default : System.Text.Encoding.UTF8.GetByteCount(html),
Redirects = finalUri == default || finalUri.AbsoluteUri == url ? 0 : 1
});
await DB.Default.Update<LinkPreview>()
.Match(p => p.Url == url)
.Modify(p => p.Title, preview.Title)