@page "/stargazing" @model PrivaPub.Web.Pages.StargazingModel @{ ViewData["Title"] = "Stargazing"; }

Stargazing

What this server records about other servers, and how to keep its crawler out.

Servers, never people

This server keeps statistics about the fediverse for teaching and curiosity. They name servers, never accounts: what kind of software a server runs, what it exchanges with us, and how reliably. Distinct accounts are only counted, through a key that is destroyed at the end of each day.

A server we exchange activities with is described once a week, from its public NodeInfo and, when it has one, its Mastodon instance API. Its location comes from the address we reached, looked up in an offline database: only the country is shown publicly for small servers, and only the CDN for servers behind one.

@if (Model.GeoSource is { } geo) {

Locations come from @geo, licensed under CC BY 4.0.

} else {

No location database is loaded yet, so servers are not located.

}

The crawler

@if (Model.CrawlerEnabled) {

The crawler is on on this server.

} else {

The crawler is off on this server: it only learns about servers it already exchanges with.

}

When on, it identifies itself as

@Model.UserAgent

It visits one server a minute, each at most once a week, and reads only:

It never reads accounts, posts, timelines or directories.

Keeping it out

Add this to your server's robots.txt:

User-agent: @StargazerToken
Disallow: /

If your robots.txt cannot be read because of a server error or a timeout, the crawler stays out too.

@functions { const string StargazerToken = PrivaPub.Federation.Crawler.Stargazer.Token; }