Everything on, phase 1: geolocation fetches itself, the deploy signs in as @thepra, the crawler is on, sign-up by invitation
Build / Build (push) Successful in 5m1s
Deploy / privapub.thepra.dev (push) Successful in 5m39s

Owner decisions (2026-10-04, recorded in docs/ROADMAP.md): production runs everything that is built, and nothing waits
on a person running a command.

- Geolocation updates itself. GeoUpdater, a hosted service, checks daily whether each DB-IP Lite database was built this
  month. If not, it fetches this month's, or last month's early in the month. It installs a file only once it opens as
  the right kind of database, then swaps it in atomically, and the locator reloads at once. Lookups now run under the
  lock, so a reload can no longer dispose a reader mid-lookup. The systemd timer, its script and their setup.sh lines
  are gone: the root step they needed never happened, and none is needed now. /stargazing names the database in use.
- The admin CLI runs after the app is built, with every service and nothing started.
  - `create-root <login> [--admin]` takes the password on stdin; it is how the first login is made while sign-up is
    closed.
  - `smoke <persona>` keeps the root `deploy-smoke` and an undiscoverable persona, and gives the root a new password
    on every run.
- The deploy signs in as @thepra. It runs the CLI, gets a token through the real OAuth flow (tools/smoke/oauth.sh,
  moved out of the pasture's privapub_token, which now uses it), checks the signed-in API and that @thepra is
  undiscoverable, then revokes the token. PRIVAPUB_SMOKE_TOKEN is gone.
- The deploy also fails when:
  - NodeInfo and the instance API disagree about registrations;
  - /stargazing does not say the crawler is on;
  - the geolocation databases are missing or more than 40 days old.
- The crawler is on in production, seeded with ten large servers of different kinds. FEDERATION.md now describes it
  and how to opt out.
- One registrations switch (Registrations:Mode, default Invitations; Open in tests and the pasture). It is read by
  open sign-up (403 when closed), NodeInfo `openRegistrations`, and v1 and v2 of the instance API, so they can no longer
  disagree. Before, NodeInfo said open and the instance API said closed. Group invitations always work, so
  invites_enabled is true.
- A persona edit through /clientapi no longer resets what the Mastodon API set (discoverable, locked, quote policy…):
  the theme is merged into the settings instead of replacing them.

650 tests pass. The deploy's smoke step was rehearsed against the pasture's PrivaPub.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ELjqpznMFMNrJoJUj6K5p2
This commit is contained in:
thepraandClaude Opus 5.5 committed 2026-10-04 02:37:38 +02:00
1 parent cd5eb25948
commit 5f56681c01
33 files changed
+909 -157

No files matched your search

@@ -154,6 +154,34 @@ namespace PrivaPub.Tests.Http
Assert.DoesNotContain(persona.Root.UserName, update.ToJsonString());
}
[Fact]
public async Task A_theme_change_here_keeps_what_the_mastodon_api_set()
{
var persona = await _host.Persona(await _host.SignUp("keep"), "keep");
await DB.Default.Update<Avatar>().MatchID(persona.Id)
.Modify(a => a.Settings.IsDiscoverable, false)
.Modify(a => a.Settings.IsLocked, true)
.Modify(a => a.Settings.QuotePolicy, QuotePolicies.Nobody)
.ExecuteAsync(TestContext.Current.CancellationToken);
using var client = _host.As(persona.Root.Jwt);
var response = await client.PostJson("/clientapi/avatar/private/update", new
{
avatarId = persona.Id,
name = "Kept",
biography = "same settings",
settings = new { isDefault = false, darkThemeIndexColour = 100, themeIsDarkMode = true }
});
Assert.Equal(HttpStatusCode.OK, response.StatusCode);
var settings = (await DB.Default.Find<Avatar>().MatchID(persona.Id).ExecuteFirstAsync(TestContext.Current.CancellationToken)).Settings;
Assert.True(settings.ThemeIsDarkMode);
Assert.Equal(100, settings.DarkThemeIndexColour);
Assert.False(settings.IsDiscoverable);
Assert.True(settings.IsLocked);
Assert.Equal(QuotePolicies.Nobody, settings.QuotePolicy);
}
[Fact]
public async Task Updating_another_roots_persona_is_refused()
{
+4 -2
View File
@@ -28,7 +28,8 @@ namespace PrivaPub.Tests.Http
Assert.Equal("4.2.0 (compatible; PrivaPub)", v1.Body.Text("version"));
Assert.Equal(InstanceController.Version, v1.Body.Text("version"));
Assert.Equal(PrivaPubHost.Host, v1.Body.Text("uri"));
Assert.False(v1.Body.Flag("registrations"));
Assert.True(v1.Body.Flag("registrations"));
Assert.True(v1.Body.Flag("invites_enabled"));
Assert.True(v1.Body["stats"].Number("user_count") >= 0);
Assert.Equal(5000, v1.Body["configuration"]!["statuses"].Number("max_characters"));
Assert.Equal(4, v1.Body["configuration"]!["polls"].Number("max_options"));
@@ -41,7 +42,8 @@ namespace PrivaPub.Tests.Http
Assert.Equal(InstanceController.Version, v2.Body.Text("version"));
Assert.Equal(PrivaPubHost.Host, v2.Body.Text("domain"));
Assert.Equal($"wss://{PrivaPubHost.Host}", v2.Body["configuration"]!["urls"].Text("streaming"));
Assert.False(v2.Body["registrations"].Flag("enabled"));
Assert.True(v2.Body["registrations"].Flag("enabled"));
Assert.True((await anonymous.Get("/nodeinfo/2.1")).Ok().Body.Flag("openRegistrations"));
Assert.False(v2.Body["configuration"]!["translation"].Flag("enabled"));
Assert.Equal(7, v2.Body["api_versions"].Number("mastodon"));
+50
View File
@@ -0,0 +1,50 @@
using Microsoft.AspNetCore.Hosting;
using Microsoft.AspNetCore.Mvc.Testing;
using PrivaPub.Infrastructure;
using PrivaPub.Tests.Support;
using PrivaPub.Tests.Support.Host;
using System.Net;
namespace PrivaPub.Tests.Http
{
[Trait("Category", "Integration")]
public sealed class RegistrationsTests : IAsyncLifetime
{
WebApplicationFactory<Program> _closed;
public async ValueTask InitializeAsync()
{
Assert.SkipUnless(MongoFixture.Enabled, MongoFixture.Skip);
var shared = await PrivaPubHost.Shared();
_closed = shared.WithWebHostBuilder(builder => builder.UseSetting("Registrations:Mode", nameof(RegistrationMode.Invitations)));
}
public async ValueTask DisposeAsync()
{
if (_closed != default)
await _closed.DisposeAsync();
}
[Fact]
public async Task Invitations_only_refuses_sign_up_and_every_document_says_closed()
{
using var client = _closed.CreateClient(new WebApplicationFactoryClientOptions { BaseAddress = new Uri(PrivaPubHost.Base + "/"), AllowAutoRedirect = false });
client.DefaultRequestHeaders.Add(PrivaPubHost.ClientHeader, $"10.9.{Random.Shared.Next(256)}.{Random.Shared.Next(1, 255)}");
var signUp = await client.PostJson("/clientapi/user/signup", new { userName = $"closed{Guid.NewGuid():N}"[..20], password = "Closed-Door-Pass-1" });
var nodeInfo = (await client.Get("/nodeinfo/2.1")).Ok().Body;
var v1 = (await client.Get("/api/v1/instance")).Ok().Body;
var v2 = (await client.Get("/api/v2/instance")).Ok().Body;
Assert.Equal(HttpStatusCode.Forbidden, signUp.StatusCode);
Assert.Contains("invitation", await signUp.Content.ReadAsStringAsync(TestContext.Current.CancellationToken));
Assert.False(nodeInfo.Flag("openRegistrations"));
Assert.False(v1.Flag("registrations"));
Assert.True(v1.Flag("invites_enabled"));
Assert.False(v2["registrations"].Flag("enabled"));
Assert.Equal(RegistrationOptions.ClosedMessage, v2["registrations"].Text("message"));
}
}
}
@@ -1,3 +1,5 @@
using Microsoft.Extensions.DependencyInjection;
using MongoDB.Entities;
using PrivaPub.ClientModels;
@@ -37,8 +39,8 @@ namespace PrivaPub.Tests.Infrastructure
{
var root = await _host.SignUp("promote");
Assert.Equal(0, await AdminCommands.Run(new[] { "promote", root.UserName.ToUpperInvariant() }));
Assert.Equal(0, await AdminCommands.Run(new[] { "promote", root.UserName }));
Assert.Equal(0, await AdminCommands.Run(new[] { "promote", root.UserName.ToUpperInvariant() }, _host.Services));
Assert.Equal(0, await AdminCommands.Run(new[] { "promote", root.UserName }, _host.Services));
Assert.Equal(new[] { Policies.IsAdmin, Policies.IsModerator, Policies.IsUser }.Order(), await StoredPolicies(root));
}
@@ -47,10 +49,10 @@ namespace PrivaPub.Tests.Infrastructure
public async Task Demote_leaves_a_plain_user()
{
var root = await _host.SignUp("demote");
Assert.Equal(0, await AdminCommands.Run(new[] { "promote", root.UserName }));
Assert.Equal(0, await AdminCommands.Run(new[] { "promote", root.UserName }, _host.Services));
Assert.Equal(0, await AdminCommands.Run(new[] { "demote", root.UserName }));
Assert.Equal(0, await AdminCommands.Run(new[] { "demote", root.UserName }));
Assert.Equal(0, await AdminCommands.Run(new[] { "demote", root.UserName }, _host.Services));
Assert.Equal(0, await AdminCommands.Run(new[] { "demote", root.UserName }, _host.Services));
Assert.Equal(new[] { Policies.IsUser }, await StoredPolicies(root));
}
@@ -58,8 +60,8 @@ namespace PrivaPub.Tests.Infrastructure
[Fact]
public async Task An_unknown_root_exits_1()
{
Assert.Equal(1, await AdminCommands.Run(new[] { "promote", $"nobody{Guid.NewGuid():N}" }));
Assert.Equal(1, await AdminCommands.Run(new[] { "demote", $"nobody{Guid.NewGuid():N}" }));
Assert.Equal(1, await AdminCommands.Run(new[] { "promote", $"nobody{Guid.NewGuid():N}" }, _host.Services));
Assert.Equal(1, await AdminCommands.Run(new[] { "demote", $"nobody{Guid.NewGuid():N}" }, _host.Services));
}
public static TheoryData<string[]> Misuses() => new()
@@ -75,7 +77,7 @@ namespace PrivaPub.Tests.Infrastructure
[MemberData(nameof(Misuses))]
public async Task Misuse_exits_2(string[] args)
{
Assert.Equal(2, await AdminCommands.Run(args));
Assert.Equal(2, await AdminCommands.Run(args, _host.Services));
}
[Fact]
@@ -84,7 +86,7 @@ namespace PrivaPub.Tests.Infrastructure
var root = await _host.SignUp("policies");
Assert.Equal(HttpStatusCode.Forbidden, await AdminRoute(root));
Assert.Equal(0, await AdminCommands.Run(new[] { "promote", root.UserName }));
Assert.Equal(0, await AdminCommands.Run(new[] { "promote", root.UserName }, _host.Services));
Assert.Equal(HttpStatusCode.OK, await AdminRoute(root));
using (var client = _host.As(root.Jwt))
{
@@ -92,8 +94,54 @@ namespace PrivaPub.Tests.Infrastructure
Assert.Contains(Policies.IsAdmin, refreshed["policies"]!.AsArray().Select(p => p!.GetValue<string>()));
}
Assert.Equal(0, await AdminCommands.Run(new[] { "demote", root.UserName }));
Assert.Equal(0, await AdminCommands.Run(new[] { "demote", root.UserName }, _host.Services));
Assert.Equal(HttpStatusCode.Forbidden, await AdminRoute(root));
}
async Task<HttpStatusCode> LogIn(string login, string password)
{
using var client = _host.Client();
return (await client.PostJson("/clientapi/user/login", new { userName = login, password })).StatusCode;
}
[Fact]
public async Task Create_root_makes_a_login_from_a_password_on_standard_input()
{
using var scope = _host.Services.CreateScope();
var login = $"made{Guid.NewGuid():N}"[..20];
var output = new StringWriter();
Assert.Equal(0, await AdminCommands.Run(new[] { "create-root", login, "--admin" }, scope.ServiceProvider, new StringReader("Typed-On-Stdin-1\n"), output));
Assert.Equal(1, await AdminCommands.Run(new[] { "create-root", login }, scope.ServiceProvider, new StringReader("Typed-On-Stdin-2\n"), new StringWriter()));
Assert.Equal(1, await AdminCommands.Run(new[] { "create-root", $"weak{Guid.NewGuid():N}"[..20] }, scope.ServiceProvider, new StringReader("short\n"), new StringWriter()));
Assert.Equal(2, await AdminCommands.Run(new[] { "create-root", login, "--root" }, scope.ServiceProvider, new StringReader("Typed-On-Stdin-3\n"), new StringWriter()));
Assert.Equal(HttpStatusCode.OK, await LogIn(login, "Typed-On-Stdin-1"));
var stored = await DB.Default.Find<RootUser>().Match(u => u.UserName == login).ExecuteFirstAsync(TestContext.Current.CancellationToken);
Assert.Contains(Policies.IsAdmin, stored.Policies);
}
[Fact]
public async Task Smoke_keeps_one_undiscoverable_persona_and_a_new_password_each_run()
{
using var scope = _host.Services.CreateScope();
var persona = $"smoke{Guid.NewGuid():N}"[..20];
var first = new StringWriter();
var second = new StringWriter();
Assert.Equal(0, await AdminCommands.Run(new[] { "smoke", persona }, scope.ServiceProvider, output: first));
Assert.Equal(0, await AdminCommands.Run(new[] { "smoke", persona }, scope.ServiceProvider, output: second));
var (login, oldPassword) = (first.ToString().Trim().Split(' ')[0], first.ToString().Trim().Split(' ')[1]);
var newPassword = second.ToString().Trim().Split(' ')[1];
Assert.Equal(AdminCommands.SmokeLogin, login);
Assert.NotEqual(oldPassword, newPassword);
Assert.Equal(HttpStatusCode.OK, await LogIn(login, newPassword));
Assert.NotEqual(HttpStatusCode.OK, await LogIn(login, oldPassword));
var avatars = await DB.Default.Find<Avatar>().Match(a => a.UserName == persona).ExecuteAsync(TestContext.Current.CancellationToken);
var avatar = Assert.Single(avatars);
Assert.False(avatar.Settings.IsDiscoverable);
Assert.False(avatar.Settings.IsIndexable);
}
}
}
@@ -0,0 +1,125 @@
using MaxMind.Db;
using Microsoft.Extensions.Logging.Abstractions;
using PrivaPub.Infrastructure.Geo;
using PrivaPub.Infrastructure.Statistics;
using PrivaPub.Tests.Support;
using System.IO.Compression;
namespace PrivaPub.Tests.Infrastructure
{
public sealed class GeoUpdaterTests : IAsyncLifetime
{
static readonly DateTime Now = new(2026, 10, 4, 12, 0, 0, DateTimeKind.Utc);
static readonly DateTime LastMonth = new(2026, 9, 2, 0, 0, 0, DateTimeKind.Utc);
readonly string _directory = Path.Combine(Path.GetTempPath(), $"privapub-geo-{Guid.NewGuid():N}");
Peer _peer;
GeoUpdater _updater;
DbIpLocator _locator;
public async ValueTask InitializeAsync()
{
_peer = await Peer.Start();
var options = new StaticOptions<StatisticsOptions>(new StatisticsOptions
{
GeoDirectory = _directory,
Geo = new GeoOptions { DownloadBase = $"{_peer.A}/free" }
});
_locator = new DbIpLocator(options, NullLogger<DbIpLocator>.Instance);
_updater = new GeoUpdater(new Clients(), options, _locator, NullLogger<GeoUpdater>.Instance);
}
public async ValueTask DisposeAsync()
{
_locator?.Dispose();
await _peer.DisposeAsync();
if (Directory.Exists(_directory))
Directory.Delete(_directory, recursive: true);
}
sealed class Clients : IHttpClientFactory
{
public HttpClient CreateClient(string name) => new();
}
static byte[] Gzip(byte[] bytes)
{
using var output = new MemoryStream();
using (var gzip = new GZipStream(output, CompressionLevel.Fastest))
gzip.Write(bytes);
return output.ToArray();
}
void Publish(string kind, DateTime month, byte[] database) =>
_peer.ServeFile($"/free/dbip-{kind}-lite-{month:yyyy-MM}.mmdb.gz", Gzip(database), "application/gzip");
static byte[] City(DateTime built) => MiniMmdb.Build("DBIP-City-Lite", built);
static byte[] Asn(DateTime built) => MiniMmdb.Build("DBIP-ASN-Lite (compat=GeoLite2-ASN)", built);
DateTime Built(string kind)
{
using var reader = new Reader(Path.Combine(_directory, GeoUpdater.FileOf(kind)));
return reader.Metadata.BuildDate;
}
[Fact]
public async Task Missing_databases_are_fetched_checked_and_read_at_once()
{
Publish("city", Now, City(Now));
Publish("asn", Now, Asn(Now));
Assert.Equal(2, await _updater.Update(Now, TestContext.Current.CancellationToken));
Assert.Equal(Now.Date, Built("city").Date);
Assert.Equal(Now.Date, Built("asn").Date);
Assert.Equal("DB-IP Lite 2026-10", _locator.Source);
Assert.Empty(Directory.GetFiles(_directory).Where(f => f.EndsWith(".part") || f.EndsWith(".new")));
}
[Fact]
public async Task Early_in_the_month_last_months_database_is_used_and_not_fetched_again()
{
Publish("city", LastMonth, City(LastMonth));
Publish("asn", LastMonth, Asn(LastMonth));
Assert.Equal(2, await _updater.Update(Now, TestContext.Current.CancellationToken));
var asked = _peer.Requests.Count;
Assert.Equal(0, await _updater.Update(Now, TestContext.Current.CancellationToken));
Assert.Equal(LastMonth.Date, Built("city").Date);
Assert.DoesNotContain(_peer.Requests.Skip(asked), r => r.Path.Contains("2026-09"));
}
[Fact]
public async Task A_current_database_asks_nothing()
{
Publish("city", Now, City(Now));
Publish("asn", Now, Asn(Now));
await _updater.Update(Now, TestContext.Current.CancellationToken);
var asked = _peer.Requests.Count;
Assert.Equal(0, await _updater.Update(Now.AddDays(1), TestContext.Current.CancellationToken));
Assert.Equal(asked, _peer.Requests.Count);
}
[Fact]
public async Task A_broken_or_wrong_database_is_refused_and_the_old_one_kept()
{
Publish("city", LastMonth, City(LastMonth));
Publish("asn", LastMonth, Asn(LastMonth));
await _updater.Update(LastMonth.AddDays(1), TestContext.Current.CancellationToken);
Publish("city", Now, "not a database at all"u8.ToArray());
Publish("asn", Now, City(Now));
Assert.Equal(0, await _updater.Update(Now, TestContext.Current.CancellationToken));
Assert.Equal(LastMonth.Date, Built("city").Date);
Assert.Equal(LastMonth.Date, Built("asn").Date);
Assert.Empty(Directory.GetFiles(_directory).Where(f => f.EndsWith(".part") || f.EndsWith(".new")));
}
}
}
+1 -1
View File
@@ -39,7 +39,7 @@ namespace PrivaPub.Tests.Support.Host
public static async Task<Root> Admin(this PrivaPubHost host)
{
var root = await host.SignUp("admin");
Assert.Equal(0, await AdminCommands.Run(new[] { "promote", root.UserName }));
Assert.Equal(0, await AdminCommands.Run(new[] { "promote", root.UserName }, host.Services));
return await host.LogIn(root);
}
@@ -73,6 +73,8 @@ namespace PrivaPub.Tests.Support.Host
["Federation:AllowPrivateNetworks"] = "true",
["Federation:AllowPlainHttp"] = "true",
["Federation:FetchLinkPreviews"] = "false",
["Registrations:Mode"] = "Open",
["Statistics:Geo:AutoUpdate"] = "false",
["Media:Root"] = _mediaRoot,
["Logging:LogLevel:Default"] = "Warning",
["Serilog:MinimumLevel:Default"] = Environment.GetEnvironmentVariable("PRIVAPUB_TEST_LOGS") == "1" ? "Information" : "Fatal"
+79
View File
@@ -0,0 +1,79 @@
using System.Text;
namespace PrivaPub.Tests.Support
{
// The smallest MaxMind DB the reader accepts (https://maxmind.github.io/MaxMind-DB/): an IPv4 search tree of one node
// whose two records both mean "no data", an empty data section, and the metadata map, which is all a test of the updater
// needs to tell a city database from an ASN one and to read its build date.
public static class MiniMmdb
{
public static byte[] Build(string databaseType, DateTime built)
{
using var output = new MemoryStream();
const uint nodeCount = 1;
for (var record = 0; record < 2; record++)
output.Write(new byte[] { 0, 0, (byte)nodeCount });//24-bit records equal to node_count: not found
output.Write(new byte[16]);//data section separator; the data section itself is empty
output.Write(new byte[] { 0xAB, 0xCD, 0xEF });
output.Write(Encoding.ASCII.GetBytes("MaxMind.com"));
Map(output,
("binary_format_major_version", w => UInt16(w, 2)),
("binary_format_minor_version", w => UInt16(w, 0)),
("build_epoch", w => UInt64(w, (ulong)new DateTimeOffset(built).ToUnixTimeSeconds())),
("database_type", w => Text(w, databaseType)),
("description", w => Map(w, ("en", x => Text(x, "a test database")))),
("ip_version", w => UInt16(w, 4)),
("languages", w => Array(w, x => Text(x, "en"))),
("node_count", w => UInt32(w, nodeCount)),
("record_size", w => UInt16(w, 24)));
return output.ToArray();
}
// type in the top three bits (0 and an extra byte for the extended types), size below 29 in the low five, or 29 and the
// rest in one more byte (names longer than 28 characters need it)
static void Control(Stream output, int type, int size)
{
var low = size < 29 ? size : 29;
output.WriteByte((byte)((type <= 7 ? type : 0) << 5 | low));
if (type > 7)
output.WriteByte((byte)(type - 7));
if (size >= 29)
output.WriteByte((byte)(size - 29));
}
static void Text(Stream output, string value)
{
var bytes = Encoding.UTF8.GetBytes(value);
Control(output, 2, bytes.Length);
output.Write(bytes);
}
static void Unsigned(Stream output, int type, ulong value)
{
var bytes = BitConverter.GetBytes(value).Reverse().SkipWhile(b => b == 0).ToArray();
Control(output, type, bytes.Length);
output.Write(bytes);
}
static void UInt16(Stream output, ushort value) => Unsigned(output, 5, value);
static void UInt32(Stream output, uint value) => Unsigned(output, 6, value);
static void UInt64(Stream output, ulong value) => Unsigned(output, 9, value);
static void Map(Stream output, params (string Key, Action<Stream> Value)[] entries)
{
Control(output, 7, entries.Length);
foreach (var (key, value) in entries)
{
Text(output, key);
value(output);
}
}
static void Array(Stream output, params Action<Stream>[] items)
{
Control(output, 11, items.Length);
foreach (var item in items)
item(output);
}
}
}