Town: client cells from their latest run; the backlog after a fresh village
decePub's e2e runs append to out/client.jsonl. The report kept the newest row per test and account, and the backlog every row, so a failure from a run that picked other accounts stayed in both after later runs passed. Each client cell now counts as its latest run left it (check.latest_client: the rows within an hour of its newest). The backlog comes from a fresh village on today's build: 2556 checks pass and 4 fail, three of them peer-to-peer counts, and one GoToSocial losing a status from its cache (an Update handled as a Create; a restart heals it), recorded in INTEROP.md rather than as a gap, which would hide our own edits failing there. The community vote and the Lemmy thread that failed before were data from older builds. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
This commit is contained in:
thepraandClaude Opus 5.5 committed
1 parent
5860b71223
commit
fb90558dec
5 files changed
+27
-27
No files matched your search
+6
-22
@@ -1,40 +1,24 @@
|
||||
# Interop backlog
|
||||
|
||||
Generated by `tools/pasture/town.sh backlog --write` from `village-20261004-234452` on 2026-10-04. Do not edit by hand: fix the cause, or record a known gap in `tools/pasture/town/gaps.json`, and generate it again.
|
||||
Generated by `tools/pasture/town.sh backlog --write` from `village-20261005-111857` on 2026-10-05. Do not edit by hand: fix the cause, or record a known gap in `tools/pasture/town/gaps.json`, and generate it again.
|
||||
|
||||
Checks: 2392 pass, 23 fail, 1 known gaps, 19 known gaps now passing.
|
||||
Checks: 2556 pass, 4 fail, 1 known gaps, 0 known gaps now passing.
|
||||
|
||||
## New failures
|
||||
|
||||
- **`deliver.community|lemmy|privapub|in`**: 3 failing. Example: p81 expected `held`, got `missing`.
|
||||
- **`deliver.community|lemmy|akkoma|via`**: 3 failing. Example: p84 expected `held`, got `missing`.
|
||||
- **`deliver.followers|akkoma|mastodon|control`**: 1 failing. Example: p36 expected `held`, got `missing`.
|
||||
- **`deliver.community|lemmy|mastodon|via`**: 1 failing. Example: p166 expected `held`, got `missing`.
|
||||
- **`count.boost.public|*|misskey|control`**: 1 failing. Example: p19 expected `>=1`, got `0`.
|
||||
- **`count.like.community|*|mastodon|control`**: 1 failing. Example: p161 expected `>=1`, got `0`.
|
||||
- **`count.boost.public|*|sharkey|control`**: 1 failing. Example: p175 expected `>=1`, got `0`.
|
||||
- **`client.render.text.public|akkoma|decepub|in`**: 1 failing. Example: p1 expected `None`, got `Timeout 30000ms exceeded. Call log: - waiting for Locator("article[data-uri='https://akkoma.test/objects/c7942cfe-4fee-4127-a0c3-20974b8c1f9`.
|
||||
- **`client.render.cw.public|sharkey|decepub|in`**: 1 failing. Example: p20 expected `None`, got `Locator expected to contain text 'map food' But was: 'Authoracct xavi_sk33@sharkey.testuri https://sharkey.test/users/arxwyny1ewcv0007create`.
|
||||
- **`client.render.mention.public|akkoma|decepub|in`**: 1 failing. Example: p54 expected `None`, got `Timeout 30000ms exceeded. Call log: - waiting for Locator("article[data-uri='https://akkoma.test/objects/d75be046-217f-4081-bf4d-46c979893d5`.
|
||||
- **`client.render.image.public|sharkey|decepub|in`**: 1 failing. Example: p77 expected `None`, got `Locator expected to be visible Error: element(s) not found Call log: - Expect "ToBeVisibleAsync" with timeout 5000ms - waiting for Locator("`.
|
||||
- **`client.render.image.public|akkoma|decepub|in`**: 1 failing. Example: p18 expected `None`, got `Locator expected to be visible Error: element(s) not found Call log: - Expect "ToBeVisibleAsync" with timeout 5000ms - waiting for Locator("`.
|
||||
- **`client.render.poll.public|akkoma|decepub|in`**: 1 failing. Example: p11 expected `None`, got `Timeout 30000ms exceeded. Call log: - waiting for Locator("article[data-uri='https://akkoma.test/objects/85cd4293-d8fb-4f68-92fa-909a2d2a7de`.
|
||||
- **`client.render.text.unlisted|akkoma|decepub|in`**: 1 failing. Example: p32 expected `None`, got `Timeout 30000ms exceeded. Call log: - waiting for Locator("article[data-uri='https://akkoma.test/objects/38ba2cb1-37f8-4c58-9a17-ef3d71418f6`.
|
||||
- **`client.action.boost|misskey|decepub|in`**: 1 failing. Example: p19 expected `None`, got `undoing the boost never reached misskey`.
|
||||
- **`client.action.boost|akkoma|decepub|in`**: 1 failing. Example: p1 expected `None`, got `Timeout 30000ms exceeded. Call log: - waiting for Locator("article[data-uri='https://akkoma.test/objects/c7942cfe-4fee-4127-a0c3-20974b8c1f9`.
|
||||
- **`client.action.reply|gts|decepub|in`**: 1 failing. Example: p28 expected `None`, got `the reply never landed under the post on gts`.
|
||||
- **`client.action.reply|akkoma|decepub|in`**: 1 failing. Example: p1 expected `None`, got `Timeout 30000ms exceeded. Call log: - waiting for Locator("article[data-uri='https://akkoma.test/objects/c7942cfe-4fee-4127-a0c3-20974b8c1f9`.
|
||||
- **`client.action.favourite|akkoma|decepub|in`**: 1 failing. Example: p1 expected `None`, got `Timeout 30000ms exceeded. Call log: - waiting for Locator("article[data-uri='https://akkoma.test/objects/c7942cfe-4fee-4127-a0c3-20974b8c1f9`.
|
||||
- **`edit.text|privapub|gts|out`**: 1 failing. Example: p4 expected `(edited)`, got `l market harvest rain quiet tide garden festival bridge (p4)`.
|
||||
|
||||
## Known gaps still failing
|
||||
|
||||
- **G-0003** (server, P7): Votes and moderation that a Lemmy community relays inside Announce (Like, Dislike, Undo, Delete, Block) are dropped. 1 checks. Code: `PrivaPub/Federation/Inbox/Handlers/AnnounceHandler.cs:203`.
|
||||
- **G-0005** (peer, upstream): Misskey keeps a post's renote count when a remote Undo{Announce} deletes the renote, so an undone boost still counts. 1 checks. Code: `misskey-dev/misskey packages/backend/src/core/NoteDeleteService.ts: only repliesCount is decremented on delete`.
|
||||
|
||||
## Known gaps now passing (close them)
|
||||
|
||||
- **G-0001**: Likes, reactions, votes and downvotes from followers on followers-only posts, and from members on circle posts, are dropped. 16 checks pass.
|
||||
- **G-0003**: Votes and moderation that a Lemmy community relays inside Announce (Like, Dislike, Undo, Delete, Block) are dropped. 3 checks pass.
|
||||
None.
|
||||
|
||||
## Scenario failures
|
||||
|
||||
- akkoma: undo of like or boost not applied on Akkoma
|
||||
None.
|
||||
@@ -214,6 +214,10 @@ Findings:
|
||||
- `likes`/`shares`;
|
||||
- attachment `width`/`height`.
|
||||
- **Deleted statuses:** 0.22 keeps a stub and answers 410.
|
||||
- **It can lose a status from its cache** (seen once, town 2026-10-05): it looks an Update's object up by URI, and when
|
||||
that fails it handles the Update as a Create, which changes nothing for a status it already has. One followers-only
|
||||
post kept its first text through three Updates, each logged as `activityType=Create`, though its row was there; after
|
||||
a restart the same Update applied. Fresh posts, liked or not, were edited at once.
|
||||
|
||||
**Expects**
|
||||
- **Signed requests:** every GET and POST is signed, draft-cavage only, with RSA keys. **No RFC 9421 in either
|
||||
|
||||
@@ -27,7 +27,7 @@ def main(args):
|
||||
with open(os.path.join(run_dir, "results.json")) as f:
|
||||
result = json.load(f)
|
||||
gaps = {g["id"]: g for g in check.load_gaps()}
|
||||
client = check.classify([dict(c) for c in load_jsonl(os.path.join(seed.OUT, "client.jsonl"))], list(gaps.values()))
|
||||
client = check.classify(check.latest_client([dict(c) for c in load_jsonl(os.path.join(seed.OUT, "client.jsonl"))]), list(gaps.values()))
|
||||
scenarios = load_jsonl(os.path.join(seed.OUT, "scenarios.jsonl"))
|
||||
checks = result["checks"] + client
|
||||
|
||||
|
||||
@@ -438,6 +438,21 @@ def canary_hits(platform, canaries):
|
||||
return sorted(c for c in canaries if c in dump)
|
||||
|
||||
|
||||
def latest_client(lines):
|
||||
"""decePub's e2e cells (out/client.jsonl, appended by every run) as each cell's latest run left them: the rows from
|
||||
within an hour of the cell's newest, the newest per test. A run that picked other accounts leaves no stale failure."""
|
||||
from datetime import datetime, timedelta
|
||||
at = lambda c: datetime.fromisoformat(c["t"].replace("Z", "+00:00"))
|
||||
newest = {}
|
||||
for c in lines:
|
||||
newest[c["cell"]] = max(newest.get(c["cell"], at(c)), at(c))
|
||||
latest = {}
|
||||
for c in sorted(lines, key=at):
|
||||
if at(c) >= newest[c["cell"]] - timedelta(hours=1):
|
||||
latest[(c["cell"], c.get("ref"))] = c
|
||||
return list(latest.values())
|
||||
|
||||
|
||||
def load_gaps():
|
||||
try:
|
||||
with open(os.path.join(HERE, "gaps.json")) as f:
|
||||
|
||||
@@ -175,10 +175,7 @@ def main(args):
|
||||
import check
|
||||
with open(client) as f:
|
||||
lines = [json.loads(line) for line in f if line.strip()]
|
||||
latest = {}
|
||||
for c in lines:
|
||||
latest[(c["cell"], c.get("ref"))] = c # the latest result of each test
|
||||
checks = check.classify(list(latest.values()), check.load_gaps())
|
||||
checks = check.classify(check.latest_client(lines), check.load_gaps())
|
||||
for cell, v in check.summarise(checks).items():
|
||||
result["cells"][cell] = v
|
||||
from collections import Counter
|
||||
|
||||
Reference in new issue
Block a user