12bb75809f495f9a385ceb92929934ff003bc695
5
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
  thepraandClaude Opus 5.5
|
3ca29603ed |
The media proxy is bounded
Anyone could mint signed proxy URLs (a remote account changes its icon, an anonymous lookup returns the URL), and each anonymous request held up to 40 MB in memory; the cache grew without bound between hourly trims; two clients asking for the same new file downloaded it twice and wrote over each other in place, so a reader could get half a file with a 7-day cache header; a file over the limit was downloaded twice on every request; a failed fetch, a 404, was cached by browsers for a week; cached media of a server suspended later were still served, and RejectMedia skipped avatars, emoji, covers, video variants, link cards and remote edits; /clientapi/group/members returned remote pictures raw. Now a download is shared by everyone asking at once, streamed into a .part file and renamed into place (FederationHttp.DownloadMedia copies bounded, never into memory), at most eight at a time; a file too big to cache is remembered for an hour and only streamed, a failure for five minutes; the cache's size is counted as it grows and trimmed as soon as it passes the cap; a cached file is opened before it is answered; browsers may cache only a success; nothing of a suspended server, or of one whose media are rejected, is proxied (everything remote a client sees goes through the proxy, so that covers every kind), and blocking one purges its cache; the proxy has its own rate limit per client address; group members' pictures are proxied; the proxy's key is loaded once, the oldest if two were made. This changes what PrivaPub serves its clients, not what it sends to other servers. Tests: clients asking at once share one download, a failure isn't cached by browsers, an over-limit file is fetched three times for two requests instead of four, a blocked server's media are refused and its cache purged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw |
||
|
thepraandClaude Opus 5.5
|
8e59145826 |
Audio and video are processed off the request
Every audio and video upload was probed and remuxed inside the request, whatever its length; ffmpeg would read any protocol and probe any format; `-map 0` kept the data tracks iPhones add, which mp4 refuses; nothing was ever transcoded, so HEVC or MPEG-4 Part 2 reached browsers that can't play them, and the advertised video_matrix_limit and frame rate limit were never applied; the output was read whole into memory, the video was saved before its poster could fail, and the poster's frame leaked in /tmp. FLAC uploads were served as 404. Now an upload sent to /api/v2/media is stored as sent in media-incoming (beside the media root, never served) and answered with 202 and no url, while a ProcessMedia job, one at a time, does the work; GET /api/v1/media/:id answers 206 until it is ready, or 422 with why, and media still processing can't be posted. v1 processes before answering. ffmpeg reads only that file (protocol whitelist, format forced from the probe) and drops data and subtitle tracks. A video browsers play as it is (H.264, VP8, VP9, AV1 within Media:MaxVideoPixels and MaxFrameRate) is remuxed, anything else transcoded to H.264 that fits, as Mastodon does; longer than Media:MaxSeconds is refused. Outputs move into place only once everything succeeded, every temporary file goes, durations are kept, FLAC is served as audio/flac, and the unit gets PrivateTmp. The instance API advertises the limits that are now applied. No pasture scenario uploads audio or video through PrivaPub, so the sweep could not see this. MastodonMediaTests: v2 answers 202 then the job makes it playable (and an unreadable file 422 once processed), media still processing can't be posted, MPEG-4 Part 2 becomes H.264, a video over the limit is made smaller, FLAC is served. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw |
||
|
thepraandClaude Opus 5.5
|
e4f9d0b61e |
An image is checked before it is decoded
An upload went straight to libvips: whatever loader recognised the bytes ran (an SVG sent as image/png was rasterised), nothing bounded how many pixels it would decode to (a small PNG could decode to gigabytes, three times over), a GIF was loaded frame by frame and never resized, and all of it ran inside the request with nothing limiting how many at once. A GIF was typed gifv but stayed a .gif, which a gifv player can't play; its metadata was kept; colours lost their ICC profile without being converted; HEIC was advertised but the bundled libvips can't decode it. Now: - only libvips' JPEG, PNG, GIF, WebP and HEIF loaders ever run on an upload (every other loader is blocked); - the header alone says how big an image would decode, refused above Media:MaxPixels (40 MP) or MaxFrames; - a still image is shrunk on load, turned by its orientation and brought into sRGB (thumbnail), then written without metadata, a profile picture the same way; - an animated GIF becomes a looping silent H.264 mp4 typed gifv, as on Mastodon (PostMedia.Kind keeps it a gifv), and a remote GIF is an image; - processing runs Media:Concurrency at a time, and uploads have their own rate limit per credential; - HEIC and HEIF are no longer offered. Tests: only the upload formats load, the header tells the size, an SVG posing as a PNG and an image too large are refused before decoding, an animated GIF becomes a gifv and a still one an image, HEIC isn't advertised. The media scenarios against the pasture (GoToSocial, Mastodon, Misskey, Akkoma, Pixelfed, Smithereen, Vernissage, Castopod, PeerTube) pass: 329 checks. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw |
||
|
thepraandClaude Opus 5.5
|
52d201eee9 |
A focal point is two finite numbers
float.TryParse takes "NaN" and "Infinity", and Math.Clamp keeps a NaN, so `focus=NaN,NaN` on an upload or a media PUT was stored as given. Once the media was posted, every status and timeline response holding the post, and the Note and its delivery, failed to serialise, for every viewer. FocalPoint.Parse takes finite numbers only (anything else is ignored, as an unreadable focus was); the mapper and the renderer leave out a focus that isn't sound, and migration _016 removes the ones stored before, from the uploads and from the copy each post keeps. What PrivaPub sends is unchanged for any focus that could be sent before. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw |
||
|
thepraandClaude Opus 5.5
|
fc5bb9511f |
T6: the Mastodon API over HTTP
88 integration tests drive the Mastodon client API through the whole server
(PrivaPubHost), with remote actors on an in-process Peer and deliveries read
from the job queue. Helpers live in Support/Host/MastodonHelpers.cs.
Coverage:
- Accounts: verify_credentials (no root id or login name); update_credentials
with indexed and array fields_attributes (form and JSON), source[*],
quote_policy, locked/bot, avatar and header uploads resized and stripped of
EXIF and XMP; lookup (local, @domain, remote; a circle, the instance actor
and a circle's id answer 404); search with and without resolve (only a
signed-in persona resolves, the Peer is untouched otherwise), by post and
actor address, hashtags, undiscoverable personas; account statuses with
pinned, exclude_replies, exclude_reblogs, only_media, tagged and Link paging
both ways; followers-only posts for followers (local and remote authors);
community accounts; followers/following only to their owner; follow (open,
locked, remote Follow delivery), unfollow and Undo; follow requests from
local and remote followers answered with the original Follow;
remove_from_followers; blocks with Reject and Block/Undo deliveries; mutes
with duration and the notifications choice, never federated; domain blocks;
relationships with junk ids; a banned login's tokens; reports forwarded as a
Flag from the instance actor only; account stub routes.
- Statuses: each visibility's to/cc as delivered; CW as summary; replies to
local and remote posts (mention, inReplyTo, the author's inbox); polls and
votes (local, and remote votes only to the author without published); media
attached only by its owner; quotes, the quotes list and revocation; edit
history, source and the Update delivery; delete for redraft, the 410
Tombstone and the Delete delivery; favourite/reblog counts with Like,
Announce and their Undos; favourited_by, reblogged_by; bookmarks; pins;
interaction_policy matching canQuote in the note and in the Update;
strangers get 404 for followers-only and direct posts; a located post is
unreachable by id for anyone else on every route; statuses?id[];
Idempotency-Key; scopes; deleting a reblog.
- Timelines: home paging with max_id, since_id and min_id; public local and
remote; tag (anonymous); list stub; favourites; conversations and read;
markers; notifications with types[], exclude_types[], account_id, paging,
get, dismiss, clear and unread_count.
- Instance: v1 and v2 (4.2.0 (compatible; PrivaPub)), peers, activity, rules,
extended_description, apps and every stub route.
- Media: v1 and v2 uploads, owner-only GET and PUT, 422 for unsupported or
unreadable files, video and audio made with ffmpeg lavfi sources and checked
with ffprobe; every remote media address goes through the proxy; the proxy
refuses unsigned URLs, streams ranges as 206 without caching, caches whole
downloads and serves them with ranges, and streams anything over
Media:MaxProxiedBytes (a SmallProxyHost) without caching.
- Provenance of local, delivered (signature) and fetched (instance actor,
no signature, the trigger as activity) posts, visibility of provenance,
instance descriptions; reading any of them makes no outbound request.
Pleroma reactions with EmojiReact and Undo deliveries, and local reaction
notifications.
Bugs fixed:
- remove_from_followers deleted the Follower row but never told a remote
follower. It now sends Reject{Follow} with the stored Follow id, through
RelationshipService.RemoveFollower, which Block now shares.
- VisibilityPolicy.CanSee refused followers-only posts to accepted followers,
so a post in their home timeline answered 404 to GET, context, favourite and
reply. Followers of the author (local or remote) may now see them.
- Account statuses of a remote account hid followers-only posts from
personas that follow it.
- exclude_replies dropped the author's own threads; like Mastodon it now
drops only replies to other accounts.
- A community account's statuses were always empty: they are now the posts
addressed to the community.
- Pinning someone else's visible post answered 404; it answers 422 like
Mastodon.
- GET /api/v1/notifications/:id answered 200 with null when the notification's
post was gone; it answers 404.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ELjqpznMFMNrJoJUj6K5p2
|