Pasture: Funkwhale joins, closing wave 1

Funkwhale 2.0.11: the API, a Celery worker with its beat and the front's nginx sharing one volume, on the shared Postgres and Redis, trusting the pasture's CA through REQUESTS_CA_BUNDLE; fwuser and its OAuth token made by its own tools. scenarios/funkwhale.sh passes its 16 checks: alice follows a channel, a track uploaded to it arrives as Audio with its file and duration, its deletion reaches PrivaPub, the unfollow, statistics. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LsXgEaXee4GCU1hwYgPJXw
2026-10-05 12:02:25 +02:00
1 parent 9f0b25e92b
commit 8ae265c8f8
@@ -560,6 +560,11 @@ tools/pasture/run.sh down                                            # removes e
  Application actor, `relay`, publishes the agenda. Its API takes a token from an OAuth password grant
  (`/oauth/login`, client_id `self`); an event is a multipart POST (`--form-string` for text starting with `<`).
  `scenarios/gancio.sh`, 17 checks.
+- **Funkwhale (2.0.11):** the API (gunicorn), a Celery worker with its beat and the front's nginx, sharing the
+  `pasture-funkwhale-data` volume, on the shared Postgres (database funkwhale) and Redis (dbs 8 and 9); Python's requests
+  trusts the pasture's bundle through `REQUESTS_CA_BUNDLE`. `funkwhale-manage fw users create` makes fwuser and an OAuth
+  access token is made in its Django shell. Its API is `/api/v1` and `/api/v2` (uploads); a track is a fresh tone made
+  by ffmpeg in its container, since it skips a file it already has. `scenarios/funkwhale.sh`, 16 checks.
 - **SecureMode:** `PRIVAPUB_ENV="Federation__SecureMode=true" run.sh up …`, as production runs. A check of what an
  unsigned reader sees uses `unserved` and `gone_unsigned` (`lib/interop.sh`), which expect 401 when SecureMode is on and
  404 or 410 when it is off.
@@ -28,6 +28,7 @@ and every run starting clean, with signed fetches required (as privapub.thepra.d
 - **Friendica 2026.05**
 - **Mobilizon 5.2.4**
 - **Gancio 1.28.2**
+- **Funkwhale 2.0.11**
 - in the town only (a seeded community checked server by server): **Hollo 0.9.19**, **Iceshrimp.NET 2026.1.2-beta**,
  **Pleroma 2.10.2**

@@ -831,11 +831,22 @@ FEP-8a8e (draft) is the common reference.
  - `Create{Audio}` whose `url` is an array: a page link plus the stream, with `bitrate` and `size`.
  - Also `duration`, `position`, `disc`, `album`, `license` and `image`; `summary` is a hashtag line (W2).
  - `Listen{Track}`.
+  - Its Accept names our Follow under an id of its own on our origin (`…/alice#follows/<uuid>`), and its own id is
+    that plus `/accept`: PrivaPub refused it as off its actor's origin (400) until 2026-10-05, so no follow of a
+    channel ever completed.
+  - A channel deletes uploads in one `Delete` with no id, the uploads' ids in a list as the object's `id`: PrivaPub read
+    no id there until 2026-10-05.
+  - Its NodeInfo discovery names the document under its swagger schema's URL as `rel`; PrivaPub now takes a link whose
+    path names NodeInfo. A channel's followers and following collections answer 500.
+  - **Pasture evidence (2026-10-05, Funkwhale 2.0.11, `tools/pasture/scenarios/funkwhale.sh`):** 16 checks pass: alice
+    follows a channel; a track uploaded to it arrives as Audio with its file and duration; its deletion; the unfollow;
+    statistics.
 - **Castopod:** an episode arrives as a link-only Note; the player comes from OpenGraph/oEmbed. Fetching the episode
  with the podcast type gives a `PodcastEpisode` with transcript and chapters.
 - **Owncast:** a `Service` actor; "go live" is a Note with a thumbnail.
 - **Gaps:**
-  - **P1:** pick the audio Link out of `url`, sniff its type, apply W2. Surfaces as `MediaAttachment{type: audio}`
+  - ~~**P1:** pick the audio Link out of `url`, sniff its type, apply W2~~ (Funkwhale's Audio arrives with its file and
+    duration, 2026-10-05). Surfaces as `MediaAttachment{type: audio}`
    with `meta.original.duration` and the cover as `preview_url`.
  - **P2:** duration, cover, album, position and licence (own `privapub.audio`); the Castopod transcript and chapters.
  - **P3:** `Listen` history; Owncast live state.
@@ -87,3 +87,8 @@ gancio.test {
 	tls internal
 	reverse_proxy pasture-gancio:13120
 }
+
+funkwhale.test {
+	tls internal
+	reverse_proxy pasture-funkwhale:80
+}
@@ -0,0 +1,78 @@
+# Funkwhale 2.0.11: music and podcasts. A channel is an actor of its own that publishes Audio, its files in `url[]`;
+# libraries follow libraries. The API (gunicorn), a Celery worker with its beat, and the front's nginx (which proxies the
+# API and serves media) share the pasture-funkwhale-data volume, on the shared Postgres (database funkwhale) and Redis
+# (dbs 8 and 9). Python's requests trusts the pasture's bundle through REQUESTS_CA_BUNDLE. Its admin is fwuser; a token
+# is an OAuth access token made in Django's shell.
+FUNKWHALE_IMAGE=${FUNKWHALE_IMAGE:-docker.io/funkwhale/api:2.0.11}
+FUNKWHALE_FRONT_IMAGE=${FUNKWHALE_FRONT_IMAGE:-docker.io/funkwhale/front:2.0.11}
+FUNKWHALE_PASSWORD=Funkwhale-Pasture-1
+. "$here/peers/shared.sh"
+
+funkwhale_env() {
+	local key="$here/.state/funkwhale/secret"
+	[ -s "$key" ] || { mkdir -p "$here/.state/funkwhale"; head -c 48 /dev/urandom | base64 -w0 > "$key"; }
+	cat <<ENV
+FUNKWHALE_HOSTNAME=funkwhale.test
+FUNKWHALE_PROTOCOL=https
+FUNKWHALE_URL=https://funkwhale.test
+FUNKWHALE_API_PORT=5000
+FUNKWHALE_API_HOST=pasture-funkwhale-api
+FUNKWHALE_WEB_WORKERS=2
+DJANGO_SECRET_KEY=$(cat "$key")
+DATABASE_URL=postgresql://pasture:pasture@postgres:5432/funkwhale
+CACHE_URL=redis://redis:6379/8
+CELERY_BROKER_URL=redis://redis:6379/9
+MEDIA_ROOT=/srv/funkwhale/data/media
+STATIC_ROOT=/srv/funkwhale/data/static
+MUSIC_DIRECTORY_PATH=/srv/funkwhale/data/music
+NGINX_MAX_BODY_SIZE=100M
+REVERSE_PROXY_TYPE=nginx
+REQUESTS_CA_BUNDLE=/ca/bundle.pem
+SSL_CERT_FILE=/ca/bundle.pem
+DISABLE_PASSWORD_VALIDATORS=true
+EMAIL_CONFIG=consolemail://
+LOGLEVEL=info
+ENV
+}
+
+funkwhale_manage() { podman exec pasture-funkwhale-api funkwhale-manage "$@"; }
+
+funkwhale_up() {
+	shared_postgres_up
+	shared_redis_up
+	pg_db funkwhale citext unaccent pg_trgm
+	mkdir -p "$here/.state/funkwhale"
+	funkwhale_env > "$here/.state/funkwhale/env"
+	podman volume exists pasture-funkwhale-data || podman volume create --label pasture=1 pasture-funkwhale-data >/dev/null
+	local common=(--network $net --label pasture=1 --env-file "$here/.state/funkwhale/env" -v pasture-funkwhale-data:/srv/funkwhale/data
+		-v "$ca/bundle.pem:/ca/bundle.pem:z,ro")
+	podman run --rm "${common[@]}" "$FUNKWHALE_IMAGE" funkwhale-manage migrate >/dev/null 2>&1
+	podman run --rm "${common[@]}" "$FUNKWHALE_IMAGE" funkwhale-manage collectstatic --noinput >/dev/null 2>&1
+	podman run -d --replace --name pasture-funkwhale-api "${common[@]}" "$FUNKWHALE_IMAGE" gunicorn >/dev/null
+	podman run -d --replace --name pasture-funkwhale-celery "${common[@]}" "$FUNKWHALE_IMAGE" \
+		celery -A funkwhale_api.taskapp worker -B -l info --concurrency=2 -s /tmp/celerybeat-schedule >/dev/null
+	podman run -d --replace --name pasture-funkwhale --network $net --label pasture=1 --env-file "$here/.state/funkwhale/env" \
+		-v pasture-funkwhale-data:/srv/funkwhale/data "$FUNKWHALE_FRONT_IMAGE" >/dev/null
+	for _ in $(seq 1 90); do
+		site funkwhale.test -s -o /dev/null -w '%{http_code}' https://funkwhale.test:6443/.well-known/nodeinfo 2>/dev/null | grep -q 200 && break
+		sleep 2
+	done
+	funkwhale_settle
+	echo "funkwhale: https://funkwhale.test:6443"
+}
+
+# fwuser, and an OAuth access token of its own
+funkwhale_settle() {
+	funkwhale_manage fw users create --username fwuser --email fwuser@funkwhale.test --password "$FUNKWHALE_PASSWORD" --superuser >/dev/null 2>&1 || true
+	podman exec -i pasture-funkwhale-api funkwhale-manage shell >/dev/null 2>&1 <<'PY'
+from datetime import timedelta
+from django.utils import timezone
+from funkwhale_api.users.models import User
+from funkwhale_api.users.models import Application, AccessToken
+user = User.objects.get(username="fwuser")
+app, _ = Application.objects.get_or_create(name="pasture", defaults={"client_type": "confidential", "authorization_grant_type": "authorization-code", "redirect_uris": "urn:ietf:wg:oauth:2.0:oob", "scope": "read write", "user": user})
+AccessToken.objects.filter(user=user, application=app).delete()
+AccessToken.objects.create(user=user, application=app, token="pasture-fwuser-token", scope="read write", expires=timezone.now() + timedelta(days=365))
+PY
+	echo "pasture-fwuser-token" > "$here/.state/funkwhale.token"
+}
@@ -0,0 +1,57 @@
+# Funkwhale 2.0.11: a channel (an actor of its own) and its tracks. alice_funkwhale follows the channel; a track uploaded
+# to it arrives as Audio with its file and duration; its deletion; the unfollow. Funkwhale's API takes fwuser's OAuth
+# token; a track is a fresh tone made by ffmpeg in its own container (Funkwhale skips a file it already has); what it
+# holds is read from the shared Postgres (database funkwhale).
+FWB=https://funkwhale.test:6443
+fw() { curl -sk --resolve funkwhale.test:6443:127.0.0.1 -H "Authorization: Bearer $(cat "$here/.state/funkwhale.token")" "$@"; }
+fw_sql() { podman exec pasture-postgres psql -U pasture -d funkwhale -tAc "$1" 2>/dev/null; }
+p_home_has() { curl -s -H "$PH" "$P/api/v1/timelines/home?limit=40" | j "print(next((o['id'] for o in ((s.get('reblog') or s) for s in d) if '$1' in (o['content'] or '') or '$1' in ((o.get('privapub') or {}).get('title') or '')), ''))"; }
+
+echo "funkwhale"
+[ -s "$here/.state/funkwhale.token" ] && ok "Funkwhale token for fwuser" || { ko "Funkwhale token"; return 1; }
+[ "$(fw -o /dev/null -w '%{http_code}' "$FWB/api/v2/users/me")" = "200" ] && ok "fwuser's token works" || { ko "fwuser's token is refused"; return 1; }
+PT=$(privapub_token alice_funkwhale)
+PH="Authorization: Bearer $PT"
+[ -n "$PT" ] && ok "PrivaPub token for alice_funkwhale" || { ko "PrivaPub token for alice_funkwhale"; return 1; }
+
+echo "  the channel and its follower"
+fw -o /dev/null -X POST "$FWB/api/v1/channels/" -H 'Content-Type: application/json' -d '{"name":"Pasture sounds","username":"pasturesounds",
+	"description":{"text":"sounds of the pasture","content_type":"text/markdown"},"tags":["pasture"],"content_category":"music"}'
+channel=$(fw "$FWB/api/v1/channels/?scope=me" | j "print(next(c['uuid'] for c in d['results'] if c['actor']['preferred_username'] == 'pasturesounds'))")
+[ -n "$channel" ] && ok "fwuser runs the channel pasturesounds" || ko "the channel could not be made"
+channel_on_p=$(curl -s -H "$PH" "$P/api/v2/search?q=pasturesounds@funkwhale.test&resolve=true&type=accounts" | j "print(d['accounts'][0]['id'])")
+[ -n "$channel_on_p" ] && ok "PrivaPub resolves @pasturesounds@funkwhale.test" || ko "PrivaPub cannot resolve the channel"
+# a run before this one left the follow: unfollow first, so the follow below is a new request
+curl -s -o /dev/null -X POST -H "$PH" "$P/api/v1/accounts/$channel_on_p/unfollow"; sleep 3
+curl -s -o /dev/null -X POST -H "$PH" "$P/api/v1/accounts/$channel_on_p/follow"
+# (Funkwhale names its Accept after our Follow, `<follow id>/accept`, which PrivaPub takes since 2026-10-05)
+until_true 45 '[ "$(curl -s -H "$PH" "$P/api/v1/accounts/relationships?id[]=$channel_on_p" | j "print(d[0][\"following\"])")" = "True" ]' \
+	&& ok "alice_funkwhale follows the channel (Accept arrived)" || ko "the channel's Accept never arrived"
+
+echo "  tracks"
+run=$(date +%s); title="A pasture tone $run"
+podman exec pasture-funkwhale-api ffmpeg -loglevel error -y -f lavfi -i "sine=frequency=$((300 + run % 500)):duration=4" -c:a libvorbis \
+	-metadata title="$title" -metadata artist="Pasture sounds" "/tmp/tone-$run.ogg"
+podman cp "pasture-funkwhale-api:/tmp/tone-$run.ogg" "$here/.state/funkwhale-tone.ogg"
+upload=$(fw -X POST "$FWB/api/v2/uploads/" -F "audio_file=@$here/.state/funkwhale-tone.ogg" -F "channel=$channel" -F import_status=pending \
+	-F "import_metadata={\"title\":\"$title\"}" | j "print(d['uuid'])")
+[ -n "$upload" ] && ok "fwuser uploads a track to the channel" || ko "the upload was refused"
+until_true 60 '[ -n "$(p_home_has "$title")" ]' && ok "the track reaches alice_funkwhale's home" || ko "the track never arrived"
+track_on_p=$(p_home_has "$title")
+track=$(curl -s -H "$PH" "$P/api/v1/statuses/$track_on_p")
+[ "$(echo "$track" | j "p = d.get('privapub') or {}; print(p.get('object_type'))")" = "Audio" ] && ok "it arrives as Audio" || ko "the track is not Audio"
+echo "$track" | j "a = (d.get('privapub') or {}).get('audio') or {}; print(bool(a.get('url') or a.get('variants')) and (a.get('duration') or 0) > 3)" | grep -q True \
+	&& ok "with its file and its duration" || ko "the track's file or duration is missing ($(echo "$track" | j "print((d.get('privapub') or {}).get('audio'))"))"
+
+echo "  deletes"
+fw -o /dev/null -X DELETE "$FWB/api/v2/uploads/$upload/"
+until_true 60 '[ "$(curl -s -o /dev/null -w "%{http_code}" -H "$PH" "$P/api/v1/statuses/$track_on_p")" = "404" ]' \
+	&& ok "the deleted track leaves PrivaPub" || ko "the deleted track stays on PrivaPub"
+
+echo "  unfollow"
+curl -s -o /dev/null -X POST -H "$PH" "$P/api/v1/accounts/$channel_on_p/unfollow"
+until_true 45 '[ "$(fw_sql "select count(*) from federation_follow f join federation_actor a on a.id = f.actor_id where a.fid like '"'%alice_funkwhale%'"'")" = "0" ]' \
+	&& ok "alice_funkwhale's unfollow reaches the channel" || ko "the channel still counts alice_funkwhale"
+
+echo "  statistics"
+stats_check funkwhale.test funkwhale