diff --git a/README.md b/README.md index f39dbf5..b8490b2 100644 --- a/README.md +++ b/README.md @@ -1,13 +1,15 @@ # app-stores -Two Gitea Actions that publish an app from any project's release workflow: `play` for Google Play and `apple` for -the App Store. Each is one Python file talking to the store's own web API: Google's Play Developer API and Apple's -App Store Connect API. Nothing else is needed on the runner beyond Python 3.9 or newer and `openssl`, which every +Three Gitea Actions for any project's release workflow: `play` publishes to Google Play, `apple` to the App Store, +and `changelog` writes what changed since the previous release from the commit messages, in every format a release +needs. Each is one Python file; the store ones talk to the store's own web API (Google's Play Developer API, Apple's +App Store Connect API). Nothing else is needed on the runner beyond Python 3.9 or newer and `openssl`, which every Linux and Mac runner has. No fastlane, no Ruby, no service in between. ```yaml - uses: https://git.thepra.dev/Arasaka/app-stores/play@v1 - uses: https://git.thepra.dev/Arasaka/app-stores/apple@v1 +- uses: https://git.thepra.dev/Arasaka/app-stores/changelog@v1 ``` > Status: checked against fakes of both stores built from their published API definitions (Google's discovery @@ -148,7 +150,59 @@ Neither store lets an API create an app, so the first steps happen in their cons 6. Create the TestFlight groups. An external group needs the Beta App Review information (contact and notes) filled in once. -## Notes +## Release notes: `changelog` + +**The commit messages are the changelog** (owner, 1 November 2026: every commit between two releases, the whole +message, in every place a release is shown). The action takes the commits since the previous tag of the same kind +(`prefix`: `v`, `app-v`, `server-v`; the previous one is the highest lower version among those tags), leaves out merges +and the `Co-Authored-By`, `Signed-off-by` and Claude attribution lines, and writes, into `output`: + +| File | For | +|---|---| +| `notes.md` | "What's new in ": each commit's subject as a heading, its body below; the Releases tab, a page | +| `notes.txt` | the same as plain text: Velopack's `--releaseNotes`, an update dialog, F-Droid | +| `CHANGELOG.md` | the last `history` releases, newest first | +| `changelog.yml` | the same releases for nfpm's `changelog:` (chglog), so the deb carries `changelog.Debian.gz` and the rpm its `%changelog`; bodies become one line per paragraph | +| `notes.json` | everything above as data | +| `store/en-US.txt` | the subjects, cut to 500 characters, for the `notes` folder of `play` and `apple` | + +With `release: 'true'` it also writes the notes onto this tag's release on the Releases tab: it creates the release +when there is none, and otherwise replaces only what sits between `` and `` +(adding that block at the end the first time), so a release text written by the project itself stays. Give the job +`permissions: contents: write`. + +```yaml + - uses: actions/checkout@v4 + - name: Release notes + id: notes + uses: https://git.thepra.dev/Arasaka/app-stores/changelog@v1 + with: + prefix: app-v + name: SimpleSchool + packager: Scuola di lingua Cinese di Varese + output: release-notes + release: 'true' + - run: echo "the notes are in ${{ steps.notes.outputs.dir }}, the previous release was ${{ steps.notes.outputs.previous }}" +``` + +| Input | | | +|---|---|---| +| `prefix` | `v` | the tags of this kind of release | +| `tag` | the tag that started the run | the release being made; empty on a branch, which writes everything since the newest tag as `unreleased` | +| `name` | the repository's name | the product's name in headings | +| `strip` | | a regular expression removed from the start of each subject (me-web? removes its `“me-web?” 0.15.4: `) | +| `packager` | `Arasaka di Hvozdikov Yevheniy ` | `Name
` for the deb and rpm changelogs: the package's own maintainer | +| `history` | `20` | how many releases `CHANGELOG.md` and the package changelog carry | +| `output` | `release-notes` | the folder | +| `release` | `false` | `true` writes the notes onto the Releases tab | +| `token` | the job token | only with `release` | + +A shallow checkout is fetched in full, with its tags, before anything is read. A first release lists its newest 100 +commits. The same file reads the history from Gitea's API instead of a checkout (`--gitea URL --repository +owner/name`, token in `READ_TOKEN`); that is how `thepra/packages.arasaka.software` writes the notes for every app it +publishes, so the two can never tell a release differently. + +## Store notes One file per language, the file name being the code: Google uses language and region (`en-US`, `it-IT`), Apple its own list (`en-US`, `it`, `de-DE`). Empty files are skipped. Apple takes no release notes for an app's first diff --git a/changelog/action.yml b/changelog/action.yml new file mode 100644 index 0000000..97c58d0 --- /dev/null +++ b/changelog/action.yml @@ -0,0 +1,61 @@ +name: Release notes +description: What changed since the previous tag, from the commit messages, in every format a release needs; optionally on the Releases tab +inputs: + prefix: + description: The tag prefix of this kind of release (v, app-v, server-v) + default: v + tag: + description: The tag being released; by default the tag that started the run, or everything since the newest tag + default: '' + name: + description: The product's name in headings; the repository's name by default + default: '' + strip: + description: A regular expression removed from the start of each subject + default: '' + packager: + description: Name
written into the deb and rpm changelogs + default: Arasaka di Hvozdikov Yevheniy + history: + description: How many releases CHANGELOG.md and the package changelog carry + default: '20' + output: + description: The folder the notes are written to + default: release-notes + release: + description: "'true' also creates or updates this tag's release on the Releases tab with the notes" + default: 'false' + token: + description: A token that may write this repository's releases (only with release) + default: ${{ github.token }} +outputs: + dir: + description: The folder with notes.md, notes.txt, CHANGELOG.md, changelog.yml, notes.json and store/en-US.txt + value: ${{ steps.notes.outputs.dir }} + notes: + description: The path of notes.md + value: ${{ steps.notes.outputs.notes }} + previous: + description: The previous tag, empty for a first release + value: ${{ steps.notes.outputs.previous }} +runs: + using: composite + steps: + - id: notes + shell: bash + env: + RELEASE_TOKEN: ${{ inputs.token }} + NOTES_PREFIX: ${{ inputs.prefix }} + NOTES_TAG: ${{ inputs.tag }} + NOTES_NAME: ${{ inputs.name }} + NOTES_STRIP: ${{ inputs.strip }} + NOTES_PACKAGER: ${{ inputs.packager }} + NOTES_HISTORY: ${{ inputs.history }} + NOTES_OUTPUT: ${{ inputs.output }} + NOTES_RELEASE: ${{ inputs.release }} + run: | + args=(--repo . --prefix "$NOTES_PREFIX" --name "$NOTES_NAME" --strip "$NOTES_STRIP" --packager "$NOTES_PACKAGER" \ + --history "$NOTES_HISTORY" --out "$NOTES_OUTPUT") + [ -n "$NOTES_TAG" ] && args+=(--tag "$NOTES_TAG") + [ "$NOTES_RELEASE" = "true" ] && args+=(--release) + python3 "${{ github.action_path }}/changelog.py" "${args[@]}" diff --git a/changelog/changelog.py b/changelog/changelog.py new file mode 100644 index 0000000..078a98b --- /dev/null +++ b/changelog/changelog.py @@ -0,0 +1,314 @@ +#!/usr/bin/env python3 +import argparse +import datetime +import email.utils +import json +import os +import re +import subprocess +import sys +import urllib.error +import urllib.parse +import urllib.request +from pathlib import Path + +TRAILER = re.compile(r"^(co-authored-by|signed-off-by|reviewed-by|helped-by|claude-[\w-]+):", re.I) +GENERATED = re.compile(r"generated with \[?claude code", re.I) +FIRST = 100 +STORE = 500 +START = "" +END = "" + + +def version_key(text): + return tuple(int(part) for part in re.findall(r"\d+", text)) + + +def versioned(tag, prefix): + return tag.startswith(prefix) and re.match(r"\d", tag[len(prefix):]) is not None + + +def clean(message, strip): + lines = message.replace("\r", "").split("\n") + subject = lines[0].strip() + if strip: + subject = re.sub(strip, "", subject).strip() + if subject: + subject = subject[0].upper() + subject[1:] + body = [line.rstrip() for line in lines[1:] if not TRAILER.match(line.strip()) and not GENERATED.search(line)] + while body and not body[0]: + body.pop(0) + while body and not body[-1]: + body.pop() + return subject, "\n".join(body) + + +class Git: + def __init__(self, repo): + self.repo = repo + + def run(self, *args): + return subprocess.run(["git", "-C", str(self.repo), *args], check=True, text=True, capture_output=True).stdout + + def complete(self): + shallow = self.run("rev-parse", "--is-shallow-repository").strip() == "true" + args = ["fetch", "--quiet", "--tags", "--force", "origin"] + (["--unshallow"] if shallow else []) + done = subprocess.run(["git", "-C", str(self.repo), *args], text=True, capture_output=True) + if done.returncode != 0: + print(f"::warning::could not fetch the full history and tags ({done.stderr.strip()[:200]}); using what the checkout has", file=sys.stderr) + + def tags(self, prefix): + found = {} + for line in self.run("tag", "--list", f"{prefix}*", "--format=%(refname:short)").splitlines(): + if versioned(line, prefix): + found[line] = self.run("log", "-1", "--format=%cI", f"{line}^{{commit}}").strip() + return found + + def head_date(self): + return self.run("log", "-1", "--format=%cI", "HEAD").strip() + + def commits(self, base, head): + span = f"{base}..{head}" if base else head + log = self.run("log", "--no-merges", "--reverse", f"--max-count={FIRST if not base else 100000}", "--format=%H%x1f%cI%x1f%B%x1e", span) + entries = [] + for entry in log.split("\x1e"): + entry = entry.strip("\n") + if entry: + sha, date, message = entry.split("\x1f", 2) + entries.append((sha, date, message)) + return entries + + +class Gitea: + def __init__(self, url, repository, token): + self.base = f"{url.rstrip('/')}/api/v1/repos/{repository}" + self.headers = {"Authorization": f"token {token}"} if token else {} + + def get(self, path): + request = urllib.request.Request(f"{self.base}{path}", headers=self.headers) + with urllib.request.urlopen(request, timeout=60) as answer: + return json.load(answer) + + def tags(self, prefix): + found = {} + for page in range(1, 40): + batch = self.get(f"/tags?limit=50&page={page}") + for tag in batch: + if versioned(tag["name"], prefix): + found[tag["name"]] = tag["commit"].get("created") or "" + if len(batch) < 50: + break + return found + + def commits(self, base, head): + entries = [] + query = {"sha": head, "limit": 50, "stat": "false", "verification": "false", "files": "false"} + if base: + query["not"] = base + for page in range(1, 200): + batch = self.get(f"/commits?{urllib.parse.urlencode({**query, 'page': page})}") + for item in batch: + if len(item.get("parents") or []) > 1: + continue + entries.append((item["sha"], item["commit"]["committer"]["date"], item["commit"]["message"])) + if len(batch) < 50 or (not base and len(entries) >= FIRST): + break + return list(reversed(entries[:FIRST] if not base else entries)) + + +def history(source, prefix, tag, count, strip): + dates = source.tags(prefix) + ordered = sorted(dates, key=version_key) + if tag and tag not in dates: + raise SystemExit(f"the tag {tag} does not exist") + if tag: + newer = ordered.index(tag) + heads = [(t, t[len(prefix):], dates[t]) for t in reversed(ordered[:newer + 1])] + else: + heads = [("HEAD", "unreleased", source.head_date())] + [(t, t[len(prefix):], dates[t]) for t in reversed(ordered)] + releases = [] + for position, (head, version, date) in enumerate(heads[:count]): + previous = heads[position + 1][0] if position + 1 < len(heads) else None + changes = [] + for sha, when, message in source.commits(previous, head): + subject, body = clean(message, strip) + if subject: + changes.append({"sha": sha, "date": when, "subject": subject, "body": body}) + releases.append({"tag": head, "version": version, "date": date, "previous": previous, "changes": changes}) + return releases + + +def since(release): + count = len(release["changes"]) + if release["previous"] is None: + return f"The first release, with {count} change{'s' * (count != 1)}" + (f" (the newest {FIRST})" if count >= FIRST else "") + "." + return f"{count} change{'s' * (count != 1)} since {release['previous']}." + + +def markdown(name, release, level=2): + lines = [f"{'#' * level} What's new in {name} {release['version']}", "", since(release), ""] + for change in release["changes"]: + lines += [f"{'#' * (level + 1)} {change['subject']}", ""] + if change["body"]: + lines += [change["body"], ""] + return "\n".join(lines).rstrip() + "\n" + + +def changelog(name, releases): + return f"# {name}: changes\n\n" + "\n".join(markdown(name, release, 2) for release in releases) + + +def text(release): + parts = [] + for change in release["changes"]: + parts.append(change["subject"] + (f"\n{change['body']}" if change["body"] else "")) + return ("\n\n".join(parts) or since(release)) + "\n" + + +def store(release, limit=STORE): + lines = [] + for change in release["changes"]: + line = f"• {change['subject']}" + if len("\n".join(lines + [line])) > limit - 2: + lines.append("…") + break + lines.append(line) + return ("\n".join(lines) or since(release))[:limit] + "\n" + + +def paragraphs(body): + joined = [] + for block in re.split(r"\n\s*\n", body): + current = [] + for line in block.splitlines(): + stripped = line.strip() + if re.match(r"^([-*•]|\d+[.)])\s", stripped) and current: + joined.append(" ".join(current)) + current = [] + if stripped: + current.append(stripped) + if current: + joined.append(" ".join(current)) + return joined + + +def iso(date): + return datetime.datetime.fromisoformat(date.replace("Z", "+00:00")).astimezone(datetime.timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ") + + +def chglog(releases, packager): + out = [] + for release in releases: + if release["version"] == "unreleased" or not release["changes"]: + continue + out += [f"- semver: {json.dumps(release['version'])}", f" date: {json.dumps(iso(release['date']))}", + f" packager: {json.dumps(packager)}", " deb:", " urgency: medium", " distributions: [stable]", " changes:"] + for change in release["changes"]: + note = "\n".join([change["subject"], *paragraphs(change["body"])]) + out += [f" - commit: {json.dumps(change['sha'])}", f" note: {json.dumps(note, ensure_ascii=False)}"] + return "\n".join(out) + "\n" if out else "[]\n" + + +def debian(package, revision, releases, packager): + blocks = [] + for release in releases: + if release["version"] == "unreleased": + continue + version = release["version"] + (f"-{revision}" if revision and "-" not in release["version"] else "") + lines = [f"{package} ({version}) stable; urgency=medium", ""] + for change in release["changes"] or [{"subject": since(release), "body": ""}]: + lines.append(f" * {change['subject']}") + lines += [f" {paragraph}" for paragraph in paragraphs(change["body"])] + when = datetime.datetime.fromisoformat(release["date"].replace("Z", "+00:00")) + lines += ["", f" -- {packager} {email.utils.format_datetime(when)}", ""] + blocks.append("\n".join(lines)) + return "\n".join(blocks) + + +def write(out, name, releases, packager): + out.mkdir(parents=True, exist_ok=True) + newest = releases[0] + (out / "notes.md").write_text(markdown(name, newest)) + (out / "notes.txt").write_text(text(newest)) + (out / "CHANGELOG.md").write_text(changelog(name, releases)) + (out / "changelog.yml").write_text(chglog(releases, packager)) + (out / "notes.json").write_text(json.dumps({"name": name, **newest, "history": releases[1:]}, indent=1, ensure_ascii=False) + "\n") + (out / "store").mkdir(exist_ok=True) + (out / "store" / "en-US.txt").write_text(store(newest)) + + +def release_page(server, repository, token, tag, title, notes): + base = f"{server.rstrip('/')}/api/v1/repos/{repository}/releases" + headers = {"Authorization": f"token {token}", "Content-Type": "application/json"} + + def call(method, url, body=None): + request = urllib.request.Request(url, method=method, headers=headers, data=json.dumps(body).encode() if body is not None else None) + with urllib.request.urlopen(request, timeout=60) as answer: + return json.load(answer) + + block = f"{START}\n{notes.rstrip()}\n{END}" + try: + existing = call("GET", f"{base}/tags/{urllib.parse.quote(tag)}") + except urllib.error.HTTPError as error: + if error.code != 404: + raise + existing = None + if existing is None: + made = call("POST", base, {"tag_name": tag, "name": title, "body": block, "draft": False, "prerelease": False}) + return f"created the release {tag} ({made['html_url']})" + body = existing.get("body") or "" + if START in body and END in body: + body = body[:body.index(START)] + block + body[body.index(END) + len(END):] + else: + body = (body.rstrip() + "\n\n" + block) if body.strip() else block + call("PATCH", f"{base}/{existing['id']}", {"body": body}) + return f"updated the notes of the release {tag} ({existing['html_url']})" + + +def main(): + parser = argparse.ArgumentParser(description="Release notes from the commit messages since the previous tag, in every format a release needs") + parser.add_argument("--repo", default=".", help="a git checkout (fetched in full when shallow)") + parser.add_argument("--gitea", help="read the history from this Gitea instead of a checkout") + parser.add_argument("--repository", default=os.environ.get("GITHUB_REPOSITORY", ""), help="owner/name on Gitea") + parser.add_argument("--prefix", default="v", help="the tag prefix of this kind of release (v, app-v, server-v)") + parser.add_argument("--tag", default=os.environ.get("GITHUB_REF_NAME", "") if os.environ.get("GITHUB_REF_TYPE") == "tag" else "", + help="the tag being released; empty means everything since the newest tag") + parser.add_argument("--name", default="", help="the product's name in headings; the repository's name by default") + parser.add_argument("--strip", default="", help="a regular expression removed from the start of each subject") + parser.add_argument("--packager", default="Arasaka di Hvozdikov Yevheniy ", help="Name
for deb and rpm changelogs") + parser.add_argument("--history", type=int, default=20, help="how many releases CHANGELOG.md and the package changelog carry") + parser.add_argument("--out", default="release-notes") + parser.add_argument("--release", action="store_true", help="also create or update this tag's release on the Releases tab with the notes") + args = parser.parse_args() + token = os.environ.get("RELEASE_TOKEN") or os.environ.get("READ_TOKEN", "") + name = args.name or (args.repository.split("/")[-1] if args.repository else Path(args.repo).resolve().name) + if args.gitea: + source = Gitea(args.gitea, args.repository, token) + else: + source = Git(args.repo) + source.complete() + if args.tag and not versioned(args.tag, args.prefix): + raise SystemExit(f"{args.tag} is not a {args.prefix} tag") + releases = history(source, args.prefix, args.tag, max(args.history, 1), args.strip) + out = Path(args.out) + write(out, name, releases, args.packager) + newest = releases[0] + print(f"{name} {newest['version']}: {since(newest)} Notes in {out}/ (notes.md, notes.txt, CHANGELOG.md, changelog.yml, notes.json, store/en-US.txt)") + summary = os.environ.get("GITHUB_STEP_SUMMARY") + if summary: + with open(summary, "a") as stream: + stream.write(markdown(name, newest) + "\n") + output = os.environ.get("GITHUB_OUTPUT") + if output: + with open(output, "a") as stream: + stream.write(f"dir={out.resolve()}\nnotes={(out / 'notes.md').resolve()}\nprevious={newest['previous'] or ''}\n") + if args.release: + if newest["version"] == "unreleased": + print("not a tag: nothing goes on the Releases tab") + return + server = args.gitea or os.environ.get("GITHUB_SERVER_URL", "https://git.thepra.dev") + print(release_page(server, args.repository, token, newest["tag"], f"{name} {newest['version']}", markdown(name, newest))) + + +if __name__ == "__main__": + main()