Releases & signing

Normative — synced from the repo

This page mirrors docs/RELEASING.md verbatim. In the lrc repo the docs are the spec: they move in the same commit as the behavior they describe.

This is the normative doc for how a chIRpChat release gets made: the workflows that build it, the manifest format that authenticates it, the secrets a maintainer must configure, and where the gaps still are. It moves in the same commit as the workflow/tooling it documents (AGENTS.md rule 3). See also research/OTA_DESIGN.md for the remote update design this local packaging feeds, and USERGUIDE.md for the user-facing “how do I flash this” story.

1. Workflow inventory

File	Trigger	Jobs	Artifacts
.github/workflows/ci.yml (existing, not owned by this doc)	push to main, PRs	host build+test (Linux/macOS), firmware build (xiao_wio_sx1262, rak4631)	none published
.github/workflows/release.yml	push of a tag matching v*	determine-channel → host-verify → build-firmware (matrix over all 6 platformio.ini envs) ‖ build-manifest-tool → sign-and-publish	per-board .bin + merged-flash .bin (ESP32 boards) + signed .lrcmanifest per board + SHA-256SUMS, attached to a GitHub Release
.github/workflows/nightly-canary.yml	schedule (07:00 UTC daily) + manual workflow_dispatch	host-verify → build-firmware (same matrix, builds main) → sign-canary-manifests (nightly key, separate from release key)	workflow artifacts only (canary-fw-*, canary-signed-manifests) — not a GitHub Release, see §3
.github/workflows/fuzz-smoke.yml	push to main, PRs	fuzz-smoke: 90s libFuzzer run of tests/fuzz/decode_fuzz.cpp, seeded from tests/fuzz/corpus/	crash artifacts, only uploaded on failure
.github/workflows/budget-report.yml	PRs touching firmware/, core/, vendor/, or the size-gate script	budget-report (matrix over all 6 envs): runs tests/check_firmware_size.py, posts a job-summary table	none (report only, see §5)

All four new files are additions; none edits ci.yml — see §6 for why, and what’s flagged as follow-up for whoever next owns ci.yml.

2. Channels

Channel is derived from the pushed tag’s shape, not a separate workflow or branch:

Tag pattern	Channel	GitHub Release prerelease	Manifest CANARY flag
vMAJOR.MINOR.PATCH (e.g. v1.2.3)	stable	false	not set
vMAJOR.MINOR.PATCH-beta.N (e.g. v1.2.3-beta.1)	beta	true	not set
anything else matching v*	—	—	the determine-channel job fails loudly rather than guessing

Nightly builds of main are a third, distinct thing — see §3.

3. Nightly canaries — artifact-only by default

Decision (the open question the WS-REL brief asked to be resolved with a stated default): nightly canary builds publish workflow artifacts, not a GitHub prerelease. Rationale:

• main moves multiple times a day in this repo, and AGENTS.md’s charter is that breaking changes are welcome but never accidental — that’s a healthy property for a green-field codebase, but it means an arbitrary nightly main snapshot has not been through the same “somebody decided this is release-worthy” gate a tag has.
• A nightly GitHub Release object would spam the Releases page (365+ entries/year) and visually imply the same curation level as a tagged release, which is misleading.
• A 14-day-retention workflow artifact is sufficient for “grab last night’s build and see if a fix landed” without polluting the tag/release namespace or requiring cleanup automation.

Revisit this if the project adopts a merge-queue/required-review gate that makes every commit on main release-worthy on its own — at that point a nightly prerelease channel would carry more signal.

Canary builds are signed with a separate, less-privileged key (LRC_NIGHTLY_SIGNING_KEY, distinct from LRC_RELEASE_SIGNING_KEY) and every canary manifest sets the CANARY flag (OTA_DESIGN.md §3/§6). This bounds the blast radius of a compromised nightly runner or its secret to canary-flagged offers only — it can never forge a stable- or beta-channel manifest. If the nightly signing secret isn’t configured, the workflow publishes unsigned firmware artifacts and prints a warning rather than failing the whole run; signed canary manifests are additive, not a hard requirement for the nightly job to be useful.

4. Manifest format — implemented vs. OTA_DESIGN.md §3

tools/lrc_manifest/manifest.h is the one implementation. It follows OTA_DESIGN.md §3’s field list exactly; the one thing that doc leaves implicit is the exact byte count, which this implementation makes concrete:

magic            "LRCF"      4 bytes
manifest_ver     u8          1 byte   (this codec: version 1)
fw_major         u8          1 byte  \
fw_minor         u8          1 byte   } fw_semver
fw_patch         u8          1 byte  /
fw_build         u32 LE      4 bytes /
board_id         u16 LE      2 bytes
profile          u8          1 byte   (A=1 field-hotspot, B=2 uplink-router,
                                        C=3 serial-only)
image_size       u32 LE      4 bytes
image_sha256     [32]        32 bytes
min_maj          u8          1 byte  \
min_min          u8          1 byte   } min_running_semver
min_patch        u8          1 byte  /
flags            u8          1 byte   (bit0 CANARY, bit1
                                        REQUIRES_STAGED_CONFIG)
-------------------------------------- signed region: 55 bytes
ed25519_sig      [64]        64 bytes (over all preceding bytes)
-------------------------------------- full manifest: 119 bytes

No deviation from OTA_DESIGN.md §3’s field list. The doc’s prose lists min_running_semver (3 bytes) without spelling out min_maj/min_min/ min_patch as separate bytes — this implementation makes that split explicit (three u8 fields, not a packed/varint encoding) because it is the simplest reading and matches how fw_semver’s three version bytes are already broken out in the same sentence. If OTA_DESIGN.md is ever revised to specify a different min_running_semver packing, this is a manifest_ver bump, not a silent reinterpretation — decode() already rejects any manifest_ver other than the one it implements.

board_id is a new host-side numeric table (tools/lrc_manifest/manifest.h enum class BoardId), not something OTA_DESIGN.md itself enumerates — it had to be invented to make the field concrete:

board_id	env (firmware/platformio.ini)
1	xiao_wio_sx1262
2	rak4631
3	heltec_v3
4	heltec_v4
5	xiao_wio_uplink_router
6	xiao_wio_usbnet

New boards append; IDs are never renumbered — board_id is on the wire in every manifest ever published, and a stale node must be able to reject a cross-flash offer forever (OTA_DESIGN.md §3’s whole reason for the field to exist).

CLI / API contract (for WS-OTA, Wave 3)

tools/lrc_manifest is both a library (manifest.h/manifest.cpp, zero dependency beyond core/src/sha256.cpp and vendor/ed25519) and a CLI (main.cpp) built by tools/lrc_manifest/build.sh:

lrc-manifest keygen <out-prefix>
    Writes <out-prefix>.pub / <out-prefix>.key (hex). THROWAWAY-key use only
    — see §7.

lrc-manifest build --image PATH --board NAME_OR_ID --profile A|B|C
                    --fw-version MAJ.MIN.PATCH [--fw-build N]
                    [--min-version MAJ.MIN.PATCH] [--canary]
                    [--requires-staged-config]
                    --privkey HEX_OR_@FILE --pubkey HEX_OR_@FILE
                    --out PATH
    Hashes --image, builds + signs the manifest, writes it to --out.
    Self-verifies before writing (refuses to publish an unverifiable
    manifest — a mismatched --privkey/--pubkey pair is a build error, not
    a silent bad release).

lrc-manifest verify MANIFEST_PATH PUBKEY_HEX_OR_@FILE [--image PATH]
    Exit 0 on success, 1 on any failure (bad signature, malformed manifest,
    or — with --image — an image whose size/SHA-256 doesn't match the
    manifest). Every failure prints a specific, human-readable reason.

lrc-manifest show MANIFEST_PATH
    Decodes and prints all fields, no verification — for eyeballing a
    manifest in CI logs.

WS-OTA can either shell out to this CLI (the release/nightly workflows already do exactly this — it is a known-good integration pattern) or link manifest.h/manifest.cpp directly into a future on-device/lrcd verify path. Either way, the byte layout above will not change without a manifest_ver bump, and decode() already rejects any manifest whose manifest_ver it doesn’t recognize — an on-device verifier can trust that a successful decode() means “a layout I understand,” full stop.

5. The profile gap — how far boards × profiles is real today

The WS-REL brief asked for a build matrix of boards × node profiles (A field-hotspot / B uplink-router / C serial-only, per ROADMAP.md). Profiles are real, wired build-time selectors today (core/include/lrc/profile.h’s LRC_PROFILE_FIELD_HOTSPOT / _UPLINK_ROUTER / _SERIAL_ONLY, consumed by firmware/platformio.ini’s -DLRC_PROFILE=... per env) — but they are not yet a full cross product. Today’s firmware/platformio.ini envs map to exactly one profile each:

env	profile
xiao_wio_sx1262	A (field-hotspot)
xiao_wio_uplink_router	B (uplink-router)
rak4631	C (serial-only)
heltec_v3	A (field-hotspot)
heltec_v4	A (field-hotspot)
xiao_wio_usbnet	A (field-hotspot)

So the release matrix in practice is 6 board/profile pairs, not a 6-board × 3-profile = 18-cell matrix. Every board that can run as a router or a serial-only node today has to do so via a runtime role/config change on top of an A-profile image (see role client|router <id>|relay in the console command list), not via a distinct profile-B/C build+release artifact for that board. release.yml and nightly-canary.yml both build exactly the envs platformio.ini defines and tag each resulting manifest with the profile that env’s LRC_PROFILE define actually is — they do not synthesize profile variants that don’t exist as build targets.

Closing this gap is firmware-env work (adding e.g. heltec_v3_router / heltec_v3_serial envs with the appropriate -DLRC_PROFILE), which is explicitly not this wave’s file ownership (firmware source/env additions belong to whichever workstream owns firmware/). This doc and the release workflow are written so that the day new profile-variant envs land in platformio.ini, they show up in the release matrix by adding one line to each workflow’s matrix.env list and one line to each ENV_TO_PROFILE map — no structural change needed.

6. What this doc’s workflows deliberately do NOT do

• They do not edit .github/workflows/ci.yml. Per this wave’s file ownership, existing workflow files are read-only to this workstream. budget-report.yml calls tests/check_firmware_size.py — the same size-gate script — as a standalone report; it does not fork the size-gate logic, but it also does not make it a hard blocking gate inside ci.yml’s firmware job, because doing that requires editing ci.yml. Flagged follow-up: whoever next owns ci.yml should wire tests/check_firmware_size.py into the firmware job as a real gate (it currently runs in exactly one place, budget-report.yml’s PR-triggered report, which is continue-on-error for the size numbers themselves but still fails the job step if the script’s own exit code is nonzero).
• They do not touch core/, daemon/, web/, sim/, or firmware source. tools/lrc_manifest deliberately does not hook into the root CMakeLists.txt test list either — see tools/lrc_manifest/build.sh and tools/lrc_manifest/tests/run.sh’s file headers for why (this wave’s worktree-parallel agents would otherwise collide on that file).
• They never create a real tag, Release, or push anything. Everything in this workstream is verified by local execution: ./tools/lrc_manifest/ build.sh, ./tools/lrc_manifest/tests/run.sh, and YAML-parsing the workflow files. No git tag, no gh release create, no network calls were made while building this.

7. Signing keys — secrets a maintainer must configure

Secret	GitHub Environment	Used by	Format
LRC_RELEASE_SIGNING_KEY	release-signing	release.yml	hex-encoded 64-byte orlp-ed25519 expanded private key (the exact format lrc-manifest keygen writes to *.key)
LRC_RELEASE_PUBLIC_KEY	release-signing	release.yml	hex-encoded 32-byte Ed25519 public key matching the above. Not secret by itself (it ships baked into every firmware image per OTA_DESIGN.md §2) but kept in the same protected environment so it can never drift from the private half it’s paired with.
LRC_NIGHTLY_SIGNING_KEY	nightly-signing	nightly-canary.yml	same format, a different, less-privileged keypair used only for CANARY-flagged manifests
LRC_NIGHTLY_PUBLIC_KEY	nightly-signing	nightly-canary.yml	public half of the above

Environment protection setup (do this before the first tag push)

1. Repo Settings → Environments → New environment → name it exactly release-signing.
2. Add required reviewers (at least one human maintainer) so every release.yml run that reaches the sign-and-publish job pauses for manual approval before it can read the signing secret — this is the actual control that keeps “push a tag” from being equivalent to “unilaterally publish a signed release.”
3. Add the two secrets above scoped to that environment (Environment secrets, not repo-level secrets — an environment-scoped secret is only readable by a job that declares environment: release-signing, which sign-and-publish does).
4. Repeat with a nightly-signing environment for the nightly key pair. Required reviewers are optional there (it runs unattended on a schedule) but the environment-scoping still matters: it keeps the nightly key out of jobs that don’t need it, and out of pull_request workflow runs entirely (GitHub does not expose environment secrets to workflow runs triggered by a fork PR).
5. Generate the real keypairs offline, on a machine that never runs CI, with tools/lrc_manifest/build.sh && ./tools/lrc_manifest/build/lrc-manifest keygen release (or nightly). Paste the two hex files’ contents into the GitHub secrets UI, then delete the local .key file or move it to an offline store — it must never be committed (.gitignore already blocks *.lrckey, but these are plain .key/.pub hex files from a different tool, so delete them by hand).

THROWAWAY-key warning: lrc-manifest keygen reads /dev/urandom directly and is meant for two things only — generating the real keys above offline, and generating disposable keys for local/CI-internal testing of the tooling itself (that’s what tools/lrc_manifest/tests/manifest_tests.cpp does, with fixed seeds so the tests are deterministic — those are not even real random keys, let alone release keys). No private key is ever committed to this repository, generated inside a workflow run and left lying around as an artifact, or reused between the two environments above.

8. Local verification (the bar this workstream was held to)

No tag was pushed and no GitHub Release was created while building this — everything is checked by running it locally:

# Build the CLI
./tools/lrc_manifest/build.sh

# Run its unit tests (codec roundtrip, tamper detection, board/profile
# table pins, hex helpers, wire-length arithmetic)
./tools/lrc_manifest/tests/run.sh

# Manual end-to-end: keygen -> build -> show -> verify (+ negative cases:
# wrong key, tampered image, truncated manifest, bit-flipped signature —
# all confirmed to fail loudly with a specific message and exit 1)
./tools/lrc_manifest/build/lrc-manifest keygen /tmp/testkey
./tools/lrc_manifest/build/lrc-manifest build --image firmware.bin \
  --board xiao_wio_sx1262 --profile A --fw-version 1.0.0 \
  --privkey "@/tmp/testkey.key" --pubkey "@/tmp/testkey.pub" \
  --out /tmp/m.lrcmanifest
./tools/lrc_manifest/build/lrc-manifest verify /tmp/m.lrcmanifest \
  "@/tmp/testkey.pub" --image firmware.bin

# Workflow YAML syntax
python3 -c "import yaml, glob; [yaml.safe_load(open(f)) for f in glob.glob('.github/workflows/*.yml')]"

# Full host verify loop, unaffected by anything in this workstream
cmake --build build -j && ./build/lrc_tests   # CI-gated host suite, still green