Storage

Three storage tiers with strictly separated jobs:

Tier	Medium	Holds	Loss tolerance
Hot	PSRAM (8 MB on XIAO ESP32S3)	channel rings, reassembly buffers, dedup rings, directory LRU, runtime identity-record cache, DCC staging	lost on reboot — fine
Warm	Flash, log-structured store	router channel history, DM mailboxes, telemetry rollups	survives reboot
Cold	Flash, K/V store	identity records, channel records (founder/ops/bans/modes/topic/JOIN key), config, peer/registration table, sequencer/client cursors	must never corrupt

The built DM mailbox slice uses hot queues on the recipient router and flushes them when the local IRC identity reconnects. The core exposes a MailboxStore seam and has reboot tests for reloading those queued records; lrcd --state-dir wires that seam to per-recipient mailbox files. The portable LrclogMailboxStore adapter is also built and host-tested: it writes append-only per-recipient mailbox snapshots into lrclog using caller-provided scratch storage, reloads the latest snapshot after reboot, and is covered by a full Node offline-DM reboot test over the warm log. XIAO ESP32S3 and Heltec V3 ESP32S3 firmware builds use partitions_lrc_8mb.csv, mount the labelled lrclog partition through the FlashArea seam, and expose that store through a deferred mailbox wrapper so radio/peer packet handling only queues encoded snapshots; physical appends and segment erases run from the firmware service loop. Firmware images without a labelled lrclog partition, including the nRF52 RAK4631 serial-only build, keep mailbox persistence disabled instead of failing boot. Loaded snapshots are treated as a mailbox set keyed by (SRC, MSGID) before delivery: duplicate live rows are dropped with first row winning, deletion tombstones win over live rows for the same key, and rows are re-keyed to the recipient UID named by the snapshot key before the cleaned snapshot is saved back through the same seam. Warm stores accept only version-2 tombstone-capable LCMB snapshots — the legacy version-1 read path is removed — and write version-2 tagged rows, where tombstones carry only SRC, MSGID, and their suppression horizon instead of fake empty message text.

The channel-history warm-log seam is also built in portable core. Node can be configured with a ChannelHistoryStore; local router-owned CHANMSG frames and accepted remote mirror frames are saved into it, and lazy channel backfill reloads retained records into the PSRAM ChanStore ring after a router reboot. LrclogChannelHistory stores those records as scoped CHAN4+RTR2+SEQ3 append-only records and host tests cover low-level rebuild, SEQ3 wrap, invalid input rejection, and IRC backfill after a simulated router reboot. It lazily builds a PSRAM resident sparse index of retained channel records, where each entry points at a warm-log record reference bound to the segment generation observed at scan/append time; point reads recheck the segment header and CRC before returning bytes, so reclaimed flash offsets cannot be mistaken for old history. Firmware uses a bounded deferred channel-history wrapper over the same labelled lrclog partition, so packet handling queues StoredMsg records and the service loop performs physical appends and segment erases off the radio path. The index can replay retained ranges older than a small hot ChanStore ring, and Node can use that prepared index to serve a complete CHANSYNC request when the hot ring no longer holds the range. A request never triggers a cold flash scan; if the prepared warm index cannot prove it has the complete range, the node falls back to the hot-cache partial reply behavior and lets the request continue upstream.

ESP32 partition plan (8 MB flash)

nvs        20 KB     ESP-IDF NVS — radio calibration, boot counters
otadata     8 KB
app0        2 MB     firmware (OTA slot A)
app1        2 MB     firmware (OTA slot B)
lrckv     512 KB     cold tier — chIRpChat K/V store
lrclog    3.4 MB     warm tier — log-structured message store

(4 MB parts: single app slot, lrckv 256 KB, lrclog 1.2 MB. lrcd maps the same tiers onto directories; a Pi router’s “flash” is just ext4, with sizes in the config.)

Why not LittleFS for messages

LittleFS (Meshtastic’s choice) is fine for config blobs but pathological for high-rate small appends: each file update rewrites metadata blocks, wears the same regions, and a power cut mid-write can force a slow fs check. Our warm tier is append-only by construction, so we use the simplest structure that is also the most flash-friendly:

`lrclog`: a segmented append-only store (LSM-lite)

The partition is divided into 64 KB segments. One segment is active; records append sequentially ([u16 len][u8 type][record bytes][u32 crc]). Erases happen one whole segment at a time, only when reclaiming the oldest — exactly the wear pattern NOR flash wants, no write amplification, no compaction stalls.
Records: sequenced channel messages (keyed CHAN4+RTR2+SEQ3), mailbox snapshots, telemetry rollups. Mailbox snapshots are latest-wins records: saving an empty snapshot clears that UID’s mailbox while older snapshots age out through normal segment reclaim; saving a tombstone-bearing snapshot suppresses older live rows for the same (SRC, MSGID) when the snapshot is reloaded or future standby set-union is applied. The mailbox snapshot helper now also has a host-tested deterministic merge rule for standby exchange: the recipient UID is restored from the snapshot key, tombstones win over live rows and keep the longest suppression horizon, and duplicate live rows collapse by a stable horizon/content tie-breaker. MAILBOXSYNC uses LCMI inventories for lower-bandwidth set repair: each inventory row carries (SRC, MSGID), expiry, tombstone state, and the first 64 bits of SHA256("LCMB-row-v1" || DST || SRC || MSGID || EXPIRES || KIND || len(FROM) || FROM || len(TEXT) || TEXT) for live rows, with the string fields omitted for tombstones. The digest is only a repair hint between authenticated peers; exact data still moves as LCMB row subsets and is applied through the same tombstone-wins merge rule. Inventories, requests, and row subsets use bounded LPP fragmentation when they do not fit one frame, and Node persists the merged snapshot while keeping delivered tombstones from resurrecting stale standby live rows.
Indexes live in PSRAM, never flash: lrclog exposes generation-bound record references, and the core channel-history adapter builds a sparse CHAN4+RTR2+SEQ3 → record-ref PSRAM index over retained channel records. The adapter still rebuilds each configured hot ChanStore ring on demand, but it can also replay retained warm-history ranges directly from verified refs without rewriting flash metadata.
Writes are buffered in PSRAM and flushed at segment-page granularity (256 B) either when full or at a 5 s tick — big synchronous flash writes never sit on the radio/logic path; the flush runs from the idle task. A crash loses at most the unflushed tail (acceptable: the network’s CHANSYNC recovery refills it from peers — the network is the real WAL).
Reclaim = erase oldest segment(s) when free segments < 2. Retention is therefore size-based and honest: a busy router holds less history time than a quiet one. /msg *lrc stats store reports the current horizon.

`lrckv`: the cold tier

Tiny copy-on-write K/V (record = key + len + value + crc, compacted through alternating 4 KB journal pages for atomicity — a miniature, auditable LittleFS-without-the-filesystem). Everything in it is small and rarely written: verified identity records, channel admin records, config, lane plans, peer list, local identity seeds, the next assignable SEQ3 for each router-owned (CHAN4,RTR2) stream, the client’s latest accepted last_seq for each mirrored remote stream, and each local identity’s joined IRC channel list, plus bounded firmware crash reports. lrcd --state-dir stores the client cursor side as clientseq-*.last files and joined-channel state as joins-*.lst files; ESP32 firmware mounts the labelled lrckv partition when present and uses the same s<userhash> seed keys as the older NVS store, plus bounded deferred adapters for q<chan4>r<rtr> router next-seq keys, c<chan4>r<rtr> client cursor keys, and j<uidhash> joined-channel snapshots. Verified cached identity records are deferred into signed identity payloads under i<uid16> keys. Router-owned channel admin records are deferred into h<chan4>r<rtr> records: the ESP32 format keeps modes to 8 bytes, JOIN keys to 64 bytes, topics to 96 bytes, and op+ban UID entries to 8 total so each signed/current channel record fits the 256-byte K/V value cap. Router-local registration rows are deferred into r<uid16> records containing the router UID, client UID, IK pubkey, SK, caps, last-seen time, and nick so a rebooted router can keep verifying already-registered clients’ tagged frames while clients eventually refresh through normal HELLO/WELCOME. Restore rejects rows written by a different router key and rebases last_seen into the current boot’s monotonic clock before normal idle expiry runs. Signed Admin safe policy/config values are stored as one compact admin record in a single canonical format: ADM1 magic, a u16 present-mask, then a fixed array of 16 u32 safe-key values in the order telem.interval_ms, history.time_skew_guard_s, directory.limit, mailbox.ttl_ms, mailbox.limit_per_uid, checkpoint.message_interval, checkpoint.time_interval_ms, checkpoint.verify_window, checkpoint.audit_limit, relay.store_forward_limit, relay.store_forward_ttl_ms, admin.quorum, admin.quorum_window_ms, rtrsync.interval_ms, rtrsync.delta_ttl_ms, and rtrsync.tombstone_ttl_ms. Decode accepts exactly that one length and rejects any other; it verifies the magic, rejects unknown mask bits, and range-checks every present value before applying it. A field is restored only when its mask bit is set. Firmware peer policy is stored as compact peers bytes (PEP1, bounded target count, bounded pin count, {port,host} targets, then UID8 pins), with up to three TCP targets and six pinned peer UIDs in one 256-byte K/V value. The first target is also mirrored as legacy peer0 bytes (PER1, port, host length, host), and old peer0 records load as a one-target policy, so field nodes keep redialing the same lrcd backbone after reboot and can roll back to older firmware. Current firmware loads the bounded target list into runtime peer slots and redials all stored targets; if the policy contains stored pins, firmware requires the same authenticated peer-link handshake that lrcd uses before a lane becomes ready. Configured lane plans are stored as compact laneplan bytes (LNP1, lane count, then {lane_id,preset,freq_slot,period_s,offset_s,window_s} for each lane). Restore accepts only bounded plans with lane 0 as the always-on P0 anchor, unique lane IDs, presets below 8, and sane non-anchor windows. This record stores only configured schedule data; BEACON runtime fields such as router fingerprint, time, and boot_id are filled when the beacon is emitted and are never restored from cold storage. Allowlisted signed Admin can update or clear this record through lane.plan; that value is not stored in admin.cfg and does not change RF retune policy. ESP32 firmware now consumes stored anchor-only plans for BEACON construction by default; valid non-anchor plans stay stored but are advertised and used only when the local device config has lane retune on. The same partition also retains up to four compact xNN crash records captured on boot after abnormal ESP32 reset reasons, then cleared when an operator reads them. If the partition is absent or fails to mount, ESP32 falls back to the older NVS seed/cursor/joined-channel keys, keeps cached identity records, channel admin records, safe Admin config, and the TCP peer target volatile, keeps the registration table hot-only, and disables retained crash reports; nRF52 continues to use LittleFS files. The sequencer value is restored before the first post-boot stamp and must never be restored to the 0xFFFFFF unstamped sentinel. Hot-tier backfill walks retained records in SEQ3 order, skipping that sentinel even when a request range crosses 0xFFFFFE → 0, so scrollback and CHANSYNC replay use the same wrap discipline as stamping. The currently shipped channel-record slice persists the founder UID, op UIDs, ban UIDs, simple mode string, topic, keyed-JOIN secret when +k is set, and replication revision for each router-owned channel. Every write is read-back-verified. Verified identity records for loaded channel founders and ops are treated as pinned cold-cache rows: ordinary cached peers stay bounded by the configured limit, but authority records are not deleted merely to make room for unrelated lookups. Loss of lrclog loses scrollback; loss of lrckv loses who owns #pizza and risks a backward sequence jump — hence the paranoia asymmetry.

On lrcd, --state-dir supplies the same cold seams with POSIX files: identity seeds (*.seed, read by key export and overwritten by key import for that username), TUI defaults (tui.cfg), one atomic seq-<chan4>-rtN.next cursor per router-owned channel stream, one clientseq-<chan4>-rtN.last cursor per mirrored remote stream, idrec-<uid>.idr serialized verified identity records, idrot-<uid>.rot accepted higher-epoch CTK rotation overlays for cached peers, idrot-archive.bin bounded local and accepted-remote rotation archive rows for local audit and local CTK restart restore, idrev-<uid>.rev local identity-revoke-state records, one joins-<uid>.lst newline-separated joined-channel list per local identity, and one chan-<chan4>-rtN.rec channel-admin record per persisted channel. The current POSIX channel-record file version is 2; readers still accept version 1 records that predate the keyed-JOIN field. Signed Admin safe keys live in admin.cfg as key=value lines for telem.interval_ms, rtrsync.interval_ms, rtrsync.delta_ttl_ms, history.time_skew_guard_s, directory.limit, mailbox.ttl_ms, mailbox.limit_per_uid, relay.store_forward_limit, relay.store_forward_ttl_ms, admin.quorum, admin.quorum_window_ms, checkpoint.message_interval, checkpoint.time_interval_ms, checkpoint.verify_window, and checkpoint.audit_limit; a daemon only returns set ok for those numeric keys after the atomic write succeeds. ESP32 firmware stores the same numeric safe-key slice as the compact admin lrckv record and leaves those values volatile if the partition is absent. Stored admin.quorum still has to be available under the node’s current distinct operator allowlist when the node loads it; otherwise the node ignores that one stored value and keeps its configured quorum. Firmware stores its local bounded Admin operator UID allowlist and local lane retune enable bit with the device config (Preferences on ESP32, the LittleFS config blob on nRF52); this is local bootstrap/RF-opt-in state, not a remote Admin SET row. Signed Admin lane.plan writes the configured laneplan record on targets that wire a lane-plan store; it is not an admin.cfg row, and lrcd targets without such a store reject it as unsupported. Signed tx.enabled off updates the target’s local transmit master switch through a platform callback and is not stored in admin.cfg. Signed Admin identity.revoke / identity.unrevoke updates use the same idrev-<uid>.rev records as local /msg *lrc key revoke; they are per-router generationed denylist state, not admin.cfg rows. New lrcd records use fixed LCRV v1 bytes (generation, writer_rtr, state), while legacy empty marker files load as generation-1 revoked rows and are rewritten on the next change. Clear rows from key unrevoke/identity.unrevoke persist as tombstones so stale RTRSYNC revoke rows cannot resurrect after restart. The bounded local checkpoint archive is stored as chkpt-audit.log v2 rows with status (ok, mismatch, or unverified), covered MSGID range, signed checkpoint hash, optional channel/DM labels, and receipt time. Retained remote router TELEM rollup history is stored as telem.rollups; bounded hourly remote-TELEM aggregates are stored separately as telem.agg rows keyed by router identity and local receipt bucket. The state-dir files keep a daemon restart from reusing already-seen SEQ3 values on the same channel, let a restarted client resume CHANSYNC from its last accepted remote seq, and restore cached identity records, accepted rotation overlays, bounded rotation archive rows, local identity revokes, joined channels, safe Admin policy, recent checkpoint archive output, and the /msg *lrc routers latest, history, and aggregate remote TELEM views before normal operation resumes; those retained rollups and aggregate buckets also feed /metrics remote TELEM gauges. They also prevent founder/topic/mode/op/ban state from being forgotten before the first post-restart JOIN.

On firmware, the current build wires local identity seeds, verified cached identity records, channel admin records, router-local registration rows, safe Admin config, stream cursors, joined-channel cold seams, and one TCP peer target to the dedicated ESP32 lrckv partition when available, deferring identity/channel-record/registration/cursor/joined-channel writes out of packet handling and using NVS fallback/mirroring for older seed/cursor/join layouts; nRF52 still uses LittleFS files keyed by username, (CHAN4,RTR2), or UID hash. ESP32 firmware also converts abnormal reset reasons into compact retained crash records during boot and exposes them through the shared CrashStore read-and-clear surface. This preserves the next assignable SEQ3 across router reboot, lets a client role rejoin saved channels, keeps local seed key export/import, verified identity cache, channel founder/topic/mode/op/ban/key state, restored local registrations, safe Admin SET values, configured TCP peer targets, and the configured lane plan on the cold tier. Runtime consumes bounded TCP peer target lists, stored peer pins for authenticated outbound links, stored lane plans for BEACON construction and local opt-in single-radio retuning, and field nodes get a bounded postmortem trail.

PSRAM budgets (XIAO ESP32S3, 8 MB)

Consumer	Budget
channel rings (hot scrollback, 32 chans × 64 msgs)	1.5 MB
router directory / identity LRU	1 MB
dedup (8192 × 8 B), frag buffers, queues	0.5 MB
DCC staging	1 MB
lrclog write-behind + indexes	1 MB
TUI/IRC session buffers	0.5 MB
headroom (fragmentation, growth)	2.5 MB

Allocation goes through lrc::Arena (core/include/lrc/arena.h) — a portable, heap-free bump+free-list allocator over a caller-provided region. The firmware binds a PSRAM region to it at init; hot-tier consumers register fixed-block pools (ChanStore slots, directory entries, dedup keys) whose counts are capped by the table above, so a runaway consumer can’t exhaust the budget. On a host test the region is a plain buffer; on PSRAM-less boards the region is empty and alloc() returns null, which the caller treats as “fall back to SRAM / client-only” rather than scattering #ifdefs. No PSRAM = client-only feature set (a no-PSRAM router build fails at compile time with a clear error, not at 2 a.m. in the field).

Arena is unit-tested on the host (tests/test_arena.cpp): capacity enforcement, free-list reuse, multi-pool independence, the no-region fallback, and the region-as-hard-cap property. The ChanStore/directory/dedup consumers are std::vector-backed today; adopting the Arena for them is an incremental, board-validated migration that lifts the ESP32 chan_ring ceiling (currently SRAM-bound at 16) toward the documented 1.5 MB ring budget.

lrcd is not PSRAM-bound. Its stock-client DCC sender sidecar stages accepted CTCP-originated transfer bytes in an anonymous temp file under --state-dir or /tmp, then serves initial and RESUME-selected DCCDATA chunks from that file. The bound is the existing DCC chunk sequence space, not an 8 MiB memory cap; receive-side local-client writes still use the bounded core reassembler buffer.