Scenario Testbed — `lrcsim`, a unix-native network laboratory

Research — non-normative

This page is synced from the repo’s docs/research/ tree: exploratory design work, forward-looking plans, or historical analysis. It may describe behavior that is not built or that shipped differently. The normative spec lives in the other docs sections.

Status: DESIGN (forward-looking, not normative). Everything here is host-testable [HOST] work — no wire change, no RF. It extends the existing tiers in ../TESTING.md; it does not replace them.

1. Why a new tier

The current ladder — lrctest units → test_node.cpp in-process sims → ChaosNet fault injection → VirtualRadio multi-lane PHY → smoke_lrcd.py real sockets — answers “is this behavior correct?” It cannot yet answer “how does the network behave?”: What happens when a node moves across three cells? When the one router on the ridge dies at hour 6? When a hilltop node’s downlink reaches 200 nodes but its uplink hears only 5? Those are scenarios — topology + physics + time + faults — and they need a dedicated laboratory with a physical-layer model, mobility, and replayable traces.

lrcsim is that laboratory: a unix-native binary (macOS/Linux, CI-runnable) that composes the pieces the codebase already has — lrc::Node is pure logic, VirtualRadio already models lanes/SNR floors/capture — into whole-network scenario runs with assertions and trace output.

2. Two execution modes, one field model

Mode A — in-process (fast, deterministic, CI default). One process hosts N lrc::Node instances wired through the field model. Virtual clock (the Clock seam), seeded Rng: a scenario run is a pure function of (scenario file, seed). Thousands of sim-hours per CPU-minute; failures bisectable by seed.

Mode B — multi-process (the “unix-native binaries” bed). Real lrcd processes (and, later, firmware images under QEMU/Wokwi if that ever pays) joined by lrcbridge: a small broker that carries opaque radio frames over UDP and applies the same field model to decide who hears what, when, and how mangled. Mode B exercises what Mode A cannot: the actual daemon binary, its poll() loop, --state-dir persistence across real process kills (SIGKILL mid-write), TCP federation alongside simulated RF, and true wall-clock timing races. CI runs a small Mode-B smoke set; the full matrix is Mode A.

The field model (shared library, used by both modes) extends VirtualRadio from per-pair SNR into geometry:

• Geometry: each node has (x, y, z) meters, TX power dBm, antenna gain. Received SNR = link budget under log-distance path loss (exponent per-scenario, 2.7–3.5 urban/rural) + optional log-normal shadowing σ.
• Asymmetry as a first-class citizen: per-node TX power and height make asymmetry emerge (the hilltop node is loud downhill but deaf to 20 mW handhelds), and per-pair directional overrides let a scenario pin exact pathologies. This is the single most important modeling requirement — the routing redesign lives or dies on asymmetric links.
• PHY realism kept honest: preset SNR floors (reuse VirtualRadio::preset_floor), co-SF collision unless capture (≥6 dB), imperfect inter-SF orthogonality (~−16 dB rejection matrix, not the “SFs are orthogonal” myth — matters for the channel-sharing workstream), duty-cycle enforcement via each node’s real AirtimeLedger.
• Terrain (cheap version): optional per-pair obstruction penalties from a height map is deliberately out of scope v1; explicit overrides cover the hill scenario without writing a propagation engine.

3. Scenario files

Flat, diff-friendly, zero-dependency format (same spirit as lrctest.h — hand-rolled parser, no TOML library):

hilltop_asym.scn

seed 42
field pathloss 3.0 shadow 2.0
node hill  role=router x=0 y=0 z=150 tx_dbm=27
node plain[1..200] role=client grid=1000x1000 tx_dbm=14
link hill->plain[*] snr=+8       # everyone hears the hill
link plain[6..200]->hill snr=off # the hill hears only plain[1..5]
link plain[1..5]->hill snr=-14   # …barely
at 60s  chanmsg from=plain[137] chan=#field text="can anyone hear me"
at 300s kill hill
at 420s revive hill              # reboot-with-persistence
mobility plain[42] waypoints (0,0)@0s (900,900)@600s
assert converged chan=#field by 900s
assert delivered from=plain[137] to=plain[1..200] ratio>=0.95
assert duty_legal all
assert no_dup_delivery all

Constructs: node groups with ranges/grids, directional link overrides, timed events (chanmsg, dm, join, kill, revive, partition, heal, skew), waypoint mobility, and an assertion vocabulary drawn from the chaos harness’s convergence invariants plus network-level metrics (delivery ratio, per-node duty legality, credit-drop counts, path-length distributions, anchor-lane occupancy).

4. Traces — the observatory’s food

Every run emits a JSONL event stream (tx, rx, drop{reason}, dedup, reg, lane, credit_debit, node_state) with virtual timestamps and positions. Consumers:

1. CI: assertion failures attach the trace as an artifact; a failure is reproducible from (scn, seed) alone.
2. The web observatory (CHIRPSCOPE workstream in ../ROADMAP.md) replays traces — scrub through a partition heal, watch the credit budget drain hop by hop, see the ERB return path diverge from the uplink. The same viewer that debugs a live network debugs a simulated one; one visualization pipeline, two sources.
3. Tuning sweeps: the credit-routing constants (routing-redesign/CREDIT_ROUTING.md §7) and contention-window calibration read distributions straight off traces.

5. The canonical scenario library

Checked into tests/scenarios/, run by CI, named in commit messages when they gate a change:

Scenario	What it proves
hilltop_asym	200-hear/5-heard asymmetric routing: registration, chat, and returns all survive gross asymmetry
router_death	critical-node failure: solo-scope degradation is documented-graceful; federated scopes fail over; revival heals via boot-id + CHANSYNC
roam_walk	a client walks across 3 cells: make-before-break roaming, mailbox flush, no dup/no gap
partition_heal	backbone splits 10 min under load; RTRSYNC anti-entropy converges without anchor-lane flooding
metro_500	500 clients / 6 routers: anchor occupancy stays under the cadence controller’s ceiling; promotion ladder drains the anchor
eu868_duty_starve	saturation under a 1% duty subband: ledgers never exceed budget; priority traffic still lands
multi_sf_share	co-channel multi-SF lanes under the −16 dB rejection matrix: measured (not assumed) concurrency win
credit_depth	delivery vs. diameter per preset; the empirical basis for the AC budget table
reboot_storm	rolling reboots with persistence: SEQ3/boot-id discipline holds; no gap storms

6. Build plan (agent-executable, in order)

1. sim/field.h — field model as pure library + unit tests (extends VirtualRadio patterns; no new deps).
2. sim/scn.h — scenario parser + tests (golden parse vectors).
3. lrcsim binary (Mode A): compose Nodes + field + events + assertions; port 3 existing chaos tests into .scn form to prove expressiveness.
4. Trace writer + assert vocabulary; wire the scenario library into CI (matrix job, artifacts on failure).
5. lrcbridge + Mode B: UDP frame bus, spawn/kill real lrcd processes, port smoke_lrcd.py’s federation cases; add the SIGKILL-mid-write persistence torture.
6. Nightly job: randomized seeds across the library; new-failure seeds get committed into a regression-seeds list (a failure found once is a test forever).

Non-goals: RF propagation research (this is a protocol laboratory — the field model needs to be adversarial and plausible, not geodetically true); replacing the chaos harness (it stays as the fast unit-grade tier); simulating silicon timing (Mode B + the bench cover that).