Conformance

How an implementation demonstrates it is correct, rather than asserting it.

The key words MUST, MUST NOT, SHOULD, SHOULD NOT and MAY are to be interpreted as described in RFC 2119.

This document is authoritative on what conformance means and what an implementation must do to claim it. conformance/ holds the data; this states the rules over it.

The corpus is the contract; the runner is not

conformance/ holds data files describing what the control protocol looks like on the wire. Every implementation ships its own runner over the same files.

That split is the whole mechanism. A fixture living inside a Rust tests/ directory is a fixture only Rust can run, which checks that one implementation agrees with itself. Data outside any crate can be read by anything — the Python binding needs no Cargo, the C# one needs no Python.

ls conformance/control/v1/encoding.json

What the corpus contains today

One file, conformance/control/v1/encoding.json, with 12 cases covering the ToRuntime and ToNode envelopes of ranvier.control.v1.

python -c "import json;print(len(json.load(open('conformance/control/v1/encoding.json',encoding='utf-8'))['cases']))"

Each case carries:

Field	Meaning
name	the case identifier a runner reports
covers	the message types the case exercises
message	the envelope — ToRuntime or ToNode
value	the message in a textual form, field by field
frame	the exact bytes on the wire, including the length prefix
note	why the case exists

The file also states the framing it assumes, which an implementation MUST match — four bytes of big-endian length, then that many bytes, ceiling 16,777,216. It is the same framing the data plane uses, specified in wire.md §1.

python -c "import json;f=json.load(open('conformance/control/v1/encoding.json',encoding='utf-8'))['framing'];print(f['prefix_bytes'],f['byte_order'],f['max_payload'])"

The frames were encoded by hand, not by prost. conformance/write-encoding.py builds each one from the field numbers in proto/ranvier/control/v1/control.proto, in about eighty lines of varints and length prefixes. That is what makes the comparison meaningful: decoding them with a protobuf library compares two independent encoders rather than one library against itself. A reviewer checking a case by hand needs only the .proto and the wire-format rules — a field is (number << 3) | wire_type, a string is type 2 with a varint length, and proto3 writes nothing at all for a field holding its default.

Regenerate with python conformance/write-encoding.py.

The two directions are different assertions

The corpus declares both, and they demand different things.

Decode. An implementation MUST read every frame and produce a value equal to the case's value. This is strict: the bytes are given and there is one correct interpretation. No encoding that produces different field numbers or a different wire type passes.

Encode. An implementation MUST produce bytes that decode to the case's value. It MUST NOT be required to produce the identical byte sequence. Protobuf permits more than one valid encoding of the same message — field ordering and varint padding among them — so asserting byte equality would fail implementations that are correct. An implementation MAY additionally check byte equality against its own previous output to detect unintended change, but that is a regression check on itself, not conformance.

What an implementation MUST do to claim conformance

1. Read conformance/control/v1/encoding.json from this repository. It MUST NOT vendor a copy. A vendored corpus goes stale, and the staleness is silent.
2. Run every case in both directions.
3. Report a case it cannot run as a failure, not a skip. A skipped case reads as a passing case.
4. Fail the build on any failing case.

An implementation that runs a subset MUST say which subset, in its own documentation, and MUST NOT describe itself as conformant.

What stops a new message from skipping the corpus

A rule nothing checks is not enforced. The Rust runner reads the committed descriptor.bin, lists every message declared in ranvier.control.v1, and fails if one of them appears in no case (crates/runtime/session/tests/conformance.rs:11-12). Adding a message without a fixture row therefore breaks the build in the repository that owns the contract, which is the only place that can catch it before three implementations diverge.

cargo test -p ranvier-session --test conformance

What is not covered

The sequence half. The corpus covers encoding. It does not cover ordering: nothing here checks that a Register precedes a publish, that a Stop is honoured within its deadline, or that a refused join ends the connection. Those are stated as requirements in control.md and are unenforced across implementations.

What the sequence half would be is one ordered transcript per scenario — the direction of each frame and the expected end state. The transcript file format is itself unspecified; three implementations will have to parse it, and whether a transcript can assert timing portably is an open question nobody has tried.

Eight of the ten scenarios are covered in this repository by tests in crates/runtime/session/tests/plane.rs, which spawn real processes against a real listener and are named for the scenario each holds down. device-returns and device-swapped are not; the identity half of the second is a_swapped_device_is_reported_and_the_old_address_held. That checks the Rust implementation and checks nothing about a second one, which is exactly the gap the transcripts exist to close.

Everything except the control plane. There is no corpus for framing beyond what the control cases carry incidentally, none for stream sequence attribution or gap accounting, and none for the node manifest.

Gap accounting is the most consequential of those. Get it subtly wrong in a binding and nothing fails: the samples decode, the counters read plausibly, the tests pass, and the loss numbers in the resulting study are not comparable with the ones another implementation produces for the same session. The rules are specified in streams.md §3 and enforced by no corpus.

The state of this today

One implementation runs the corpus, and there is only one implementation to run it. The runner is crates/runtime/session/tests/conformance.rs. Neither the JavaScript nor the Python binding reads the corpus, because both sit on the C ABI and do not speak the wire at all. A binding over the ABI cannot disagree with the corpus, because the bytes are produced by the same Rust the runner checks. What that closes is the disagreement between these two and Rust; what it leaves entirely open is the case below.

grep -rln "encoding.json" bindings/javascript/ bindings/python/ --include=*.ts --include=*.py --include=*.mjs --exclude-dir=node_modules --exclude-dir=dist --exclude-dir=target | wc -l | tr -d ' '

When that check starts failing, this section is what has to change. It asserts zero bindings read the corpus; the day one does, the build fails here and this paragraph is rewritten. That is the point.

So the corpus currently checks that Rust agrees with a fixture Rust's authors wrote, which is weaker than it looks and is not what the corpus is for.

The corpus has a generator and no regeneration gate. conformance/write-encoding.py produces encoding.json, and nothing in continuous integration checks that the committed file matches what the generator produces — unlike the wire contract, which has exactly that gate. An implementation MUST NOT assume the corpus is current until that check exists.

Both are gaps in this repository rather than in the specification, and both are recorded here rather than left as an absence, because an absence is the one thing nobody notices.