Skip to main content

The Ranvier wire format

Normative specification of the bytes two Ranvier processes exchange. Write a binding in C#, Python or JavaScript from this document and it will interoperate with the Rust implementation in this repository without reading the Rust.

The key words MUST, MUST NOT, SHOULD, SHOULD NOT and MAY in this document are to be interpreted as described in RFC 2119.

0. How to read this​

This document is authoritative. Where it and any other document in this repository disagree about the bytes on the wire, this one governs and the other is the one to correct. docs/decisions/ records why a rule was chosen; it does not state what the rule is.

Every requirement carries the file and line that establishes it, written as path:line, or a check block whose output is asserted. Paths are relative to the repository root. docs/docs.tools/check-spec.py resolves every citation and runs every check block in continuous integration, so a citation that stops resolving fails a build.

Line numbers move; the symbol named beside each citation does not. A citation that has drifted is a defect in this document — re-find it by grepping for the symbol, and fix the number here.

Requirement and accident are kept apart. Where the implementation does something nothing obliges it to do — the order it writes protobuf fields in, the size of its read buffer — this document says MAY and then says what the Rust does. Writing those down as requirements would ossify an accident, and a second implementation would be judged against a choice nobody made.

Where nothing has decided a behaviour, this document says so and stops. §8 collects every such case. A specification with visible holes is more use than one that guesses, because an implementer can see which decisions are still theirs.

Scope. Two channels use this framing: the data plane, where samples travel between nodes (§4), and the control plane, where a node and its runtime talk (§5). Both sit directly on TCP. What a sample's values bytes mean is not here: that is the schema the stream announces, and the canonical schemas belong to CA3.

1. Framing​

1.1 The problem the prefix solves​

TCP delivers a stream of bytes with no message boundaries. A publisher that writes two samples may have them arrive as two reads, as one, split down the middle of either, or a byte at a time. Each of those is TCP behaving correctly. On loopback with small messages the bytes almost always arrive in one piece, which is why a decoder written against whole messages passes its tests and then corrupts data on a real network (crates/message/src/framing.rs:5-15).

1.2 The frame​

A frame is a four-byte length followed by that many bytes of payload.

+--------+--------+--------+--------+---------------------------+
|  len   |  len   |  len   |  len   |  payload (len bytes)      |
|  MSB   |        |        |  LSB   |                           |
+--------+--------+--------+--------+---------------------------+
  • The prefix MUST be exactly four bytes (crates/message/src/framing.rs:42, pinned to the literal 4 by the test at crates/message/src/framing.rs:194).
  • The prefix MUST be an unsigned 32-bit integer in big-endian byte order — most significant byte first (crates/message/src/framing.rs:92 writes to_be_bytes, :128 reads from_be_bytes; the golden vectors at :198-209 fail for any little-endian reading in either direction).
  • The prefix MUST count the payload alone. It does not include itself. A writer emits four prefix bytes and then len bytes (crates/message/src/framing.rs:92-93); a reader takes the payload from PREFIX..PREFIX + len (:139).
  • A payload of zero bytes MUST be accepted as a complete frame, and a reader MUST NOT treat it as end of stream (crates/message/src/framing.rs:302-310). This is not a curiosity: proto3 encodes a message whose every field holds its default as zero bytes, so 00 00 00 00 is what an empty message looks like.
grep -c 'pub const PREFIX: usize = 4;' crates/message/src/framing.rs

Big-endian here, little-endian inside. The frame prefix is big-endian; protobuf's own fixed32/fixed64 fields are little-endian, by the protobuf encoding rules rather than by anything this project chose. Both byte orders appear in the worked example at §4.5, a few bytes apart. An implementation that uses one byte order for both reads timestamps that are wrong by many orders of magnitude and reports no error.

1.3 Reading​

A reader MUST accumulate bytes and yield a frame only when the whole frame has arrived. Specifically:

  1. With fewer than four bytes buffered, yield nothing and wait (crates/message/src/framing.rs:123-125). Two bytes of a four-byte length is not a length, and treating it as one produces a plausible wrong number rather than an error (:284-299).
  2. Read the length from the first four bytes.
  3. If the length is above the ceiling, fail — see §2.
  4. With fewer than 4 + len bytes buffered, yield nothing and wait (crates/message/src/framing.rs:133-135).
  5. Otherwise yield the payload and consume 4 + len bytes (crates/message/src/framing.rs:137-145).

A reader MUST produce the same frames whether the bytes arrive one at a time or a megabyte at a time (crates/message/src/framing.rs:248-269, which feeds three frames byte by byte), and MUST yield every complete frame in a single read before waiting for more (:272-281, where one read carries three frames).

A reader SHOULD bound the memory it holds by what has arrived rather than by how long it has waited. The Rust reader holds at most one incomplete frame and drains consumed bytes rather than reallocating (crates/message/src/framing.rs:144, and the ten-thousand-frame test at :364-401).

Bytes left buffered when a connection ends are a truncated stream. A reader MUST NOT deliver a frame assembled from them (crates/message/src/framing.rs:133-135 yields nothing). One implementation stands behind this rule rather than two: every binding here sits on the same Rust reader, so a reader wanting a second, independent opinion does not have one in this repository.

1.4 Writing​

A writer MUST emit the prefix and the payload as one logical unit and MUST NOT interleave another frame's bytes between them. On a socket that accepts a partial write, a writer MUST track how many bytes were actually written and resume from there; re-sending the whole buffer repeats bytes already on the wire, and the next length prefix is then read as payload — every frame after it misaligned, on a connection that reports itself as healthy (crates/runtime/node/src/transport.rs:395-436, which is what send_all exists for).

A writer MAY coalesce several frames into one socket write. The Rust publisher writes one frame per call (crates/runtime/node/src/transport.rs:368-377) and disables Nagle's algorithm so a small frame leaves when it is written rather than when the kernel has accumulated a segment (crates/runtime/node/src/transport.rs:345-347). Disabling Nagle is a MAY: it is insurance rather than a measured improvement, and the measurement that failed to show an effect is recorded at crates/runtime/node/src/transport.rs:325-336.

2. Limits​

MAX_PAYLOAD is 16,777,216 bytes (16 MiB), pinned to that literal by a test as well as by the constant (crates/message/src/framing.rs:39 and :195).

grep -c 'pub const MAX_PAYLOAD: usize = 16 \* 1024 \* 1024;' crates/message/src/framing.rs
  • A reader MUST treat a prefix claiming more than MAX_PAYLOAD as an error and MUST NOT allocate on the strength of it (crates/message/src/framing.rs:130-132). The number came from the network and there is no reason to believe it; a hostile or corrupt four bytes would otherwise ask for four gigabytes.
  • The boundary is inclusive: a payload of exactly 16,777,216 bytes is legal and 16,777,217 is not (crates/message/src/framing.rs:331-346, which pins the boundary with the four bytes 01 00 00 00 and 01 00 00 01 rather than with arithmetic on the constant).
  • A writer MUST NOT emit a frame whose payload exceeds MAX_PAYLOAD, and MUST report the refusal to its caller (crates/message/src/framing.rs:86-90). The Rust writes nothing into the output buffer when it refuses (:349-360).

What an implementation does after a TooLarge error is per channel, and the three readers in this repository do three different things. See §6.

The ceiling is not negotiated, and since 2026-08-23 it is written once: crates/message/src/framing.rs:39. It was written three times — the TypeScript and Python clients each hardcoded it — and those two are deleted; a binding over the C ABI inherits the ceiling instead of restating it. Two files still carry the number without implementing the framing, and a change to it has to reach them: conformance/write-encoding.py:348, which publishes it in the corpus, and conformance/bindings/wire.py:54, which builds a prefix deliberately above it.

3. Topology​

3.1 Two planes​

Control traffic and data traffic MUST NOT share a channel.

  • A published sample MUST travel directly from the publishing node to each subscribing node. No sample passes through the runtime: the registry is a directory consulted over the control plane at bind time, not a relay.
  • Data MUST NOT travel on the control connection.

The two planes also dial in opposite directions, which is the part most easily got wrong:

PlaneWho listensWho dials
Controlthe runtimethe node
Datathe publisherthe subscriber

3.2 The control plane's topology​

The runtime binds a loopback listener before it spawns anything, and one listener serves the whole graph, so an accepted socket is anonymous until the node names itself (crates/runtime/session/src/plane.rs:10-14).

The runtime MUST put four variables into every child's environment:

VariableValue
RANVIER_CONTROLhost:port — the runtime's listener
RANVIER_TOKENthe connection token
RANVIER_NODE_IDthe node's identity in the graph
RANVIER_PARAMSa JSON object of settings

A node reads them at crates/runtime/session/src/launch.rs:57, :63, :69 and :107. control.md §2 is authoritative on this contract, and says which of the four a node MUST read rather than only what each carries — this table said three until 2026-08-19, which is what a second copy of a rule costs. The token is a separate variable rather than part of the address so that a diagnostic can print where a node is dialling without printing the secret.

With RANVIER_CONTROL unset there is no control connection and the node is being run by hand (crates/runtime/session/src/control.rs:6-11).

RANVIER_BINDINGS MUST NOT be used. Bindings arrive on the connection as Bind messages (§5.2). A node that reads bindings from the environment cannot be told an address nobody knew when the process started.

3.3 The data plane's topology​

The publisher listens and the subscriber dials, on loopback and across devices alike. One listening socket serves any number of subscribers (crates/runtime/node/src/transport.rs:153-178), and reconnection lives in the subscriber (:185-213).

A publisher is frequently an acquisition node, and an acquisition loop never waits — bookkeeping about who has gone away belongs anywhere but there. So a publisher MUST NOT track subscribers that have disappeared, and reconnection is the subscriber's responsibility.

On one machine a publisher binds 127.0.0.1:0 — an ephemeral port, deliberately.

The transport is TCP. Nothing else is implemented.

3.4 Crossing a device boundary​

No connection may cross a device boundary in this version. Authentication and encryption do not exist, and the control plane's token is a session secret passed through the environment rather than a credential (crates/runtime/session/src/plane.rs:642-653). This is a block rather than a deferral: the work is not scheduled behind other work, it is waiting on a security design that has not been done.

A cross-device publisher would use a fixed, configurable port rather than an ephemeral one, so that an administrator can be asked to permit it. The default port, the configurable range, and whether that range is per publisher or per host are all open — see §8.

4. The data plane​

4.1 The conversation​

subscriber → publisher   Request { stream_name }        exactly once, first
publisher  → subscriber  Frame { announce }             once per stream
publisher  → subscriber  Frame { sample }               repeatedly

Both message types are defined in proto/ranvier/message/v1/transport.proto and encoded as proto3.

  • A subscriber MUST send one Request as the first frame on the connection (crates/runtime/node/src/transport.rs:447-453).
  • A subscriber MUST NOT send any other frame on that connection. The publisher reads exactly one frame and never reads the socket again (crates/runtime/node/src/transport.rs:221-223 is the only read; serve_one writes from there on). Bytes sent after the Request are not read, and any that arrived in the same read are discarded with the reader (crates/runtime/node/src/transport.rs:349-366).
  • A publisher MUST NOT send anything before it has read the Request (crates/runtime/node/src/transport.rs:221-238).

Request.stream_name is a name, not a stream. Several streams may publish under one name, and a subscriber asking for a name receives all of them — exactly as it would in the same process (proto/ranvier/message/v1/transport.proto:37-41).

4.2 The announcement rule​

A Sample carries opaque bytes and a timestamp on a clock nobody has named. The StreamInfo is what makes both readable. So:

  • A publisher MUST send Frame { announce } for a stream before the first Frame { sample } carrying that stream's stream_ref, for every stream publishing under the requested name — not the first one it finds (crates/runtime/node/src/transport.rs:238-240 sweeps at attach, :255-262 sweeps again the first time an unheard stream_ref arrives, :289-315 is the sweep). Announcing one of several leaves the subscriber holding samples it has no schema for, and holding an announcement that looks like the answer.
  • Announcements and samples interleave. A stream that starts publishing an hour into a session announces itself an hour in (proto/ranvier/message/v1/transport.proto:14-16).
  • A publisher MUST NOT send two different StreamInfo values for one stream_ref on one connection. A StreamInfo is fixed for the life of the stream it describes — every field in it is constant by construction, and the announcement is built once when the stream is opened (crates/runtime/node/src/lib.rs:769-779). The publisher tracks what it has already announced per connection and does not repeat it (crates/runtime/node/src/transport.rs:234, :300-313).
  • A subscriber MUST accept a sample whose stream_ref it holds no announcement for, and MUST NOT attribute it to another stream or guess a schema from a stream sharing the name (proto/ranvier/message/v1/transport.proto:75-81). This is a real hole rather than a theoretical one: the publisher's registry holds streams weakly, so a stream withdrawn between publishing a sample and the announcement sweep leaves nothing to announce, and the sample is sent anyway because an uncounted loss is the worse outcome (crates/runtime/node/src/transport.rs:45-52).
  • A subscriber SHOULD accumulate announcements keyed by stream_ref rather than holding one per connection (crates/runtime/node/src/lib.rs:405, Runtime::adopt_remote_stream, which keys by the reference and says at :417-425 why the name and node are not enough).

4.3 Sample​

proto/ranvier/message/v1/sample.proto. Five fields.

#NameTypeMeaning
1monotonic_timestamp_nsfixed64When the publishing node handed the observation to Ranvier
2sequenceuint64Position in this stream's own count, from 1
3valuesbytesThe measurement, shaped by the announced schema
4source_timestamp_nsfixed64When the instrument measured it, on the instrument's clock
5stream_refuint32Which announced stream this came from
  • Every timestamp in this contract MUST be fixed64, so a reader never has to ask which encoding a particular one uses (proto/ranvier/message/v1/sample.proto:43-44, :126-129).
  • monotonic_timestamp_ns MUST be a reading of the publishing host's system-wide, sleep-inclusive monotonic clock, in nanoseconds (proto/ranvier/message/v1/sample.proto:35-36; the Rust reads it at crates/runtime/node/src/lib.rs:937). It is not wall-clock time and not the instrument's own timestamp. Two such readings MUST NOT be compared unless the announcements agree on both host_id and boot_id (proto/ranvier/message/v1/stream.proto:253-267): a monotonic clock restarts at an arbitrary point on every boot, so subtracting across a reboot produces a plausible number that means nothing.
  • sequence MUST start at 1 and MUST count each stream independently (proto/ranvier/message/v1/sample.proto:55-70; crates/runtime/node/src/lib.rs:931-935). Zero is unrepresentable: proto3 omits a zero-valued scalar, so "sequence 0" and "no sequence" are identical bytes. Two streams published under one name MUST NOT share a counter.
  • A forwarder MUST NOT renumber or re-stamp a sample. The sequence number is what makes a gap mean a lost sample, and a receiver that assigned its own would destroy the one thing it is for (proto/ranvier/message/v1/transport.proto:86-89; the Rust forwards the sample unchanged at crates/runtime/node/src/transport.rs:264-274, and the republishing path passes the sequence and both timestamps through at crates/runtime/node/src/lib.rs:1022-1061).
  • values is opaque. An implementation MUST NOT require it to parse, and MUST NOT parse it in the framework layer (proto/ranvier/message/v1/sample.proto:72-96). It may hold a nested protobuf message or a packed numeric block; which one is the schema's business.
  • source_timestamp_ns MUST be passed through untouched: never rebased onto the host clock, never pre-corrected by the publisher (proto/ranvier/message/v1/sample.proto:108-112). Zero means absent, and absent is ordinary — an analysis stage or a stimulus program has no upstream clock.
  • If the announcing StreamInfo.source_clock_domain is empty, samples from that stream MUST carry source_timestamp_ns = 0. A timestamp on a clock nobody named is unmappable, so the Rust drops one given to such a stream rather than sending it (crates/runtime/node/src/lib.rs:909-914, and source_timestamp_is_carried at :869-871).
  • stream_ref MUST be ≥ 1 in any sample that names a stream; 0 means absent (proto/ranvier/message/v1/sample.proto:155-158). A receiver MUST NOT attribute a sample with stream_ref = 0 to any stream: the Rust loss tracker counts it as arrived and otherwise leaves it alone, because guessing produces exactly the false gaps the tracker exists to avoid (crates/runtime/node/src/gap.rs:172-178).
  • stream_ref is scoped to the runtime that announced it. A receiver MUST NOT assume references are unique across publishers: each runtime allocates from 1, so two publishers both have a stream 1 (proto/ranvier/message/v1/stream.proto:225-227; crates/runtime/node/src/lib.rs:199 and :760-767). A subscriber tracking sequence numbers MUST key on the pair (which connection, which reference) (crates/runtime/node/src/gap.rs:40-43, :166).

4.4 StreamInfo​

proto/ranvier/message/v1/stream.proto. Everything about a stream that does not change from sample to sample.

#NameType
1namestring
2nodestring
3schemaSchemaInfo
4originOrigin enum
5source_clock_domainstring
6host_idstring
7boot_idstring
8operatorIdentity
9authorisationAuthorisation
10stream_refuint32
11productionProduction
grep -c 'Production production = 11;' proto/ranvier/message/v1/stream.proto

Field 10 is out of sequence because it was added after fields 1-9 existed (proto/ranvier/message/v1/stream.proto:218-232). Field numbers are permanent from the first recording anyone intends to keep, and names are not (proto/ranvier/message/v1/stream.proto:15-16). A decoder MUST decode by number.

  • name MUST be non-empty. It is the only address in the system, and the runtime refuses a stream without one (crates/runtime/node/src/announce.rs:292-297).
  • stream_ref MUST be ≥ 1, for the same reason it must be in a sample (proto/ranvier/message/v1/stream.proto:229-231).
  • SchemaInfo.schema_id is the message's fully-qualified protobuf name, such as ranvier.example.v1.Reading. It is identity, not interpretation (proto/ranvier/message/v1/stream.proto:115-121).
  • SchemaInfo.descriptor is a serialized FileDescriptorSet describing that message and everything it imports. It is what lets a consumer that was never compiled against the schema read the values. A framework implementation MUST NOT parse it — carry the bytes to whoever wants them, exactly as with a sample's values (proto/ranvier/message/v1/stream.proto:122-137). A producer SHOULD announce the descriptor for the one message it carries rather than a whole descriptor set, so the announcement describes what the stream actually carries.
  • origin (field 4) says what the data is — captured, computed, or produced for display. It changes no behaviour on its own, and an implementation MUST NOT derive a loss policy from it (proto/ranvier/message/v1/stream.proto:24-28). The wire values are ORIGIN_UNSPECIFIED = 0, ORIGIN_RAW = 1, ORIGIN_DERIVED = 2 and ORIGIN_EPHEMERAL = 3 (proto/ranvier/message/v1/stream.proto:33-48).
grep -c '^  ORIGIN_' proto/ranvier/message/v1/stream.proto
  • operator records who the application said was operating, and Identity.asserted_by how it knew. An implementation MUST NOT present either as verified: Ranvier records the identity it was given and does not authenticate it (proto/ranvier/message/v1/stream.proto:50-79).
  • Authorisation.valid_from_unix_ns and valid_until_unix_ns are wall clock — nanoseconds since the Unix epoch — and not the host's monotonic clock (proto/ranvier/message/v1/stream.proto:101-110). This is the one place in this contract where a fixed64 timestamp is not on the monotonic clock, and a reader that treats it as monotonic gets a date fifty years out.
  • Production.settings is a repeated Setting pair rather than a map, and a producer SHOULD write the settings in a stable order. Protobuf leaves a map's entry order unspecified, so the same settings serialize to different bytes on two runs, which makes a content hash over an announcement unstable (proto/ranvier/message/v1/stream.proto:197-204).

4.5 A frame, byte by byte​

Frame { sample: Sample { monotonic_timestamp_ns: 1, sequence: 1, values: "A", source_timestamp_ns: 0, stream_ref: 1 } }, as the Rust encoder puts it on the wire:

00 00 00 12  12 10 09 01 00 00 00 00 00 00 00 10 01 1a 01 41 28 01

Twenty-two bytes, read left to right:

BytesMeaning
00 00 00 12frame length: 18 bytes follow. Big-endian
12Frame field 2 (sample), wire type 2
10that submessage is 16 bytes long
09Sample field 1 (monotonic_timestamp_ns), wire type 1
01 00 00 00 00 00 00 00fixed64 value 1 — little-endian, per protobuf
10 01field 2 (sequence), varint, value 1
1a 01 41field 3 (values), 1 byte, A
28 01field 5 (stream_ref), varint, value 1

source_timestamp_ns is absent: proto3 writes nothing for a scalar holding its default, which is why absence costs no bytes.

The Request that opened this connection, for gaze:

00 00 00 06  0a 04 67 61 7a 65

— six payload bytes: field 1, wire type 2, length 4, gaze.

And a Frame with no payload variant set, which is what an implementation emits if it forgets to set the oneof:

00 00 00 00

A four-byte prefix and nothing else. It is a well-formed frame carrying a well-formed Frame message, and the Rust subscriber skips it silently (crates/runtime/node/src/transport.rs:497).

4.6 Field order​

A writer MAY emit a message's fields in any order the protobuf encoding permits, and a reader MUST accept any order. Protobuf serialisation is not canonical, which is why the conformance corpus checks encoding by round trip rather than by byte equality — see conformance.md.

The Rust writes fields in ascending field-number order, which is prost's behaviour and not a requirement. An implementation that writes stream_ref first is not wrong.

4.7 Ending a connection​

Either end MAY close at any time. A publisher that drops its listening socket stops accepting and leaves open connections to finish, because cutting a publisher off mid-sample would create a loss nobody asked for (crates/runtime/node/src/transport.rs:84-106).

A subscriber that reads end-of-stream MUST treat it as the connection ending rather than as an error (crates/runtime/node/src/transport.rs:460).

5. The control plane​

proto/ranvier/control/v1/control.proto, package ranvier.control.v1. It uses the framing in §1 unchanged — four bytes big-endian, then that many bytes, with the same 16 MiB ceiling — reused rather than re-specified.

grep -c 'ranvier_message::framing' crates/runtime/session/src/plane.rs

It is a separate protobuf package from ranvier.message.v1, so that no control message appears inside the descriptor a stream announces, and so that field-number permanence — which binds at the first recording anyone intends to keep — does not bind on messages that never enter a recording (proto/ranvier/control/v1/control.proto:18-25).

What crosses this connection, and what each message means, is specified in control.md. This section covers only what belongs to the wire: who dials, the message order, and the bytes.

5.1 Who dials​

The node dials and the runtime listens — the opposite of the data plane. See §3.2 for the listener and the environment variables that carry its address.

5.2 The conversation​

node    → runtime   ToRuntime { register }               first frame, always
runtime → node      ToNode { bind }                      or
runtime → node      ToNode { refuse }  then closes
node    → runtime   ToRuntime { listening }              when its publishers are up
node    → runtime   ToRuntime { alive }                  on the period
runtime → node      ToNode { bind | transport | stop }   at any time
node    → runtime   ToRuntime { stopping }               then exits
  • A node MUST send ToRuntime { register } as the first frame on the connection (crates/runtime/session/src/plane.rs:657-660; crates/runtime/session/src/control.rs:841-854). A runtime that receives any other message first MUST close the connection (crates/runtime/session/src/plane.rs:703-708).
  • A node MUST NOT send a second Register on one connection. The Rust runtime closes the connection if it sees one (crates/runtime/session/src/plane.rs:684-689).
  • Register.protocol MUST be 1. A runtime that speaks a different version MUST refuse with REFUSE_REASON_PROTOCOL (crates/runtime/session/src/plane.rs:72 and :774-784). This is the static-versus-runtime split: a disagreement at match time is fatal, and a liveness miss forty minutes later is an observation (proto/ranvier/control/v1/control.proto:228-232).
grep -c 'pub const PROTOCOL: u32 = 1;' crates/runtime/session/src/plane.rs
  • A runtime that refuses MUST send ToNode { refuse } and then close (crates/runtime/session/src/plane.rs:692-700). A node that is refused SHOULD report it and exit non-zero; there is nothing for it to do in a session it is not part of (crates/runtime/session/src/control.rs:461-466). The Rust node does not redial after a refusal (crates/runtime/session/src/control.rs:970-973).
  • The runtime refuses for exactly five reasons, and each has an enum value: protocol, token, unknown node, port drift, already registered (crates/runtime/session/src/plane.rs:774-839; proto/ranvier/control/v1/control.proto:211-225).
  • A node MUST treat each Bind as its complete binding set, replacing whatever it held; there is no unbind verb, and a node holding a stale binding cannot exist (proto/ranvier/control/v1/control.proto:194-199; crates/runtime/session/src/control.rs:941-947). A node SHOULD treat an unchanged entry as a no-op and not tear down a working subscription.
  • A BindingInfo with an empty from means bound to this stream, address not yet known. The node MUST bring the port up and MUST NOT dial (proto/ranvier/control/v1/control.proto:179-184). A fresh Bind carries the address when the publisher's Listening arrives (crates/runtime/session/src/plane.rs:992-1001).
  • A node MUST NOT wait for a second Bind at the five-second timeout. proto/ranvier/control/v1/control.proto:180-184 describes the same binding being issued again with from still empty at the timeout. Nothing issues it. What the expiry produces is a NotComing event (crates/runtime/session/src/plane.rs:1193-1195, :1216-1220) — what expires is the runtime's claim to be waiting, not the binding.
  • A node MUST send Alive at least as often as Bind.heartbeat_period_ms, from the moment that Bind arrives. The runtime declares a node gone after three consecutive periods of silence (crates/runtime/session/src/plane.rs:92, :1170-1191), and the node reads the period on every pass rather than once, so a shorter period takes effect immediately (crates/runtime/session/src/control.rs:886-893, :923-934). A heartbeat_period_ms of 0 leaves the node on its default of two seconds (crates/runtime/session/src/control.rs:948-950; crates/runtime/session/src/plane.rs:83).
grep -n 'const MISSES' crates/runtime/session/src/plane.rs
  • Alive carries a sequence number and nothing else — no rates, no counts, no queue depths. Liveness is not health, and a message carrying both would report a node unhealthy at the moment it was working hardest (proto/ranvier/control/v1/control.proto:121-130). A node MUST send the field. A peer MUST NOT rely on another implementation acting on it: the Rust runtime counts beats and discards the message body (crates/runtime/session/src/plane.rs:971-976), so the redial detection the contract describes at proto/ranvier/control/v1/control.proto:127-129 is not performed by any implementation here.
  • A node that receives Stop SHOULD stop producing, drain, seal what it has open, send ToRuntime { stopping }, and exit within Stop.deadline_ms. At the deadline the runtime kills what is left and records that it killed rather than stopped (proto/ranvier/control/v1/control.proto:252-265; crates/runtime/session/src/control.rs:703-723).
  • A node that did not set Register.accepts_transport MUST ignore a Transport message and SHOULD report having received one (proto/ranvier/control/v1/control.proto:98-100; crates/runtime/session/src/control.rs:984-993).
  • A node whose connection closes for any reason other than a refusal MAY redial. The Rust redials on the heartbeat period, without backoff, forever, and re-sends its last Listening on the new connection because it never stopped publishing (crates/runtime/session/src/control.rs:811-828, :864-879).

5.3 A control frame, byte by byte​

From the conformance corpus, case alive-at-the-default — ToRuntime { alive: Alive { sequence: 0 } }:

00 00 00 02  1a 00
BytesMeaning
00 00 00 02frame length: 2 bytes follow
1aToRuntime field 3 (alive), wire type 2
00that submessage is 0 bytes long

An implementation that cannot read an empty submessage fails here rather than in the field (conformance/write-encoding.py:259). With sequence: 7 the same frame is 00 00 00 04 1a 02 08 07.

6. Error handling​

A specification that omits the error cases produces implementations that disagree exactly when something goes wrong. This section states every case the implementation treats as a protocol violation, and what it does.

6.1 Framing errors​

CaseWhat the Rust doesWhere
Fewer than 4 bytes bufferedWaits. Not an errorcrates/message/src/framing.rs:123-125
Prefix claims > MAX_PAYLOADReturns TooLarge { claimed } without allocatingcrates/message/src/framing.rs:130-132
Whole frame not yet arrivedWaits. Not an errorcrates/message/src/framing.rs:133-135
Payload handed to the writer > MAX_PAYLOADReturns PayloadTooLarge { size }, writes nothingcrates/message/src/framing.rs:86-90, :360

A reader MUST distinguish not yet from wrong. Ok(None) on a socket is the ordinary case and is not a failure (crates/message/src/framing.rs:120-121); treating a short read as an error produces a connection that drops under load and works on loopback.

6.2 After a framing error​

Once a prefix cannot be believed, the byte stream is no longer aligned and every byte after it is suspect. An implementation MUST NOT attempt to resynchronise by scanning for a plausible next prefix; there is no framing marker to scan for.

  • Data plane, subscriber: closes the connection (crates/runtime/node/src/transport.rs:501-505 — closing is more honest than guessing where the next one starts).
  • Data plane, publisher, while reading the Request: closes the connection, silently (crates/runtime/node/src/transport.rs:357-362).
  • Control plane, both ends: the framing error is turned into an InvalidData I/O error (crates/runtime/session/src/plane.rs:1265-1282), which the runtime's loop treats as the connection ending (:717) and the node's loop treats as a lost connection to redial (crates/runtime/session/src/control.rs:913-920).

6.3 A frame that is not the message it should be​

Here the three readers in this repository diverge, and the divergence is deliberate on the control plane and undocumented on the data plane.

ReaderPayload that fails to decodeWhere
Data plane subscriberSkips the frame, keeps the connectioncrates/runtime/node/src/transport.rs:478-480
Data plane publisher (the Request)Closes the connectioncrates/runtime/node/src/transport.rs:358
Control plane runtimeCloses the connection — a node speaking something elsecrates/runtime/session/src/plane.rs:676-682
Control plane nodeIgnores the frame, keeps the connectioncrates/runtime/session/src/control.rs:906

An implementation MUST NOT treat a payload that fails to decode as a framing error: the frame boundary was correct, so the stream is still aligned and the next frame is readable. Whether to skip or to close is per channel, and the four rows above are what the Rust does.

6.4 A well-formed message with nothing in it​

A Frame with no payload variant set, and a ToRuntime or ToNode with no message variant set, are legal protobuf. A receiver MUST NOT treat one as an error.

  • The data plane subscriber skips it (crates/runtime/node/src/transport.rs:497).
  • The control plane runtime ignores it (crates/runtime/session/src/plane.rs:711). The runtime relies on this: when a control message somehow exceeds the frame ceiling, the sender emits an empty frame instead, which the far end decodes as a message that does nothing rather than raising a panic in a session (crates/runtime/session/src/plane.rs:1226-1241).

6.5 Timeouts and silence​

There is no heartbeat on the data plane. What exists:

  • Both ends of a data-plane connection set a 30-second socket timeout (crates/runtime/node/src/transport.rs:77, :187, :218-219).
  • A read timeout at the subscriber is a quiet publisher, not a broken one. The subscriber continues and does not close (crates/runtime/node/src/transport.rs:462-464). A subscriber MUST NOT close a connection because no sample arrived.
  • A publisher whose socket buffer is full MUST NOT treat that as the subscriber being broken. A subscriber that cannot keep up is not a broken subscriber: the Rust waits, resetting its patience on every byte of progress, and gives up only when the peer has accepted nothing at all for the whole 30 seconds (crates/runtime/node/src/transport.rs:379-436). Treating a full buffer as fatal disconnected subscribers silently, and the subscriber had no way to know (:383-393).
  • The publisher never reads its socket after the Request (crates/runtime/node/src/transport.rs:221), so it cannot detect a subscriber that has gone silent without closing. It notices when a write fails.

On the control plane, the runtime declares a node gone after MISSES × heartbeat_period_ms of silence — three periods (crates/runtime/session/src/plane.rs:92, :1170-1191). Being declared gone MUST NOT tear down anything on the data plane; it is a report (crates/runtime/session/src/plane.rs:127-133).

7. Versioning​

7.1 What version means where​

The data plane has no version field. There is no negotiation and no handshake beyond the Request. The version lives in the protobuf package name — ranvier.message.v1 — because that is how a breaking change is made at all: field numbers are permanent within a message, so a genuinely incompatible contract is published as v2 alongside v1, and both can exist in one binary while old recordings stay readable (crates/message/src/lib.rs:17-24).

An implementation therefore MUST NOT expect to be told which version its peer speaks on the data plane. Two peers speaking different major versions of the contract discover it as fields that decode into the wrong shape, or not at all.

The control plane has one version number, checked once. Register.protocol is 1 (crates/runtime/session/src/plane.rs:72), and a mismatch is refused at the moment the node registers rather than misread forty minutes in (:774-784).

7.2 Unknown fields​

A decoder MUST accept a message containing field numbers it does not know and MUST decode the fields it does know. A Sample with an extra field 9 appended decodes with sequence, stream_ref and values intact.

This is what makes additive evolution work, and the drift it already covered is recorded at crates/message/README.md:89-98: a decoder generated from an older copy of stream.proto skipped StreamInfo.production (field 11) without error. What that decoder could not do is tell anybody the field was there.

A decoder MAY discard unknown fields rather than preserving them. The Rust discards them — prost has no unknown-field set — so a program that decodes a message and re-encodes it drops any field it did not know. Nothing in this repository re-encodes a Frame, so nothing is currently lost this way; a relay would be affected, and a relay is required to forward frames without opening them (proto/ranvier/message/v1/transport.proto:51-53), which sidesteps the question. No relay exists — see §9.

7.3 Unknown oneof variants​

A decoder MUST NOT fail on a Frame, ToRuntime or ToNode carrying a variant field number it does not know. It decodes as a message with no variant set, and §6.4 applies. A known variant appearing in the same payload as an unknown one still arrives.

7.4 Unknown enum values​

Every enum in ranvier.control.v1 declares an _UNSPECIFIED = 0, and an unrecognised value reads as unspecified — an unknown value falls to the weakest claim rather than to the nearest one (proto/ranvier/control/v1/control.proto:27-29).

A decoder MUST NOT reject a message carrying an undeclared enum value. Where the Rust converts an enum for its own use it applies the rule above: RefuseReason falls to Unspecified (crates/runtime/session/src/control.rs:959-960), and an unrecognised TransportAction produces no action at all rather than the nearest one (:994-999).

The value itself survives decoding. These fields are open enums carried as integers, so a value of 99 decodes as 99 and is available to an implementation that wants to log it. An implementation MAY surface the raw value; it MUST NOT act on an undeclared one as though it were a declared neighbour.

7.5 Truncated and corrupt payloads​

A payload that ends mid-field is a decode failure, not a framing failure, and §6.3 applies.

An empty payload needs care: it is a valid message with every field at its default, so a receiver that reads it as a sample sees sequence = 0 and stream_ref = 0. Both are the "absent" values this contract reserves, which is why sequence counts from 1 and stream_ref starts at 1.

8. Not yet specified​

Each of these is behaviour the implementation happens to have rather than behaviour anything decided, or a question nothing in this repository answers. An implementation is free here. Closing one of these is a change to this document.

On the wire

  1. Two payload variants in one message. A Frame whose payload carries both an announce and a sample is legal protobuf, and prost keeps whichever came last. A sender MUST NOT emit one. A receiver's behaviour is not specified.
  2. What a receiver does with an unknown Frame variant beyond not failing. The Rust subscriber counts it as an unreadable frame and keeps the connection (crates/runtime/node/src/transport.rs:606-608). Nothing requires that. This entry recorded a divergence — the Rust skipped it silently and the hand-written Python client counted it — and the divergence is gone twice over: 0048 made the Rust count it, and the Python client was deleted on 2026-08-23. It stays unspecified because one implementation agreeing with itself decides nothing.
  3. Whether unknown fields survive a decode-and-re-encode. See §7.2. Open until something re-encodes.
  4. Any bound on SchemaInfo.descriptor. Nothing limits its size below the 16 MiB frame ceiling, and nothing states what a receiver should do with an announcement that approaches it.

Liveness and reconnection

  1. Data-plane liveness. There is no heartbeat, no keepalive frame, and no rule about how long a subscriber may wait before concluding a publisher has gone. The Rust waits indefinitely (crates/runtime/node/src/transport.rs:462-464). The 30-second socket timeout in §6.5 is a socket option, not a protocol deadline: nothing decided its value, which end enforces it, or what the other end observes.
  2. Whether a subscriber may reconnect and what it should expect. The Rust subscriber does not reconnect on its own; the caller opens a new connection, which re-runs the whole handshake and receives every announcement again. Sequence numbers continue from where the publisher's counters are, so a reconnecting subscriber sees its first sample at whatever number the stream has reached — which the loss tracker treats as having missed nothing (crates/runtime/node/src/gap.rs:180-193).
  3. Ordering between streams multiplexed on one connection. Samples from one stream arrive in sequence order because numbering and delivery happen under one lock (crates/runtime/node/src/lib.rs:220-232). Nothing states an ordering between two streams sharing a connection, and TCP's in-order delivery means a stalled retransmission on one delays the others — a cost the contract records and does not resolve (proto/ranvier/message/v1/sample.proto:191-198).

Announcement consistency

  1. A stream that names a source_clock_domain and publishes no source timestamps. The reverse case is enforced (§4.3). This one is described as reported rather than refused (crates/runtime/node/src/announce.rs:287-291), and nothing reports it: SpecError::ClockDomainWithoutTimestamps is declared and given a Display arm in that one file, and constructed nowhere.
grep -rl 'ClockDomainWithoutTimestamps' crates --include=*.rs

Beyond one machine

  1. Transport security and authentication. Both planes are plain TCP. There is no TLS, no authentication on the data plane, and the control plane's token is a session secret passed through the environment rather than a credential (crates/runtime/session/src/plane.rs:642-653). Identity in a StreamInfo is recorded, never verified. This blocks any connection crossing a device boundary — see §3.4.
  2. The cross-device default port and range. Whether the range is per publisher or per host is open too.
  3. Discovery. _ranvier._tcp over mDNS and unicast DNS-SD is settled, but what the SRV record resolves to, what a runtime answers with, and how a subscriber then asks a runtime which streams exist are not.
  4. Transports other than TCP. Nothing else is implemented, and nothing states how this framing would map onto a datagram transport, where a length prefix is redundant.

9. Relay​

A relay is not built. Upstream::Via parses, validates, and is refused at bind time with a message saying so. When one is built:

  • A relay MUST forward frames transparently, without decoding a sample or re-stamping a sequence number.
  • Two properties follow: a relay needs no codec, no schema knowledge and no clock; and a frame it drops under pressure shows up as a sequence gap the subscriber already counts.
  • A relay is a supported fallback, not the default.

The one piece that must be right before a relay is built is the indirect path in BindingInfo — the binding step has to be able to express one, and adding it afterwards is the expensive change, because BindingInfo crosses the control plane every implementation speaks. It has no relay field today.

10. Re-deriving everything here​

# The two constants, and the tests that pin them to literals.
grep -n "pub const PREFIX\|pub const MAX_PAYLOAD" crates/message/src/framing.rs
cargo test -p ranvier-message --lib framing

# The contract itself.
ls proto/ranvier/message/v1 proto/ranvier/control/v1

# The control-plane corpus, and the runner that reads it.
python -c "import json;d=json.load(open('conformance/control/v1/encoding.json',encoding='utf-8'));print(d['framing']);[print(c['name'],c['frame']) for c in d['cases']]"
cargo test -p ranvier-session --test conformance

# Every citation in this document, resolved; every check block above, run.
python docs/docs.tools/check-spec.py

The worked examples in §4.5 were produced by encoding the messages with the committed ranvier-message crate and printing the bytes. To reproduce them, build a small program against crates/message that calls prost::Message::encode_to_vec on the values named there and passes the result to ranvier_message::framing::write_frame.