0004 — Converters are Rust, behind one C ABI

• Status: Accepted, 2026-09-02. The four questions it was proposed with are settled and recorded under What was open; one dependency choice still needs confirmation and is marked.

• Implementation status: Nothing built. No crates/export, no crates/ffi, no ca3 export verb.

  ls crates/                                     # ca3  ca3-cli
  cargo run -q -p ca3-cli 2>&1 | grep -c export  # 0

• Date: 2026-09-02

• Depends on: 0003 — a converter operates on a finished file, never on the capture path.

Context

Nothing in this repository writes anything but .ca3. Every export to another format is done by an application, there were two of them when this was written, and they were a fork of each other. Measured 2026-09-02, over the two TypeScript files each of them carried: 101 differing lines in the CSV writer and 146 in the exporter around it. Neither file is in this repository, so the diff that produced those two numbers cannot be re-run from this checkout.

The copy said so in its own header — "PORTED from … on 2026-08-13 … Re-sync rather than edit" — and had already diverged: it carried a helper giving an empty stream list the meaning "every stream", and the original did not. The channel reader table existed three times across the two applications.

One mistake made twice by the arrangement that made it available, and the arrangement rather than the authors is what produces it: two programs that each need to read numbers out of a recording, in a language with no shared library between them, will each write one. The divergence was small when this was written because the code was three weeks old, which is the argument for deciding now rather than later.

Both applications reach a recording the same way: spawn ca3 play, read newline-delimited JSON, decode each payload from base64. bindings/python does the same. That path has a measured ceiling, recorded in bindings/python/src/ca3/backend.py against the sixty-second ca3 mock recording: 75,018 messages walk in 0.64 s, about 117,000 messages a second, and decoding them costs seventy times less than moving them. The transport is the cost.

Decision

Converters are written once, in Rust, in this workspace. Every other language reaches them through one C ABI, and until that ABI exists, through the ca3 binary.

1. Where they live, and the rule they do not break

crates/container stays at zero dependencies and never parses a descriptor. A converter that names a column must parse a descriptor, so it cannot go there. It goes in a sibling crate, crates/export — package ca3-export, on the rule every directory under crates/ follows — held to the rule ca3-cli already lives under: a crates.io dependency is allowed; a dependency on another repository in this ecosystem is not.

cargo tree -p ca3          # one line, before and after

That is enough, because everything a converter needs is in the recording. A Schema record carries the transitive closure of its imports, and a schema annotated with (ca3.options.v1.unit) imports the grammar that declares it — so options.proto itself is inside every descriptor a converter opens. The converter resolves the annotations by extension number, from the file, the way bindings/python/src/ca3/_grammar.py already does, and consults nothing compiled in:

grep -oE "[a-z_]+ = 500[0-9]+" proto/ca3/options/v1/options.proto
# unit = 50001, frame = 50002, validity_for = 50003, cardinality = 50004, tensor_form = 50010

The copy vendored here is what ca3 mock's example schemas compile against. It is not consulted at export time, and a converter that reached for it instead of the file would be one that could not read a recording annotated by a grammar it had never seen.

This is a rule, not a convenience: a conversion needs the .ca3 file and nothing beside it. No sidecar, no registry, no vocabulary, no grammar file. Anything a conversion turns out to need that the file does not carry is a gap in the file, closed by writing it into the file — §13.3's Metadata record is where a fact with no other home goes — and never by a reference the file has to be accompanied by.

What a converter reads out of a schema, and what it declines to, is 0005.

2. The sequence: the binary first, the ABI on a measured trigger

ca3 export is the first consumer of crates/export. It serves Python, JavaScript, R, MATLAB and a shell the day it lands, because every one of them can spawn a process, and bindings/python already does.

The C ABI — crates/ffi, package ca3-ffi, building a cdylib and a staticlib whose library name is ca3 — is built when either of these is true, and not before:

• a consumer needs the container in-process: a browser, a .NET host, or a Python caller that cannot stage a binary beside itself; or
• the subprocess ceiling is reached. backend.py's figure is the reference: 0.64 s per sixty-second recording is ample for one session and is eleven minutes for a thousand, and the second is where an ABI earns its cost.

Nothing written for the binary is thrown away at that point. bindings/python's RecordingBackend is four methods — describe, messages, manifest, close — and its header says a native reader "implements the same four methods and changes no user code." The converter crate is the same code under both.

3. The ABI conventions, stated once

When the ABI is built it takes these: opaque handles; the library allocates and frees; int32_t return codes with values through out-parameters; variable-length data copied into a caller's buffer, a null destination measuring; named int32_t constants and never a C enum; one handle from one thread; an ABI version compared for equality. None of it is novel — it is the set a C ABI under several language bindings converges on, and re-deriving it per repository is how two of them end up incompatible.

Two additions, because a converter's product is a file rather than a sample:

• A converter takes a destination path and reports progress through a polled counter. No callback crosses the boundary. A callback into a .NET finalizer thread is the class of fault a "null-tolerant, non-blocking, callable from any thread" rule exists to prevent, and nothing here needs one: a caller that wants progress asks for it.
• Options cross as string pairs, ca3_export_set(handle, key, value) — the shape Stream.metadata and Session.custom already have. No document format crosses the ABI, so the ABI crate parses nothing, and an option a binding has never heard of still reaches the converter.

4. The verb is export, because convert is taken

ca3 convert applies a format feature to a finished recording and writes another .ca3 — 0003's transformation. Leaving the format is a different act and gets a different word, so that the two never share a Plan, a flag list, or a sentence of documentation. The crate follows the verb: ca3-export.

Rejected: overloading convert with --to csv. It would put "produce a recording with alignment applied" and "produce a spreadsheet" behind one verb whose documentation is about the first. Rejected: one verb per format — ca3 csv, ca3 parquet. Five verbs by the time there are five formats, and a binding function per verb behind an ABI that cannot change.

5. The crate is a registry of one, because the ABI freezes and the crate does not

ca3-export defines an Exporter trait and ships one implementation, CSV. The trait is not there for Parquet, MCAP, EDF or HDF5, though each is visible from here; it is there because §3's ABI is the part that cannot change once a second binding exists, and an ABI shaped around one format's arguments is what the second format has to break. ca3_export_create(format, …) takes a name from the first commit. Internally, one implementation is fine.

Rejected: a CSV crate now, generalised at the second format. Cheaper today, and it designs the frozen surface around the wrong axis.

6. WebAssembly stays a claim until a job checks it

CONTEXT.md argues a WebAssembly build is "unusually viable here because crates/container has no dependencies and no platform calls outside recorder.rs." That argument is about crates/container and says nothing about a crate with a reflection dependency beside it. It stays true or it does not, and a sentence cannot tell which, so:

cargo check -p ca3 -p ca3-export --target wasm32-unknown-unknown

is a CI step from the commit that creates the crate. A failure is a finding, not a blocker — the binary and the ABI do not need it — but a browser reader is the one consumer the subprocess path can never serve, and it should not be discovered to have been lost a year after it was.

What this rules out

• A converter in TypeScript, Python, or any language but Rust, shared or not. An application's own CSV writer is deleted once ca3 export covers what it does, with its tests moved rather than dropped.
• A converter that links a vocabulary, a grammar, or a transport. The vocabulary question is 0005's, and its answer has to hold without any of them.
• A conversion that needs a file beside the recording. A sidecar of descriptions, a stated element type kept in a settings file, a grammar looked up outside the descriptor — each is a reference the recording has to travel with, and a recording outlives whatever it travelled with.
• A callback across the ABI, for progress or for anything else.
• A second ABI, in this or any other repository, for the same converters.

What was open, and how it was settled

	Question	Settled as	Alternatives, and why not
A	What a converter reads out of a schema	0005, with the condition that every field kind — audio blocks included — has a representation	a reader hook across the ABI; readers beside the vocabulary; the TypeScript export. Each argued in 0005
B	The verb and crate name	ca3 export, crates/export — §4	convert --to; one verb per format. §4
C	The WebAssembly claim	a CI cargo check step; failure is a finding — §6	a note only, which the repository's own discipline forbids; building the binding now, which pays the ABI cost for a consumer nobody has asked for
D	Registry or single crate	an Exporter trait with one implementation — §5	a CSV crate first, which shapes the frozen surface around one format; per-format crates, which grow the ABI per format

7. The reflection dependency is prost-reflect

ca3-export decodes payloads for schemas it has never seen, so it needs a dynamic protobuf decoder. It takes prost-reflect, in ca3-export only:

cargo tree -p ca3                                # still one line
cargo tree -p ca3-export | grep -c prost-reflect # 1, once the crate exists

Rejected: a hand-rolled wire walker. The repository has precedent — CRC-32C and LEB128 are hand-rolled — but both are pinned to published vectors, and a walker covering varints, both fixed widths, length-delimited fields, packed repeated fields, nested messages and unknown-field skipping has no such vectors to pin to and would be larger than the two together. The dependency is pure Rust, so §6's WebAssembly check is unaffected by it in principle and is what will say so in practice.

Settled 2026-09-02, on confirmation.

What would change this

• A converter that cannot be written without a vocabulary — one where 0005's generic tier produces a file nobody can use — reopens §1, and the answer is not "link the vocabulary" but "the descriptor is missing a declaration; add it to the grammar."
• A second implementation of any converter appearing anywhere is this record having failed, and the question is why the ABI was not enough.
• The ABI shipping and bindings/python's Ca3Cli surviving beside it with no stated reason is §2 having been half-done.

Measurements this record rests on

Claim	Source	Re-derive
Two forks, 101 and 146 lines apart, 2026-09-02	this record	not re-derivable here: neither file is in this repository
0.64 s, ~117,000 messages/s, transport-bound	bindings/python/src/ca3/backend.py header	the README under bindings/python carries the benchmark
Five extension numbers	proto/ca3/options/v1/options.proto	the grep above
crates/container has zero dependencies	Cargo.toml	cargo tree -p ca3