0003 — Cost that can be paid after recording is paid after recording

• Status: Accepted.

• Implementation status: Built, for the one transformation this record was written alongside. §5.9's alignment guarantee is honoured by the writer when a stream declares it and applied to a finished file by ca3 convert, and the converter is the route the specification points at first.

  Re-derive that both routes exist and produce the same messages:

  cargo test -p ca3 --test aligned_blocks
  cargo test -p ca3-cli --test convert

  Re-derive what the transformation costs on a real recording, which is the measurement this principle exists to keep off the capture path:

  cargo run --release -p ca3-cli -- mock bench.ca3
  cargo run --release -p ca3-cli -- convert bench.ca3 --out dense.ca3 \
    --align 64 --block-offset 3 --stream 3
  cargo run --release -p ca3-cli -- convert bench.ca3 --out everything.ca3 \
    --align 64 --block-offset 3

• Date: 2026-08-23

Context

A container format is asked, repeatedly, to make a downstream consumer's job easier. Align these bytes. Batch these records. Sort this index. Every one of those requests is reasonable, and every one of them arrives as a proposal to change the writer — because the writer is where the bytes are produced and changing it looks like the direct route.

It is the direct route and it is usually the wrong one, because the writer is also the part of the system with the least slack. A recorder is on a capture path: a device is streaming at it, a queue is filling, and the cost of a decision is paid while data is arriving. A consumer's convenience turns into a producer's deadline.

§5.9 is the case that forced the question. Putting a tensor block on a 64-byte boundary means flushing a chunk per message and emitting a padding record between them. Done in the recorder, that is buffering and latency for the benefit of somebody who will read the file next week. Done offline on a finished file, it costs nobody anything and nothing is waiting.

Decision

Cost that can be paid after recording is paid after recording.

The capture path stays simple and low-latency. Transformations that serve a downstream consumer — alignment, padding, re-chunking, index rebuilds — belong to a converter operating on a finished file. A producer that can afford a transformation may still emit it directly, and §5.9's declaration is honoured wherever it is set. No producer is ever required to.

What this rules out

A principle that only permits is not one. This rules out:

• Adding a required transformation to the writer to serve a consumer. If the consumer can get it from a converter, that is where it goes.
• Making a downstream optimisation a conformance requirement for producers. §16's writer conformance is about producing a file that says what it means. It is not a place to put "and aligned".
• Any change that raises the floor for a producer that only wants to record data. The bar for writing a conformant CA3 file does not move because a training pipeline would prefer different bytes.

The question it answers in advance

When a new burden on the writer is proposed, the first question is "can the converter do this instead?" Most of the time it can, and then the proposal is a converter feature and the argument is over in one exchange rather than four.

What this is not

It is not a claim that conversion is free. It is a claim about where the cost lands. A converted file is a second file: it costs storage, a pass over the data, and a provenance chain somebody has to keep. Those are real and they are paid by whoever wanted the transformation, which is the point.

It is not an argument against writer support. Two routes exist deliberately. A producer that already has the buffering — one recording from pre-registered, page-aligned DMA buffers, where the data arrives aligned and staying aligned is free — should emit the aligned file directly and skip the second pass. The principle says the writer must not be required to, not that it must not be able to.

Does this follow from a goal already stated?

Partly from G6, and it adds something G6 does not say.

§0.3's G6 is "Anyone can produce a conformant file without adopting a runtime", tested by "a converter written from this document alone". That is about the difficulty of implementing a writer — no runtime, no library, no dependency. This record is about the obligations of one that exists: what a conformant writer is required to do while data is arriving. A producer can satisfy G6 completely and still be asked to buffer a chunk per message, and nothing in G6 says that request is out of order.

§0.7's G7 — "A recording converts to training data without a human deciding what the numbers mean" — is closer to the outcome and says nothing about who pays for it.

So this is recorded here rather than added as G8. A new goal overlapping two existing ones is how G4 and G5 came to contradict each other twelve lines apart, and the specification has had enough of that. A decision record is subordinate to the specification by design (docs/decisions/README.md), which is the right weight for a principle that governs how future changes are argued rather than what a file contains.

What was rejected

A writer flag alone. ca3 mock --align exists and is the writer route, but making it the only route would have put the transformation on the capture path for every producer that wanted it, and would have made an aligned file something only a writer that knew about alignment at open could ever produce. It also cannot serve the case that matters most for measurement: a converter is the only way to produce two files whose message streams are byte-identical and whose layouts differ, and without that an A/B comparison confounds the change with whatever else differs between two recordings.

A required alignment. Making every file aligned would have removed the declaration, the padding arithmetic, and this record. It was rejected on measurement: aligning every stream of the reference recording to 64 bytes grows it by 260 %, because a stream of 30-byte inertial samples pays a chunk header and a pad per message to align an array nobody asked for. A guarantee that costs that on the streams that do not want it is not a guarantee, it is a tax.

A whole-file alignment switch. Simpler to state and unable to say the thing that matters, which is that one recording holds streams with opposite answers. Per-stream is what lets a dense frame stream declare a page boundary while an event stream beside it declares nothing.

What would change this

• A transformation that genuinely cannot be done offline. Anything needing information only the recorder has — a clock reading at the instant of capture, a device state nothing persisted — is not a converter's job, and if such a case turns out to be common this principle is too broad.
• A converter pass that costs more than the recording. The principle assumes a second pass over a finished file is cheap relative to capturing it. For a multi-terabyte session on media that cannot sustain a re-read, that assumption fails and emitting the wanted form directly becomes correct.
• A producer for which alignment is free. Named above as the case that already uses the writer route. If it became the ordinary case rather than a special one, the ordering in this record would be worth revisiting — though the principle itself, that the floor does not move, would still stand.