0002 — The vocabulary is one set, and one function decides it

• Status: Accepted.

• Implementation status: Built. descriptor::vocabulary is the single predicate; declared_messages returns it and for_message gates on it, so the list a device package checks an announcement against and the registry conformance::check compares a descriptor with are the same set by construction. spec/container.md §15.3 indexes exactly that set.

  Re-derive the vocabulary's size, from the schemas and from the crate:

  grep -rhoE "^message [A-Za-z0-9_]+" proto/ca3/canonical/v1/*.proto | wc -l
  # 21
  cargo test -p ca3-canonical --lib the_vocabulary_is_the_canonical -- --nocapture
  # declared_messages() = 21 names

  Re-derive the split §15.3 indexes:

  grep -rhoE "ca3\.options\.v1\.tensor_form\) = \{" proto/ca3/canonical/v1/*.proto | wc -l
  # 5
  cargo test -p ca3-canonical --lib every_canonical_schema_either_declares

  The assertions that fail if either half regresses:

  cargo test -p ca3-canonical --lib the_annotation_grammar_is_not_something
  cargo test -p ca3-canonical --lib nothing_protobuf_declares_is_announceable
  cargo test -p ca3-canonical --lib a_synthesised_map_entry_is_not_a_schema

• Date: 2026-08-11

Context

Two questions in this repository were being answered separately, and both are the question what is the canonical vocabulary.

descriptor::declared_messages answered which names a producer may announce. Two device packages read it, each asserting that the identifier it announces appears in the list.

spec/container.md §15.3 answered which schemas must account for having no tensor form, under §0.7's rule that a gap is named rather than discovered.

They gave different answers. declared_messages returned 27 names; §15.3's table covered 12 or 17 depending on whether a reader counted rows or messages. The true vocabulary is 21.

What was wrong, in three parts

The filter did not do what its own comment said. declared_messages kept any package beginning ca3., and its comment justified that filter by saying a list carrying protobuf's internals "would answer yes to names no producer should ever announce." The list it produced carried the five ca3.options.v1 messages — TensorForm, TensorAxis, TensorComponent, FrameDeclaration, FrameCatalogue — which are the annotation grammar rather than the measurement vocabulary, and ca3.canonical.v1.Event.AttributesEntry, which protoc synthesised for a map field. Six names the comment ruled out, in the list the comment was attached to.

for_message had no filter at all, and that is what could bite. wellformed::registered_schema documents for_message as the registry §5.7 asks a validator to hold, and conformance::check reads the registry by calling it. Measured before the change, by compiling the calls and running them:

Announced schema_id	for_message	conformance::check
ca3.canonical.v1.GazeSample	72,756 bytes	Registered
ca3.options.v1.TensorForm	45,295 bytes	Registered
ca3.options.v1.FrameCatalogue	45,295 bytes	Registered
ca3.canonical.v1.Event.AttributesEntry	5,695 bytes	Registered
google.protobuf.FileDescriptorSet	13,180 bytes	Registered
google.protobuf.SourceCodeInfo	13,180 bytes	Registered
ca3.canonical.v1.NotAThing	None	Unregistered

So a producer could announce a stream of TensorForm, or of protobuf's own FileDescriptorSet, and §5.7 would call it canonical — the strongest claim the format makes about comparability, granted to a name that measures nothing. The filter that was supposed to prevent this was applied in the one place a validator does not read.

The test that guarded §15.3 was a second hand-written list. the_schemas_that_declare_no_tensor_form_are_the_ones_that_argue_they_should_not named eleven schemas; every_canonical_tensor_form_clears_the_rules_that_check_it named five. Sixteen of twenty-one, and neither list was compared against anything. Vector1, Vector2, Projection2D, EegChannel and PpgChannel could have gained or lost a form in silence.

The two lists had also drifted from §15.3 in opposite directions: the table named Vector1 and Vector2 and omitted EyeSample; the test did the reverse. Both were green.

The decision

One function decides membership. descriptor::vocabulary keeps a message when its package begins ca3.canonical. and protoc did not synthesise it — map_entry in its options is what says so. declared_messages returns that list and for_message refuses anything outside it, so registered_schema, conformance_of_canonical and conformance::check are all gated by one line.

§15.3 indexes that same set, all twenty-one. Its counts are of schemas and the table has one row per argument, which is now said out loud beside them.

The question that decides the scope, and the answer

Can a Schema record's schema_id legitimately name a component type such as EyeSample? If not, §15.3 would be payload-scoped and would owe rows to perhaps eight messages rather than twenty-one.

Yes. Three things say so, and they are independent:

• §5.1 places no restriction. schema_id is "the protobuf fully-qualified message name" and indexes descriptor directly. Nothing in §5.1 or §5.7 distinguishes a message used as a payload from one used as a field.
• The table was already vocabulary-scoped. Its Vector1–VectorN, Quaternion row indexes five shape types that are equally never announced alone. A payload-scoped reading would have to delete that row, which is the row nobody has ever suggested deleting.
• The implementation already permits it, as the table above shows: a recording announcing EyeSample with its own descriptor gets Registered, before this change and after it.

A narrower §15.3 would also have needed a second, narrower notion of vocabulary inside a crate that is here to stop having two.

What was rejected

Gating conformance::check instead of for_message. It closes the same hole and leaves registered_schema — documented as the registry — answering a different question from the checker that reads it. Two answers to "is this the registered schema" is the defect, not the symptom.

Keeping starts_with("ca3.") and excluding ca3.options.v1 by name. A listed exclusion is correct until somebody adds a sixth annotation message. The package boundary is the fact; a list of exceptions is a copy of it that rots.

Matching ca3.canonical.v1. exactly. A second version of the vocabulary would then be excluded until somebody noticed this line. ca3.canonical. puts it in by construction.

Excluding map entries by name suffix. AttributesEntry ends in Entry, and so would a message somebody wrote by hand called CalibrationEntry. The map_entry option is what protoc actually sets, and reading it means a real message with an unlucky name stays in the vocabulary.

Padding §15.3 to twenty-one rows. Four of the missing messages share arguments with schemas already in the table; giving each its own row would have produced four restatements of two arguments and made the table harder to check rather than easier. Two rows were added instead, one per argument.

Writing Projection2D's argument into gaze.proto in this change. It belongs there — decision 0001 is the reason a schema's prose travels inside the recording — but the edit changes descriptor.bin, which changes for_message's output, which requires regenerating the three §16 conformance fixtures whose whole purpose is to be an independent pin on that output. Doing that in the same change as the for_message gate would bury a four-line behavioural change under three rewritten binaries. §15.3 names the gap; the move is its own change.

What this costs, and who moves

The Python reader moves. Its noFormAdvice equivalent returns 12 entries, which is §15.3's table read as one row per schema. It must return 16.

The JavaScript reader does not. It already returns 16, having read the same sentence the other way. Both readings were defensible, which is the evidence that the sentence needed fixing rather than that either SDK was careless.

Both SDKs gain a third deferral. Projection2D joins AudioFrame and PointerSample as deferred rather than refused, because §15.3's own definition of the word applies to it: a deferral is one that names a change making the answer yes, and a form able to carry more than one element type is that change.

No caller of declared_messages breaks. Both external readers use it as a membership test — declared.contains(id) for an identifier they announce — and every identifier they announce is in ca3.canonical.v1. Narrowing a list that is only ever searched cannot fail a search that was succeeding. Every external caller of for_message passes a ca3.canonical.v1.* name too, checked by reading the seven repositories that call either.

What would change this

A canonical schema that is genuinely not announceable. If the vocabulary ever gains a message that a Schema record must not name, §5.1 needs a sentence saying so and this predicate needs a second clause. Nothing today is in that state, and the argument above is that §5.1 as written puts nothing in it.

A record for the annotation grammar to ride in. §5.5 notes that version 1 has nowhere for a file to carry its own FrameCatalogue. If such a record arrives, ca3.options.v1.FrameCatalogue becomes a thing a recording carries by name — still not as a stream's schema_id, but the boundary would be worth restating rather than assuming.