- Published on
Why Semantic Versioning Does Not Describe Data Compatibility
- Authors

- Name
- Mehdi Akiki
Reference
Semantic Versioning can tell me that a maintainer considers a release backward compatible. It cannot prove that an old consumer will understand a new payload or that historical data still means the same thing.
The SemVer specification starts with an important requirement: declare a public API. Data systems usually have several public contracts hiding behind one version number.
endpoint and authentication contract
wire encoding
schema and validation rules
field presence and defaults
business meaning
ordering and delivery behaviour
stored historical records
reader and writer implementations
I keep the version label, but I do not use it as the compatibility proof.
Compatibility has a direction
Suppose V1 knows this event:
{ "order_id": "o-17", "state": "paid" }
V2 adds paid_at:
{
"order_id": "o-17",
"state": "paid",
"paid_at": "2026-11-13T08:12:00Z"
}
There are at least two questions:
| Direction | Question |
|---|---|
| backward reading | Can the V2 reader consume data written by V1? |
| forward reading | Can the V1 reader consume data written by V2? |
Then there are two writers, persisted V1 data, replay, and round-trip preservation. “Backward compatible” without a subject and direction is incomplete.
My compatibility matrix
For each change, I run this matrix:
| Writer | Reader | What this proves |
|---|---|---|
| old | old | baseline still behaves as recorded |
| old | new | new reader accepts historical and lagging producers |
| new | old | lagging consumers survive the rollout |
| new | new | intended new behaviour works |
For relays and read-modify-write services, I add:
new writer -> old intermediary -> new reader
That path catches unknown fields that parsers preserve but application mappers delete. Unknown Fields Are the Quiet Engine of Protobuf Evolution follows this case in detail.
A SemVer minor change can still break data
Adding an optional field sounds compatible. It breaks when:
- an old JSON decoder rejects unknown properties;
- a signature or checksum covers the exact serialized bytes;
- a database writer uses positional columns;
- a consumer interprets absence differently from
null; - a payload-size limit is crossed;
- the new value changes routing, privacy, or authorization;
- an intermediary reconstructs only fields it knows.
These are not failures of SemVer arithmetic. They mean the declared public API was smaller than the real operational contract.
The dedicated article An Added Field Can Still Be a Breaking Change examines this deceptive case.
Wire compatibility is only one layer
Protocol Buffers publishes concrete message-update rules. For example, adding fields with new numbers is generally wire safe, while reusing a removed field number is dangerous.
Even a wire-safe change can be semantically unsafe:
before: timeout = 30 meaning seconds
after: timeout = 30 meaning milliseconds
Every parser still succeeds. Every number fits the schema. The behaviour is wrong.
I classify compatibility at several layers:
| Layer | Example failure |
|---|---|
| syntax | bytes or JSON cannot be parsed |
| structural schema | required field or enum validation fails |
| representation | number overflows one generated language type |
| semantic | unit or meaning changes under the same field |
| behavioural | new state changes a consumer's side effect |
| operational | payload, latency, or throughput exceeds a limit |
| governance | privacy or retention class changes |
Passing the first two layers does not imply the later ones.
Stored data is another consumer
A database, object store, or event log may outlive every deployed binary. I treat historical data as a permanent old writer.
When the new reader starts, it may encounter:
records from several schema versions
partially migrated rows
values accepted by a retired validator
old defaults that were never materialized
events replayed in a different order
If a new release can read only data produced after deployment, it is not compatible with the actual system.
My fixtures include real redacted shapes from every supported era or invented equivalents that preserve the edge cases. Private customer payloads never enter a public article or repository.
Defaults can rewrite history
Suppose risk_level was absent in V1. V2 defines its default as low.
Reading every old record as explicitly low makes a new claim about history. Perhaps the truth is “not assessed.” The reader remains technically functional while an audit or model feature becomes false.
I distinguish:
absent because the concept did not exist
absent because producer omitted an optional observation
explicit null because it was checked and unknown
explicit default because the producer chose it
The next article, Null, Missing, Empty, and Default Are Four Different Data States, develops this state model.
Version envelopes help only when used
I often put explicit identity around a payload:
{
"message_type": "order.status.changed",
"schema_version": 2,
"event_id": "evt-203",
"occurred_at": "2026-11-13T08:12:00Z",
"payload": { "order_id": "o-17", "state": "paid" }
}
This supports dispatch, monitoring, and deliberate migrations. It does not automatically make version 2 safe. The producer and consumer still need a compatibility policy.
I avoid one global “API version” when independently deployed event families evolve separately. A version must identify the contract it versions.
Deployment order is part of the proof
For an additive change, a safe rollout may be:
1. deploy tolerant readers that ignore or retain the new field
2. confirm all active reader versions are compatible
3. enable new writers behind a flag
4. observe rejects, unknown values, and semantic invariants
5. backfill only if old absence has a truthful interpretation
6. remove old support after retention and replay windows close
For a breaking meaning change, I use a new field or message version and dual-read/dual-write only where reconciliation is possible. I do not silently reinterpret stored values in place.
What I attach to a change request
Instead of “minor, therefore safe,” I want:
- the old and new schema or contract diff;
- the compatibility direction promised;
- the oldest reader and writer still active;
- four reader/writer fixture results;
- a round-trip test for intermediaries;
- historical-data and replay results;
- semantic invariants before and after;
- rollout order and rollback limits;
- observability for rejects and unknown states;
- retention date after which old support can end.
The package or service can still receive a SemVer number. That number communicates release intent. The matrix supplies evidence.
Data compatibility is a relationship among a specific writer, reader, contract, and meaning at a point in time. Once I name all four, a version becomes useful context instead of false certainty.