Measurement
This site as an engineering laboratory
Kept, rejected and upstream results from more than fifty experiments on this site.
Evidence: experiments/
Writing
The main body of this writing follows one system below its interface: a compiler, a runtime, a protocol, a build, a data pipeline. It is organized around four engineering questions.
Investigations run over several parts and keep the programs and experiments that produced their evidence. Articles stand alone. Shorter notes, beginner material and tool guides stay in the archive and in search.
Investigations
Multi-part work backed by code, measurements or reproductions that can be run again.
Through the layers · 10 parts
Types under the hood
Ten parts on what a type is and where it goes: sets of values, layouts and padding, erasure and monomorphization, the bits the processor actually sees, and what the checker knows that the binary forgets. Small programs in Rust, C, TypeScript and Python, with one script that reproduces every output.
Evidence:
experiments/rust-atlas/types-under-the-hood/- What Is a Type? A Set of Values, a Promise About Operations, or Both
- Does a Type Exist at Runtime? Following One Value From Source to Register
- A Type Is a Set: Why Human Has Two Members and Option of Human Has Three
- Shape Without Behavior: Two Structs With the Same Fields
- Behavior Without Shape: Traits, Interfaces, and Types That Own No Bytes
- Where a Type Becomes a Layout: Fields, Padding, and Reordering
- Two Ways to Make a Type Disappear: Erasure and Monomorphization
- The Processor Has No Types: What add Does to Bits It Does Not Understand
- Runtime Type Tags: What Dynamic Languages Keep That Static Ones Throw Away
- Types as Proofs: What the Checker Knows That the Binary Forgets
Derived state · 3 parts
Rust build times
A measured diagnostic tree for slow Rust builds: cargo --timings for the shape, the fingerprint log for unexpected rebuilds, -Ztime-passes for the phase inside one crate, and what splitting a workspace into more crates actually changes.
Evidence:
experiments/rust-atlas/build-times/Interrupted execution · 3 parts
Async Rust under the hood
Five async Rust problems reproduced in one small tokio program: an oversized future, a future that is not Send, a blocking call, a select! loop that loses data, and a shutdown that hangs. Then the cost of async fn through dyn Trait, and cleanup when Drop cannot await.
Evidence:
experiments/rust-atlas/async-symptoms/Through the layers
Rust Failure Atlas
Rust failures that are difficult to name and easy to misdiagnose, organized from symptom to mechanism, with a failing and a repaired fixture for most cases.
Articles
Standalone pieces, grouped by the question they answer.
Through the layers
Following a system through its layers
Source, runtime, protocol, compiler, machine. Each piece follows one behaviour down to the layer that decides it.
Investigations: Types under the hood, Async Rust under the hood, Rust Failure Atlas
HIR, THIR, and MIR: The Same Rust Function at Three Compiler Stages
One small Rust function followed through real HIR, THIR, and MIR output, with a practical explanation of what each compiler representation is for.
WebSockets From the Wire Up
Most WebSocket tutorials start at the API. This one starts at the wire — electrons, NIC interrupts, kernel socket buffers, the TCP state machine, the HTTP upgrade handshake, the binary frame format — and works up to what ws.send() actually does.
How Deno’s Rust Functions Actually Work — Wrapping V8 via rusty_v8
A practitioner’s deep dive into ops, resources, and the V8 bridge in Deno’s Rust core.
Zero-Copy Does Not Mean Zero Memory Access: Aho-Corasick Streaming in Rust
Zero-copy scanning still reads every inspected byte. It removes a second materialization in memory. I use streaming Aho-Corasick to explain automaton state, chunk boundaries, DFA dependency chains, prefilters, and honest performance measurement.
DefId vs HirId in the Rust Compiler
Two identifiers, two different jobs. DefId names a definition across the whole compiler pipeline and across crates. HirId points to a node in the syntax tree of the current crate. Mixing them up is the first stumbling block when reading rustc source code.
Derived state
Derived state, build artifacts and invalidation
What gets stored, what gets recomputed, what becomes stale, and how the system knows.
Investigations: Rust build times
What Invalidates Rust's Incremental Compilation Cache?
Rust's incremental cache is a dependency graph, not a saved compiler process. Learn what an edit dirties, how green nodes are recovered, and why Cargo may still run rustc.
rustc's Query System: Why the Compiler Computes Facts on Demand
A practical model of rustc queries, providers, memoization, dependency tracking, cycles, and red-green incremental reuse.
How to Keep Two Systems in Sync
When you sync two systems with webhooks, the copy slowly becomes wrong: some events are lost, some are duplicated, some arrive in the wrong order. Reconciliation is the way to make the two systems agree again. Here is how to design an API for it.
Why Cargo Build Scripts Rerun and How to Make Them Predictable
Cargo build scripts are cached from declared file and environment inputs. Use rerun-if rules, verbose evidence, and a rebuild matrix to remove accidental rebuilds.
Why updated_at Is a Dangerous Synchronization Cursor
Concrete simulations of the records a timestamp cursor can miss, plus safer tuple cursors, overlap windows, change logs, and reconciliation.
Interrupted execution
Correctness when execution is interrupted
Crashes, retries, cancellation, partial progress, ownership, recovery and idempotency.
Investigations: Async Rust under the hood
When to Advance a Sync Checkpoint After Partial Success
A checkpoint is a claim about durable effects, not the last page fetched. Compare commit protocols and crash points so recovery repeats work instead of losing it.
Resuming a Paginated Import After Page 417 Fails
A paginated import should resume from a committed cursor and stable item identities, not from a human page number that may no longer describe the same data.
Cancellation Safety in Async Rust, Explained
Async cancellation usually means dropping a future after Pending. See how select loops lose partial progress and how to design restartable operations.
Preventing an OAuth Token Refresh Stampede
A concurrent refresh design using local single-flight work, database versions, conditional writes, and recovery rules for providers that rotate refresh tokens.
Make Retried Agent Actions Idempotent Before Adding Autonomy
Agent retries must reuse one logical action identity and reconcile uncertain effects. An effect ledger makes autonomy bounded, inspectable, and recoverable.
Measurement
Measurement-driven systems work
Treating an assumption as a hypothesis, building an experiment, following a surprising result into a lower layer, and rejecting a change when the evidence says it loses.
Investigations: This site as an engineering laboratory, Rust build times
What Evidence an AI Feature Needs Before You Ship It
Ship readiness for an AI feature is one artifact per failure class, each with a stated minimum bar and a stated way it can be invalidated. Built here with a runnable 300 item fixture.
Eval Scores Move Between Runs: How to Compare Two Versions Honestly
A simulation of a real 5 point improvement showing how often a fixed eval threshold gives a random red build, and why a paired comparison detects the gain at a much smaller sample size.
A Record-and-Replay Harness for Agent Tests
A working record-and-replay harness for agent tests: a JSON cassette of decisions and tool results, deterministic replay with zero live calls, and a readable diff when the trajectory changes.
The Final Answer Can Be Right While the Tool Trajectory Is Unsafe
A trajectory grader for detecting unauthorized reads, duplicate effects, fabricated tool results, waste, and unsafe recovery even when an AI agent's final answer looks correct.
Calibrating an LLM Judge Against Human Disagreement
A practical agreement experiment for measuring an LLM judge against human labels, including ambiguous cases, position bias, confidence, and release thresholds.
The archive
All 267 published pieces, newest first, each labelled as an investigation, an article or reference. Reference covers beginner Rust, framework and tool guides, AI explainers, interview preparation and older short pieces. The 61 short engineering notes have their own page.
Other ways in
- Rust Systems Atlas: the Rust writing as one map, from compiler internals to async runtimes.
- Rust Under the Hood: Async, types, memory, and compiler behaviour.
- Reliable Data Integrations: Synchronization, events, schemas, and recovery.
- Reliable AI Systems in Production: Evals, retrieval, tools, and observability.