Mehdi Akiki
Writing / Measurement

Measurement-driven systems work

Treating an assumption as a hypothesis, building an experiment, following a surprising result into a lower layer, and rejecting a change when the evidence says it loses.

Investigations

3 parts

Rust build times

A measured diagnostic tree for slow Rust builds: cargo --timings for the shape, the fingerprint log for unexpected rebuilds, -Ztime-passes for the phase inside one crate, and what splitting a workspace into more crates actually changes.

Evidence: experiments/rust-atlas/build-times/

20 parts

When the benchmark lies

Twenty ways a performance number goes wrong, each reproduced on one laptop: a loop the compiler deleted, debug builds, warm-up and the JIT, CPU frequency and efficiency cores, averages and coordinated omission, clocks, run counts, blocks against interleaved runs, environment size, code alignment, caches, branch prediction, the allocator, and a microbenchmark that lost inside the real program. One script reproduces every output.

Evidence: experiments/when-the-benchmark-lies/

Articles

Browse the full archive →