01
HIR, THIR, and MIR: The Same Rust Function at Three Compiler Stages
One small Rust function followed through real HIR, THIR, and MIR output, with a practical explanation of what each compiler representation is for.
Rust Under the Hood
Async, types, memory, and compiler behaviour
Practical explanations of the Rust behaviour that only becomes obvious after reading compiler output, measuring a program, or working close to the language and its tools.
The same Rust writing is also mapped as one system, from compiler internals to async runtimes, in the Rust Systems Atlas.
The model behind the series
Rust gives us strong guarantees, but the most useful mental models often sit below the surface syntax. Futures become state machines. A harmless-looking local changes a future's size. Variance, drop checking, pinning, and pointer provenance shape which abstractions are actually sound.
This series makes those mechanics observable. Each article starts from a concrete question, builds the smallest useful model, and then connects it to decisions we make in real code.
Core principles
01
Inspect what the compiler builds
Use expanded code, type information, layouts, MIR, diagnostics, and small experiments instead of relying on folklore.
02
Connect semantics to design
Language rules matter because they change API boundaries, memory use, concurrency, and the safety of an abstraction.
03
Keep examples reproducible
Prefer focused programs and commands that a reader can run, change, and use to disprove the explanation.
Start here
Entry points into this subject, ordered as a reading path.
02
DefId vs HirId in the Rust Compiler
Two identifiers, two different jobs. DefId names a definition across the whole compiler pipeline and across crates. HirId points to a node in the syntax tree of the current crate. Mixing them up is the first stumbling block when reading rustc source code.
03
How Deno’s Rust Functions Actually Work — Wrapping V8 via rusty_v8
A practitioner’s deep dive into ops, resources, and the V8 bridge in Deno’s Rust core.
04
Behind #[tokio::main]: Peeling Back the Async Runtime
A deep dive into what actually happens when you write #[tokio::main]. We'll tear apart the macro, expose the state machines, and reveal the work-stealing scheduler hiding beneath five lines of code.
Investigations
Multi-part work with the code and measurements that produced its evidence.
10 parts
Types under the hood
Ten parts on what a type is and where it goes: sets of values, layouts and padding, erasure and monomorphization, the bits the processor actually sees, and what the checker knows that the binary forgets. Small programs in Rust, C, TypeScript and Python, with one script that reproduces every output.
3 parts
Rust build times
A measured diagnostic tree for slow Rust builds: cargo --timings for the shape, the fingerprint log for unexpected rebuilds, -Ztime-passes for the phase inside one crate, and what splitting a workspace into more crates actually changes.
3 parts
Async Rust under the hood
Five async Rust problems reproduced in one small tokio program: an oversized future, a future that is not Send, a blocking call, a select! loop that loses data, and a shutdown that hangs. Then the cost of async fn through dyn Trait, and cleanup when Drop cannot await.
Articles
Through the layers
Zero-Copy Does Not Mean Zero Memory Access: Aho-Corasick Streaming in Rust
Zero-copy scanning still reads every inspected byte. It removes a second materialization in memory. I use streaming Aho-Corasick to explain automaton state, chunk boundaries, DFA dependency chains, prefilters, and honest performance measurement.
Through the layers
How Rust Turns a Trait Bound Into Solver Obligations
A source-to-diagnostic walkthrough of trait goals, parameter environments, candidates, associated-type normalization, and nested obligations.
Through the layers
Why a Rust Future Is Not Send Across .await
A Rust future is Send only when every stored state is safe to move across threads. Trace captures and locals across await to fix spawn errors.
Through the layers
Why Rust Async Futures Get So Large
Rust async functions store every value needed after an await. Learn to measure future size, identify saved fields, and reduce it without boxing everything.
Through the layers
Building a Sandboxed Code Playground in Rust
How I built a custom code execution engine backed by a Rust/Axum microservice, embedded directly in blog articles. Covers the security model, subprocess isolation with setrlimit, HMAC request signing, and the tradeoffs behind every decision.
Through the layers
Rust Enum Niches: How Option<T> Can Cost No Extra Byte
Measured Rust enum layouts showing how invalid payload values encode Option discriminants, which optimizations are guaranteed, and what repr changes.
Through the layers
Rust Strict Provenance: Why a Pointer Is More Than an Address
A Rust pointer carries an address and provenance. Learn addr, with_addr, map_addr, exposed provenance, and sound pointer-tagging patterns.
Through the layers
A Rust FFI Boundary Is More Than repr(C)
A tested Rust-and-C FFI boundary covering ABI layout, pointer validation, ownership, panics, error reporting, callbacks, versioning, and cross-language harnesses.
Through the layers
PhantomData: Variance, Auto Traits, and Drop Check in One Type
PhantomData stores no value, yet changes variance, Send, Sync, ownership, and drop checking. Learn to choose a marker from the real contract.
Through the layers
Structural Pinning in Rust: Which Fields Are Actually Pinned?
Pinning a Rust struct does not pin every field automatically. Learn structural pinning, safe projection, Drop obligations, and Future wrappers.
Through the layers
Lending Async Callbacks With Rust's AsyncFn Traits
The usual Fn(&T) -> Fut bound cannot describe a future borrowing from each call. See how AsyncFn enables lending callbacks on stable Rust.
Through the layers
Rust 2024 Temporary Scopes: When Values Are Dropped
Follow Rust drop scopes through if let, match, and tail expressions, including the Edition 2024 changes affecting locks and temporary borrows.
Through the layers
Why Procedural Macro Errors Point at the Wrong Code
A span comparison harness showing why generated Rust errors point at the macro invocation, how to preserve input spans, and how to test diagnostics with compile-fail fixtures.
Through the layers
Rust Macro Hygiene: What $crate, Spans, and Call Sites Actually Control
Rust identifiers carry syntax context as well as text. Learn macro_rules hygiene, $crate paths, and procedural-macro spans for reliable expansion.
Through the layers
How rust-analyzer Finds a Definition Through Macros and Modules
Trace go to definition from a cursor token through syntax, macro expansion, name resolution, semantic identity, and finally back to an editable source range.
Through the layers
Rust's v0 Symbol Mangling: What Changed in Rust 1.97
Rust 1.97 made v0 symbol mangling the stable default. Compare real nm output, read the encoding, and know what changes for profilers, debuggers, and ABI boundaries.
Through the layers
Rust 1.97 Linker Messages: Which Warnings Belong to rustc?
Rust 1.97 forwards successful linker output through a special lint. Learn who produced the message, why -D warnings ignores it, and how to investigate it.
Through the layers
Where a Rust String Literal Actually Lives
When you write let s: &str = "hello world", there are really two things in one line, living in two different places. Here is what each one is, where it lives, and why that explains lifetimes.
Through the layers
Acquire Release Ordering in Rust
Learn how Rust's Acquire and Release memory orderings work, their importance in concurrent programming, and how they enable lock-free synchronization between threads.
Through the layers
How Pattern Shape Changes SIMD Search Performance in Rust
Pattern count alone cannot predict fast string search. I show how rare bytes, short literals, prefixes, alphabet size, and candidate density shape SIMD-assisted Rust performance.
Through the layers
Why More Patterns Can Make Aho–Corasick Slower Even Though It Is Linear
Linear-time search does not promise constant speed. I explain how automaton size, cache behavior, construction, match density, and pattern shape change real Rust throughput.
Through the layers
When Aho–Corasick Prefilters Help—and When They Hurt
A prefilter can skip most automaton work, but it is heuristic rather than free. I show how candidate frequency and pattern shape decide whether it helps a Rust search.
Through the layers
Why Aho–Corasick Streaming Still Needs a Buffer
An automaton can recognize a cross-chunk match with little state, yet a streaming API may still need bytes. The missing distinction is recognition state versus output state.
Through the layers
How to Search Across Chunk Boundaries in Rust Without Missing Matches
A pattern may begin in one input chunk and finish in the next. I explain the state, offsets, and split-point tests needed to keep a Rust streaming scanner correct.
Through the layers
Aho–Corasick DFA or NFA: Which One Should I Use in Rust?
Rust's Aho–Corasick crate can build three different automata. I compare their construction cost, memory layout, search behavior, and the evidence needed before forcing one.
Through the layers
Aho–Corasick Match Kinds in Rust: Standard, Leftmost-First, and Leftmost-Longest
The same patterns and haystack can produce different correct matches. I make Rust's three Aho–Corasick match kinds concrete and show how to choose by product semantics.
Derived state
What Invalidates Rust's Incremental Compilation Cache?
Rust's incremental cache is a dependency graph, not a saved compiler process. Learn what an edit dirties, how green nodes are recovered, and why Cargo may still run rustc.
Derived state
rustc's Query System: Why the Compiler Computes Facts on Demand
A practical model of rustc queries, providers, memoization, dependency tracking, cycles, and red-green incremental reuse.
Derived state
Why Cargo Build Scripts Rerun and How to Make Them Predictable
Cargo build scripts are cached from declared file and environment inputs. Use rerun-if rules, verbose evidence, and a rebuild matrix to remove accidental rebuilds.
Derived state
Cargo Feature Unification Across Workspaces, Host Tools, and Targets
A concrete Cargo dependency graph showing when features unite across workspace packages and when host, target, and platform edges stay separate.
Derived state
Denying Rust Warnings Without Throwing Away the Cargo Cache
Rust 1.97 lets Cargo deny cached lint warnings without rebuilding crates under different RUSTFLAGS. Here is the behaviour and the migration path.
Derived state
Where Rust Monomorphization Happens and Why Codegen Units Matter
A tested walkthrough of monomorphization collection, concrete generic instances, codegen-unit partitioning, compile-time parallelism, and binary-size trade-offs.
Derived state
Cargo Resolver 3: How MSRV Changes Dependency Selection
Cargo resolver 3 prefers dependency releases compatible with your declared rust-version. See where fallback helps, where it cannot, and how workspaces change the result.
Interrupted execution
Cancellation Safety in Async Rust, Explained
Async cancellation usually means dropping a future after Pending. See how select loops lose partial progress and how to design restartable operations.
Shorter pieces and reference
Older, introductory or narrower pieces on the same subject.
- Today I Learned: Understanding the Complexity of dyn Trait in Rust
- Sized vs !Sized: Understanding Sizedness in Rust
- How to Implement a Trait You Don't Own for a Type You Don't Own in Rust
- Building an Async Oneshot Channel in Rust: A Simple Guide
- Exploring Rust's macro_rules Through match_ast! Macro Examples
- Solving the Multi-Trait Object Problem in Rust
- Converting from Generic Unsized Parameter to Trait Object in Rust
- TIL: How to Display the AST in Rust
- Understanding Interior Mutability in Rust: A Simple Guide for Beginners
- Understanding BufReader in Rust
- How to Use `await` Inside `Vec::iter().map()` in an async fn
- Rust Async Libraries
- Understanding Tokio Task Functions
Engineering context
When implementation details shape the product
I work on systems where compiler behaviour, runtime constraints, correctness, and performance are part of the design, not cleanup after it.