Keep the broken input before you fix the bug

Article · Measurement

The bug disappears after a change. Nobody saved the input that triggered it. A week later, a similar report arrives and the investigation starts again.

Before editing the implementation, keep a small example of the failure. This is especially useful when an agent can produce a plausible fix faster than you can explain why the old code was wrong.

The input gives the discussion something stable. The requirement tells you what its result should be.

Preserve the part that matters

Suppose a contact importer returns two records when the product expects one normalized email identity. A synthetic Rust fixture could begin like this:

fn import_contacts(input: &[&str]) -> Result<Vec<Contact>, &'static str> {
    let mut seen = HashSet::new();
    let mut output = Vec::new();
    for raw in input {
        let email = raw.trim().to_ascii_lowercase();
        if email.is_empty() || !email.contains('@') {
            return Err("invalid fixture identity");
        }
        if seen.insert(email.clone()) {
            output.push(Contact { email });
        }
    }
    Ok(output)
}

This function uses HashSet and the small Contact type defined in the linked fixture. The normalization rule is an example product requirement, not a universal rule for every email system. Confirm the real identity policy before writing the assertion.

Use synthetic values where possible. Do not commit a customer's original document just because it reproduces the bug. Preserve the shape, ordering, encoding, or boundary condition responsible for the failure and remove unrelated private content.

The focused prompting article uses a contact-import example to make one change verifiable. A compact fixture gives both a human and an agent a clear target.

See the old implementation fail

Run the test against the affected revision through the project's normal CI path. Confirm that it fails at the intended assertion. A missing file, a compilation error, or an unreachable test service does not establish the regression.

Then apply the fix and run the same case again. Keep the input and expected result stable unless you discover that the requirement itself was wrong. Explain any change to the expectation in the review.

Rust's testing documentation covers the assertion mechanics. The important engineering work is choosing the observable behaviour that deserves protection.

A narrow fixture is not the whole specification

A fix can pass one input by special-casing it. Add the closest meaningful neighbours: a distinct email that must remain distinct, an already normalized value, and an invalid value with its documented outcome.

Avoid a hundred variations that all repeat the same assertion without increasing confidence. Pick cases that separate the intended rule from a tempting wrong implementation.

When the bug is concurrent, a static input alone is insufficient. Preserve the schedule or crash point too, using barriers or controlled dependencies. When it depends on a provider contract, record the relevant response shape and version without storing secrets.

Put the fixture in the prompt

Ask for a change that makes the failing case pass while preserving the neighbour cases. Include the expected boundary and the checks the agent may use. Review the diff for changes to the test or requirement, not only changes to the implementation.

Keep the failing input and its requirement together. The next refactor can change the implementation while the regression case protects the same behaviour. Without that saved input, “fixed” is often only a memory of a debugging session.

A retained input that catches the old behaviour

The runnable reproduction uses two strings: " [email protected] " and "[email protected]". Its product rule treats outer whitespace and ASCII case as irrelevant to contact identity. That is a fixture rule, not a claim that every mail system treats addresses this way.

The old variant copies both strings into separate contacts. The test passes that result to the same contract used for the corrected importer. It catches the resulting assertion failure. The corrected importer trims, converts ASCII letters to lowercase, and keeps the first occurrence of each resulting identity. The contract now sees one contact with the expected address.

That distinction matters. A test which only demonstrates that two strings differ would tell us little about the importer. Here, the assertion concerns the observable imported records. The deliberately broken variant shows that this assertion can detect the relevant regression.

The surrounding checks keep two distinct addresses as two contacts. A single valid address stays one. A value without an at sign returns the fixture's explicit invalid-input error. This is deliberately a small parser; it is not an email validation library. If the real product supports international addresses or provider-specific identity rules, those requirements need separate cases before this normalization is adopted.

Keep the fixture independent of production customer data. The two synthetic strings preserve the failure mechanism without retaining an address book. A future change to identity policy should update the requirement and explain the changed expectation, rather than delete the awkward input because it blocks a refactor.

The complete Rust test contains both variants and the neighbouring cases. It gives a reviewer the input, the broken result, and the intended boundary in one place.