Posts tagged "testing"

Schema validation catches only part of provider drift. Test the fields, behaviours, and semantic mappings your consumer actually depends on, then verify them safely.

A useful fake provider controls time, pagination, throttling, malformed data, and ambiguous writes. Test the integration protocol, not only JSON decoding.

A working record-and-replay harness for agent tests: a JSON cassette of decisions and tool results, deterministic replay with zero live calls, and a readable diff when the trajectory changes.

A pattern may begin in one input chunk and finish in the next. I explain the state, offsets, and split-point tests needed to keep a Rust streaming scanner correct.

A practical pipeline for learning from production AI failures while minimizing personal data, preserving the failure signal, and keeping evaluation fixtures reviewable.