Posts tagged "LLM evaluation"

A simulation of a real 5 point improvement showing how often a fixed eval threshold gives a random red build, and why a paired comparison detects the gain at a much smaller sample size.

Ship readiness for an AI feature is one artifact per failure class, each with a stated minimum bar and a stated way it can be invalidated. Built here with a runnable 300 item fixture.

A practical pipeline for learning from production AI failures while minimizing personal data, preserving the failure signal, and keeping evaluation fixtures reviewable.