Posts tagged "ai evaluation"

A practical AI failure taxonomy that separates what happened, where it happened, its impact, and the suspected cause so teams can build useful metrics.

A purely random sample hides rare and expensive AI failures. Combine baseline, risk-triggered, and exploratory samples without corrupting the denominator.

A versioned evaluation matrix for separating prompt, model, retrieval, tool, and grader changes while protecting safety, quality, latency, and cost.

A practical agreement experiment for measuring an LLM judge against human labels, including ambiguous cases, position bias, confidence, and release thresholds.