- Published on
Investigation · Part 12 of 20 · When the benchmark lies
How Many Benchmark Runs Are Enough
I ran the same sort in 300 fresh processes, in three sets of 100. Each set gave a median with a narrow bootstrap interval, and the intervals did not agree. More runs make the interval smaller, but only if the runs are independent, and on my laptop they were not.