How to Diagnose Slow CI for AI Coding Agents

Article · Measurement

An AI agent can finish a patch while CI is still installing dependencies. Before adding runners or splitting jobs, find where the wait actually happens. Queueing, setup, a focused test, and the full required build are different parts of that wait. A faster test will not repair all of them.

Suppose Maya has a small fix ready. She pushes it, opens the checks, and sees nothing useful yet. While waiting she asks for another cleanup. Then another. By the time a failure arrives, she has to remember which version it belongs to.

That story is illustrative. The timing example below is a real run from this site. It gives us somewhere firmer to start than saying CI feels slow.

Read one run before changing the workflow

The October 24 verification run checked a repair to the site's deployment test. Its job records expose these UTC times, rounded by GitHub to whole seconds:

The rows are selected observations, not an exhaustive accounting whose durations should be added together. Checkout, tool setup, formatting, evidence capture, and cleanup occupy other parts of the job. A step whose displayed start and end are identical has not been proved to take no time.

The initial two seconds include GitHub's dispatch and scheduling work. They are not a direct measurement of time spent waiting for an available runner. Likewise, the 103-second command includes the repository's content checks and production build. Calling it “103 seconds of Next.js compilation” would claim more detail than the record provides.

The captured fields and source API endpoints are in the example directory. The run API reported revision 90899778692d5ada7421ad26e7384474fe41a43d. That field alone is not a substitute for recording the revision actually checked out inside a PR job.

Ask what the developer knew at each point

The focused preview check had completed 44 seconds after the job started. The composite build finished 149 seconds after that same start. A developer could inspect the early result while broader required verification continued.

Those two times answer different questions. The preview result concerns a small behavior. The later build includes other checks required for the release. Finishing one does not authorize skipping the other.

For this run, rewriting a roughly one-second focused check would be a poor first response to the much longer overall wait. The next investigation should split the composite build into observable stages, then inspect whichever stage takes the time. That could mean recording boundaries already present in the command's logs. It does not require creating more parallel jobs immediately.

Dependency installation is another visible cost. Before changing it, compare several comparable runs and check whether cache restores, lockfile changes, and runner environments differ. This one run does not establish a typical install duration or demonstrate a safe speedup.

Martin Fowler's CI discussion explains the value of a fast build. The practical question here is which completed result helps Maya act on her current change.

Let the largest observed delay choose the next experiment

These are investigation choices, not diagnoses based only on the symptom. For example, increasing parallel jobs can repeat installation and create more waiting. It may make matters worse when runners are already busy.

If a small package change triggers most of a workspace, the affected-test selector example shows how to include dependent packages without guessing from filenames.

Keep a baseline run and change one part of the workflow. Compare the same relevant checks under reasonably similar conditions. Record what changed besides the proposed optimization. Without that context, a quieter runner can look like a successful pipeline repair.

When the build finishes promptly but nobody reviews the patch, measure that different wait with the review-queue snapshot and stage definitions. More build workers cannot assign a reviewer.

The agent can collect the timestamps and explain the configuration before editing anything. It should identify gaps in the evidence instead of filling them with plausible durations.

Keep the pending result usable

Maya still needs to resist unrelated edits to the code awaiting verification. That is a coordination problem alongside the pipeline measurement. The concurrency example explains how repeated pushes can discard useful work.

For the handoff, report the checked source, the early result, and the remaining required work separately. The commit-reporting example shows which revision fields to keep together.

A useful request is simple: identify the slow stage in this run, propose one change, and show how the same required behavior will still be checked. That gives the next patch a purpose. Another round of general cleanup does not.