- Published on
AI Code Review Bottlenecks and How to Measure the Queue
- Authors
- Name
- Mehdi Akiki
Article · Measurement
Measure where pull requests are waiting before treating all review delay as slow reviewers. Time before the first useful review, an open review conversation, author rework, and a dependency on another change call for different next steps. Counting open requests alone hides those differences.
Imagine a team that fixes its slow CI. Agents produce more patches, but the oldest pull request still waits. The usual explanation is that humans cannot keep up. That may be true. It may also be waiting for a product decision nobody assigned.
The earlier team-productivity article discusses coordination more broadly. This example is narrower: make a small review-queue snapshot, use it to choose one intervention, then check whether that intervention helped the work move.
Define the stages before collecting the numbers
Use stages that the team can identify consistently:
| Stage | Starts when | Ends when |
|---|---|---|
| First review wait | Request becomes ready for substantive review | First useful review response arrives |
| Review window | Reviewer and author are discussing an unresolved decision | Changes are requested, dependency is identified, or approval is given |
| Author rework | Requested changes are handed back to the author | Revised change is ready for review |
| Dependency wait | A specific prerequisite prevents progress | That prerequisite is resolved |
The review window is elapsed time, not minutes spent reading code. Someone might review for ten minutes and then wait a day for an answer. GitHub review timestamps alone do not measure their effort. Avoid labeling that whole day “active review time.”
Decide what counts as the first useful response. A bot saying it has started and an automatic reviewer assignment are not evidence that someone understood the change. Likewise, a draft request should not inflate the ready-for-review queue simply because it has been open for a week.
Record a transition whenever responsibility changes. If the author finishes a revision, the age of its new review wait starts there. Carrying forward the original request age is useful history, but it does not describe the current stage.
Take one labeled snapshot
The following snapshot is constructed. The request numbers, waits, and owners are illustrative; they are not measurements of this site's contributors.
| Request | Current stage | Time in this stage | Waiting for |
|---|---|---|---|
| 101 | First review wait | 27 hours | Import owner to accept the review |
| 102 | Author rework | 11 hours | Author to add a missing retry case |
| 103 | Dependency wait | 6 hours | A shared schema change to merge |
| 104 | Review window | 20 minutes | Product decision on an error message |
Keep the snapshot time next to the table. “27 hours” without an observation time becomes misleading as soon as someone revisits it. Include a link to the actual request in a real queue so an owner can correct a stale stage.
The small snapshot helper computes stage age from explicit transitions. It excludes draft entries and rejects unknown stages, duplicated request numbers, and future timestamps. It does not infer who is working from online status or message volume.
Its test for the first row uses a fixed observation time of October 24 at noon UTC and a transition at 09:00 the previous day. The expected difference is twenty-seven hours. That checks the age calculation. It does not validate the human judgment that this request is ready or that its owner is available.
Choose an intervention for the actual wait
Request 101 may need an accepted reviewer assignment. Ask whether the named owner can take it, and reassign through the team's normal process if necessary. Generating a fifth patch for that same owner would add work without resolving this wait.
Request 102 is already with the author. An agent could help add the requested retry case and update the explanation. Sending another review reminder would target the wrong person.
Request 103 needs a dependency decision. Perhaps the schema change can merge independently. Perhaps the application patch should wait rather than open overlapping alternatives. Name the prerequisite instead of describing the whole request as “slow review.”
Request 104 needs a product answer. More code reading may not produce it. Write down the two possible behaviors and get the responsible person to choose. The next diff can then implement that choice without guessing.
These are hypotheses to check with the people involved. A timestamp is a starting point for a conversation, not proof that an individual is underperforming.
Check movement without hiding the remaining work
For one week, record the same stages at a consistent time and follow the same requests. Note new arrivals, stage transitions, merges, and closures. If the oldest first-review wait shrinks only because requests were closed or returned to draft, say that. It is not equivalent to completed review.
Compare like cases where possible. A small label fix and a data migration have different review needs. An average across both can conceal where capacity is missing. The snapshot is deliberately small enough that the team can read the underlying requests instead of trusting one dashboard number.
A team might first try limiting new related requests while the current queue is overloaded. It could then use agents to answer review questions, reduce optional cleanup, or assemble missing evidence for the oldest ready change. DORA's small-batch guidance supports independently useful work; splitting a feature into disconnected fragments can increase the decisions reviewers must hold in mind.
Keep CI required while making these changes. The CI timing example separates machine feedback from the rest of the wait. A faster build and a shorter review queue are different outcomes, and the team needs to see both.
The next agent task can now be specific: help finish request 102's retry case, or explain request 103's dependency. There is a better reason for that task than simply having enough tokens to open another pull request.