More Code, Same Roadmap: What Harvard's 700-Firm Study Reveals About the AI Coding Review Bottleneck

Your engineers are writing more code than ever. Your pull request queue is longer than ever. So why isn’t your product shipping any faster?

A new Harvard working paper, covered by Ars Technica, puts hard numbers on a feeling many engineering leaders already have. Studying more than 700 software firms, the researchers found that AI coding agents lifted code output sharply, yet the rate at which teams actually resolved issues and finished epics did not move in a statistically significant way. The bottleneck simply moved downstream, to the humans who have to read, challenge and approve all that new code.

This post breaks down what the study found, why review is now the constraint, what a separate October survey says about enterprise trust in autonomous agents, and the practical steps your team can take so that faster typing turns into faster shipping.

What the Harvard Study Actually Found

The paper, “Artificial Intelligence in the Firm: Bottlenecks in Software Production” by Harvard researchers Fiona Chen and James Stratton, draws on about 300 million work events from roughly 726,000 workers at 718 firms, covering January 2021 through March 2026. Engineering telemetry came from the analytics platform Jellyfish, paired with GitHub activity.

Output went up

After firms adopted tools such as Claude Code, Cursor and Devin, lines of code rose about 30%, commits about 20% and pull requests about 23%. By any activity metric, adoption looked like a triumph.

Shipped work did not

The authors used the resolution rate of Jira issues and epics as a proxy for finished features. It showed no statistically significant change, and the size and complexity of those issues did not shift either. The paper also found no significant employment effect, which cuts against both the “AI replaces engineers” and the “AI doubles output” narratives. The authors’ own conclusion is that there is “little evidence” firms are increasing software output or reducing headcount.

Review absorbed the gain

The downstream numbers explain why. Time from pull request submission to merge rose about 49% from a 7.03-day baseline, review comments per pull request climbed 35%, and the share of workers doing code review grew 14%. The share of pull requests with requested changes nearly doubled. Notably, pull request size did not change significantly, so the slowdown is not simply a story of bigger submissions.

A short caveat belongs here. This is observational firm-level data, not a controlled experiment, it ends in March 2026 and so predates the newest agent releases, and Jira resolution is a proxy for value rather than a measure of it. Treat it as strong directional evidence, not a law of nature.

Why Review Is the New Constraint

Software delivery is a pipeline, and a pipeline moves at the speed of its slowest stage. For years that stage was writing code. Agents have made that stage dramatically cheaper, so the constraint has shifted to the stages that still depend on human judgment: understanding intent, checking correctness and accepting accountability for what ships.

This mirrors what we saw when we looked at 81 million pull requests and the 46x gap in agent activity: volume is easy to produce and hard to absorb. It also echoes the broader finding in Google’s DORA research, which frames AI as an amplifier of the engineering system you already have. Strong review, testing and release practices get stronger. Weak ones get overwhelmed.

AI reviewing AI is not yet the answer

The obvious fix is to let AI review the code. Firms are trying. By March 2026 roughly 80% of the measured firms used some AI code review, but AI authored only about 23% of review comments and touched about 11% of pull requests. Adoption is broad but shallow. Humans are still doing most of the judging.

Trust Is Moving the Same Direction

A separate data point, from VentureBeat’s own research, shows the same caution at the organizational level. In an August survey of 140 respondents, 56% of organizations deploying autonomous agents allowed, or were building toward allowing, production changes on automated evaluations alone, down from 75% in July. Over the same period, the share expecting a human to keep reviewing agent changes for the foreseeable future doubled from 20% to 42%.

The sample is small and self-selected, so it does not describe the whole market. Even so, it points the same way as the Harvard data: after the first wave of enthusiasm, leaders are rediscovering why a human approval step exists. We explored the risk side of that in what loopjacking means for human-in-the-loop guardrails, and the operational side in how on-call engineers are becoming approvers.

What Engineering Leaders Should Do Now

1. Measure delivery, not activity

Lines of code, commits and pull request counts will all flatter an AI rollout. Track what the study tracked downstream: issue and epic resolution, pull request cycle time, review turnaround, change failure rate. If activity rises while those stay flat, you have found your bottleneck.

2. Treat review capacity as a budget line

If agents multiply the work arriving at review, review capacity has to grow, be redistributed or be made cheaper per change. That means protecting senior reviewer time, setting expectations for turnaround, and rotating review duty so it does not fall on two or three people.

3. Shrink what reviewers have to read

Require agents to produce small, single-purpose pull requests with a clear description of intent, the tests added and the risks. Reviewers spend less time reconstructing context, and a smaller diff is easier to approve or reject with confidence.

4. Push verification left

The cheaper it is to prove a change is correct before a human sees it, the less review costs. Strong test suites, type checks, linters, security scanning and preview environments let machines reject obvious problems. Humans then spend their attention on design and risk. This is also where technical debt in the age of AI is either contained or compounded.

5. Use AI review as a first pass, not a verdict

Given the study’s finding that AI covers a minority of review comments, position AI reviewers as a filter that catches routine issues and summarizes changes. Keep a named human accountable for approval on anything touching security, payments, data or customer-facing behavior.

6. Price the whole workflow

Agent spend is visible on an invoice, but review hours are not. When you compare tools, include review and rework cost per merged change. Our look at the cost-per-task shift in the coding agent price war is a useful starting framework.

Conclusion

The Harvard study does not say AI coding agents are useless. Individual engineers really are producing more code. What it shows is that code was never the only constraint, and removing one bottleneck exposes the next. Firms that treat agents as a typing accelerator will see busier dashboards and unchanged roadmaps. Firms that redesign review, testing and accountability around the new volume have a real chance of converting output into shipped product.

Your actionable takeaways:

• Baseline pull request cycle time and issue resolution before and after any agent rollout.
• Invest in review capacity and automated verification at least as heavily as in generation.
• Require small, well-described agent pull requests.
• Keep humans accountable for high-risk merges, and use AI review as a first pass.
• Re-measure quarterly, because the tools are changing faster than the studies can keep up.

Frequently Asked Questions

Does the study prove AI coding agents don’t improve productivity?

No. It shows that, across 700+ firms through March 2026, higher code output did not translate into a statistically significant rise in resolved issues and epics. It is observational, uses a proxy for shipped work, and predates the newest agents.

Why did pull requests take longer to merge?

The authors report that review comments per pull request rose 35%, requested changes nearly doubled and merge time rose about 49%. More code plus more scrutiny meant reviewers became the constraint, even though pull request size did not change much.

Can AI code review solve the bottleneck?

Not yet, according to the data. About 80% of firms used some AI review by March 2026, but AI wrote only around 23% of review comments. It helps as a first-pass filter, not a replacement for accountable human approval.

Should we stop letting agents make production changes?

Not necessarily, but scope it. VentureBeat’s survey shows more enterprises expecting to keep humans reviewing agent changes. Allow automated merges only where tests, rollbacks and monitoring make failures cheap to catch.

What should we measure instead of lines of code?

Track delivery outcomes: issue and epic resolution, pull request cycle time, review turnaround, change failure rate and time to restore service. Compare them before and after adopting agents.

Sources

• Ars Technica: AI coding agents generate more code, but not more software - original news coverage of the Harvard working paper
• Slashdot: AI Coding Agents Generate More Code, But Not More Software - summary of key figures including AI review share
• AI Weekly: Harvard Study on AI Coding Agents - summary of output, review and caveats
• Winzheng: AI coding agents and the review bottleneck - dataset size and merge-time details
• VentureBeat: Share of enterprises trusting AI agents to make production changes falls from 75% to 56% - August survey of 140 respondents
• DORA: 2025 Year in Review - AI as an amplifier of existing engineering practices