AI Coding Agents Generate 30% More Code but No More Software, Harvard Study Finds

A Harvard study of engineering activity at more than 700 firms found that AI coding agents increase code output sharply — 30% more lines of code, 20% more commits, 23% more pull requests — without producing a statistically significant increase in resolved issues or epics. The reason is that human code review absorbs the gain: average pull request review time rises 49%, and the share of pull requests requiring changes nearly doubles.

The study, by Harvard University researchers Fiona Chen and James Stratton, is titled around Jellyfish analytics data and is available as a PDF at fion.ac/jellyfish.pdf. Ars Technica's Kyle Orland covered it on October 9, 2026 under the headline "AI coding agents generate more code, but not more software," with the subhead "Study finds coding efficiency gains get 'absorbed' by human review 'bottleneck.'"

The study's data covers 300 million work events from 2021 through March 2026

The research draws on aggregated analytics from Jellyfish, a platform that measures the granular output of engineering teams. According to Ars Technica, the dataset encompasses 300 million individual "work events" — commits, pull requests, and similar artifacts — plus issue management software data, spanning more than 700,000 employees at over 700 software development firms, from 2021 through March 2026. One commenter quoting the paper directly cites the firm count as 718.

To identify when AI tools entered each firm's workflow, the researchers combined directly measured AI usage with analyses of GitHub activity, distinguishing AI coding assistants (which autocomplete code primarily authored by humans) from AI coding agents (which write and submit code autonomously from prompts). They then ran a "difference of differences" regression on key variables before and after tool introduction at different points in time across different organizations.

Code volume rises sharply; shipped features do not

The raw output effects are large and consistent. Ars Technica reports that introducing AI coding agents at a firm leads to:

• A 30% increase in total lines of code generated
• A 20% rise in total commits
• A 23% increase in pull requests

The researchers found no comparable movement in delivered work. The resolution rate for Issues and Epics tracked in tools like Jira — the study's proxy for wholesale software features — did not change in a statistically significant way after AI tools were introduced. The researchers also found no "compositional shift" in the size or complexity of those Jira-tracked issues across the AI introduction, meaning the flat resolution rate is not explained by teams tackling bigger or harder tickets.

Code review is the bottleneck that absorbs the productivity gain

The study locates the missing output in the review stage. After AI coding agents are introduced:

• The average review process time — from pull request submission to merge — balloons 49%
• The share of pull requests with changes requested nearly doubles
• The number of comments per pull request increases by 35%
• The share of workers performing code reviews rises 14%

In the paper's framing, any efficiency gained during the coding phase is "absorbed by downstream constraints in the production process," and "the code review process significantly increases in length, pull requests are more likely to require revisions, and reviewers leave more comments." The study concludes there is "little evidence that firms increase software output or reduce employment" from adopting these tools.

AI code review is not yet absorbing the load

AI-assisted review exists at most of the firms studied but handles a minority of the work. By March 2026, 80% of measured firms used some form of AI code review, yet AI agents were responsible for only 23.3% of all review comments and 10.8% of all pull requests. Humans still performed the vast majority of review work.

The study found no significant employment effect

Looking at total active workers across Jellyfish data and cross-referencing LinkedIn data at the same firms, the researchers write that they "cannot attribute significant employment changes to AI." The 14% increase in the share of workers doing code review is a reallocation of existing effort rather than a net change in headcount.

Critique 1: the data is mostly from 2025, and the models are old

The most repeated objection in the Hacker News discussion is that the study describes a world that has already moved on. Commenter jefftk wrote:

As is standard for academic papers, this is an attempt to describe the state of the world as of when they did the research. But then it takes time to write it up and get it out, after which newspapers report it as current events. In this case, we're looking at a picture from 2025. The models involved in the main agent results were primarily Claude 3.5 through Sonnet 4.5, GPT-4.1 through GPT-5, and Gemini 2.5. And the paper's design means you're mostly looking at the very initial effects of deployment, before companies developed good review processes.

Commenter hyperpape, who said their director shared the paper, pulled specific baseline figures to make the same point: a baseline mean of 5,017 lines of code per worker-month with an estimated increase of 1,494 lines, a baseline of 5.34 pull requests per month increasing by 1.22, and a baseline mean code review time of 7.03 days. They argued the small percentage increases confirm the data is old, and separately criticized the use of means rather than medians:

If your average pull request sits for 7 days, you evidently don't care about shipping anything, and agents aren't going to change that.

Critique 2: "more software" is measured as tickets closed

Commenter dcre flagged the dependent variable directly:

Note that they define "more software" by "more tickets closed." Make of that what you will. They argue that this is a good measure because the scope of tickets did not increase. Worth considering whether you believe this captures what they claim it does. Are the extra tests you write thanks to coding agents represented in the tickets for your work?

Critique 3: the sample excludes exactly the teams most likely to benefit

Because the dataset comes from firms that use Jellyfish and generate a measurable footprint of work events, several commenters argued the study structurally misses solo developers, small startups, and personal projects. Commenter dolebirchwood wrote:

What about all the software being built by everyone else not using Jellyfish? What about the small companies that don't have the same operational bottlenecks and truly are building more than their larger competitors? What about the solo founders starting companies and building software which they never would have built without AI?

Commenter joshheitzman gave a concrete example — a project called FluxImp that they started with Kilo Code and that later became self-modifying, developed primarily with Mercurial on a private server so that "there is very little github activity." They added that "all of the stories about non-software engineers making small software that is useful to them also seems unlikely to generate the kind of footprint that lends itself to being easily studied."

Commenter KumaBear made the same point more briefly: "Homemade personal use software has Definately skyrocketed. Projects I never thought I'd even have the time for are now possible." Commenter juliendorra compared the moment to HyperCard, describing a boom in personal software of limited scope — designers building a tool per project, developers making art software, super users requesting plugins and custom workflows.

Critique 4: the bottleneck may be organizational, not cognitive

Several commenters argued that code review was never the real constraint. Commenter blauditore offered a racing analogy: engines eventually outpace tires, so superfuel yields only marginal lap-time gains. Commenter gchamonlive said communication is the hardest part of software production and that "software production has become a sibling field to linguistics, more than it already was." Commenter jmward01 argued the bottleneck is the team's ability to evolve a product with a clear vision and the right next step, and that "our process sucks in this industry" independent of AI.

Commenter forgotaccount3 pointed at planning rituals rather than review, describing stories written with enough fidelity that "we may as well copy it into an AI agent and have it implement it," then sitting in a backlog for two weeks because of story-point and sprint-capacity limits.

Practitioners disagree on whether the finding matches their experience

Some commenters reported the study matches their working lives. The Ars Technica staff pick, from user metavirus, was blunt:

I'm forced to use AI at work and this is absolutely 10000% true. Soooooo much slop code goes into the basin, which dozens of people need to sift through, strain out the dreck, then eventually close their eyes and release into the wild because now we have new SLAs so we're not seen as a bottleneck to progress.

Commenter PacificSpecific reported that a friend left a job because 80% of the work was reviewing Claude submissions. Commenter atleastoptimal described the arithmetic of the new review load: "instead of reviewing 1,000 lines a day I now 'review' 10,000 lines a day. My brain is not 10x faster."

Others reported the opposite. Commenter keithnz said their team is producing more software and being more ambitious, crediting heavy upfront investment in specification, architecture, coding, performance, and testing guidance. Commenter tdhz77 wrote, "I've created more complete software since ai. This is a crazy notion." Commenter mrieck argued that testing, reviewing, and refining now take more time precisely because the volume of software being produced is larger, and that startups can validate roughly 10x as many things as before.

Why code review will not disappear even if it gets automated

Commenter zmmmmm raised the accountability argument, which is distinct from code quality:

An automated review is going to help to ensure the code behaves as intended, but it is not going to ensure the intent was correct. Employees inserting back doors, bypassing tests, "cheating" on design decisions to get the work done faster etc are not going to be identified if the person driving the change tells the AI it is the intended outcome.

They concluded that the durable protection is "a human being willing to put their reputation on the line by saying 'I understand this change and I approve it.'"

Commenter staminade predicted where a change in review culture would come from first: not established teams, but "startups that adopt a no-human-review policy from jump and then maintain it as they mature," which would be largely invisible to current studies. If those companies ship significantly faster, staminade argued, the rest of the industry would be forced to follow.

What the study does not settle

The paper measures firms that already had engineering analytics in place, over a window ending in March 2026, using Jira issue resolution as the definition of shipped software. It does not measure personal projects, solo founders, or teams that replaced the pull-request workflow entirely. It also captures the early deployment period of agentic coding, before most organizations had settled on review practices for AI-generated code.

As Ars Technica notes, 95% of firms in the study had implemented AI coding agents by the data cutoff, and agent capabilities have continued to change since. The "coding time versus review time" trade-off the study documents could shift as teams learn when to hand a problem to an agent and when not to. For now, the measured result is a double-edged one: faster code generation offset by comparable increases in human review time and effort.