There Was Never a Person in That Seat
Fixed-task studies measure whether AI helps with familiar work. My repository history raises a different question: does AI change the range of work one person attempts?
Nino Chavez
Product Architect at commerce.com
A friend described the AI he wants as plus ten to a stat in Baldur’s Gate. I liked it and pushed it further: a cloak of intelligence. It makes you better. If your numbers start low, do not expect much magic.
Then I looked up what the item actually does.
Set the wearer’s Intelligence score to 17. The enchantment has no effect if their Intelligence score is higher without it.
It does not add. It sets.
That reversal gave me a clean explanation for two AI studies. It also tempted me into a claim my evidence cannot support.
The studies measure how AI affects performance within a fixed kind of work. They do not measure work someone attempts only because AI made it accessible. My repository history demonstrates breadth in one person’s activity. It does not prove that breadth predicts gain, quality, or value.
That boundary is the point. The rest is an open question.
The studies measure familiar work
Brynjolfsson, Li, and Raymond studied 5,179 customer-support agents. Access to a generative assistant increased issues resolved per hour by 14% on average. Novice and lower-skilled workers improved by 34%. The effect was minimal for experienced and higher-skilled workers.
METR’s early-2025 randomized trial followed 16 experienced open-source developers working on 246 issues from repositories they already knew. With AI available, they took 19% longer.
Both findings matter. Both also have a boundary.
The support agents still handled support conversations. The developers still worked on issues from their own repositories. The tool changed. The kind of work did not.
That is the right design for asking whether AI helps someone perform familiar work faster. It cannot answer whether AI changes the work that person chooses to attempt.
METR later encountered this distinction directly. In a 2026 update, 30% to 50% of surveyed developers said they withheld some tasks because they did not want to perform them without AI. Some also reported choosing different kinds of tasks or producing different amounts of documentation and testing.
Once the tool changes task selection, a fixed basket stops describing the whole effect.
My repository history contains a different signal
I counted 11,801 commits across 117 repositories: 104 on GitHub and 13 local checkouts that had never been pushed.
Coding is the largest category. It appears in 7,839 commits across 107 repositories. That is the work I trained to do.
But it is not the only work in the record. Design appears in 88 repositories. Operations appears in 89. Research appears in 81. Sales appears in 52. The categories overlap because the work overlaps.
Those are counts, not pictures. The record includes a therapy practice’s rebrand, with colors pulled from its existing logo. It includes accessibility corrections, social graphics, sales copy, evaluation plans, handoff notes, and thousands of documents under docs/.
There was no designer, copywriter, researcher, or documentation lead sitting beside me for most of that work. In several cases there was no open seat to fill. The work would have remained part of somebody else’s job, gone to a contractor, or not happened.
The repository history shows that the range of my activity expanded. It does not tell me why.
The record can show that I attempted more kinds of work. It cannot show that AI caused the expansion or made the results good.
I already caught the record flattering the argument
My first analysis reported that Markdown outnumbered TypeScript. That would have made a convenient headline: I write more than I code.
It was wrong.
The collection script kept only the first 40 changed files in each commit. Large scaffolds and dependency sweeps are code-heavy, so the cap removed more code than prose. I also called repeated edits “files,” which overstated both sides.
Without the cap, TypeScript leads:
| distinct files | edits | edits per file | |
|---|---|---|---|
| Markdown | 7,404 | 25,424 | 3.4 |
| TypeScript | 13,478 | 34,007 | 2.5 |
The correction did not erase the breadth in the repository history. It did erase a result that made the story cleaner.
That matters because every other limit runs in the same direction. Keyword categories can show activity without showing a delivered artifact. Private repositories prevent most readers from checking the classifications. Photography and video are undercounted because their output usually lives outside Git. A commit does not say whether a client approved the work. It says nothing about quality.
The census is evidence of breadth. It is not a score for value.
The list is a research question, not an estimate
I originally wanted breadth to explain why my experience felt different from the fixed-task studies. Then I promoted that explanation into a prediction: count the kinds of work you touch, and that count predicts what AI is worth to you.
My data cannot carry that claim.
Breadth might predict gain. It might predict nothing more than activity. The added work might be useful, mediocre, or actively harmful. AI might have caused the expansion, accelerated an expansion already underway, or merely left a visible trace in work I would have attempted anyway.
Testing that would require more than repository counts. We would need to observe which tasks people considered, which they attempted, whether AI changed that choice, what they completed, how independent reviewers judged the results, and what value the work produced.
Until then, the honest conclusion is narrower:
Fixed-task studies tell us whether AI helps someone perform familiar work faster. They do not tell us whether AI changes the range of work that person attempts. My repository history shows that range expanded in one case. It does not yet show that the added work was valuable, good, or caused by AI. Whether breadth predicts gain remains an open research question.
The companion whitepaper, The Sum of Your Gaps, separates the measured claims from the open ones and outlines what a useful study would need to observe.
There was never a person in that seat. There is still work to measure.