Kasra VaziriThe Verification Bottleneck: AI's Real Cost to Teams
Generating a draft—code, a PRD, a design—is nearly free now. Checking whether it's any good is not. That shift should change how you run a product team.
Someone on my team generated a full PRD last month in about twenty minutes. Clean structure, crisp problem statement, three solution options with trade-offs. It looked done. Then we spent the better part of three days figuring out whether it was actually right — and two of those options quietly assumed a data model we'd deprecated in the spring.
Twenty minutes to produce. Three days to trust. That gap is the whole story of building with AI right now, and it has a name: the verification bottleneck. The cost of making things has fallen through the floor. The cost of knowing they're any good has not. If you run a product team, this has quietly become the most important constraint you're not budgeting for.
Where the verification bottleneck hides
The most honest data on this is also the most uncomfortable. Last year, METR ran a randomized controlled trial with 16 experienced open-source developers working on their own repos, using frontier tools of the moment. Beforehand, the developers expected AI to speed them up by 24%. Instead, it slowed them down by 19% — and here's the part that should keep you up at night — even after finishing the tasks, they still believed AI had made them about 20% faster.
Sit with that. The tools didn't just fail to help. They broke the developers' ability to tell whether they'd been helped. That's the signature of a verification bottleneck: the work that got cheaper — typing the code, drafting the doc — is loud and visible, so it feels like progress. The work that got more expensive — checking whether any of it is correct — is silent until it bites you in production, in a board deck, or in front of a customer.
Velocity, it turns out, is a terrible proxy for progress when generation is free. You can feel fast and be slow at the same time.
Generation is easy to see. Verification is easy to skip.
There's a fundamental asymmetry here. Generating an artifact is fast and feels like momentum. Verifying it is slow and feels like friction — because to review AI output honestly, you have to reverse-engineer intent from something that looks equally confident whether it's right or wrong. A junior engineer's shaky code looks shaky. A model's wrong answer looks exactly like its right answer.
And the volume keeps climbing. GitClear analyzed 211 million changed lines of code and found that copy-pasted code rose from 8.3% to 12.3% of all changes between 2021 and 2024, while refactoring — the disciplined work of reusing what already exists — collapsed from 25% of changed lines to under 10%. For the first time in the history of their data, cloned code outpaced refactored code. AI is exceptional at adding lines and poor at reusing them, which means more surface area to review, more duplicated logic, and more places for a bug to hide in plain sight.
Doug Turnbull put the human side of this well just this week: when you stop writing things yourself and only skim the diffs, "slop flies under the radar." Passive review isn't review. Reading a flat diff of something you didn't think through is not the same as understanding it, and the model is happy to let you believe otherwise.
This is not a coding-only problem — it's the shape of all knowledge work now. It's a PRD problem, a design problem, a data-analysis problem, a support-macro problem. Law firms have been sanctioned for filing AI-hallucinated case citations. Consultancies have retracted reports built on fabricated figures. As Daniel Keller argues, "AI slop" has graduated from a social-media insult to a boardroom risk. Every one of those failures was a verification failure, not a generation failure. The draft was fine. Nobody checked it.
Make verification a first-class product decision
Here's the reframe I've been pushing with my own team: if generation got 5x cheaper, then reviewing is the job now, not the thing you do after the real work. A few concrete moves.
Budget for it out loud. Keller's provocation is to invert the old ratio — spend 30% of your effort on generation workflows and 70% on verification infrastructure, because "generation is a commodity. Verification is a moat." You don't have to hit those exact numbers to take the point: the team that can trust its output faster than yours will beat you, even if you both produce at the same rate.
Verify intent before you generate output. The cheapest review is the one you do before a single line exists. Approve the spec, the acceptance criteria, and the eval set first, so the model is filling in a shape you've already validated rather than inventing the shape and the content at once. This is the same instinct behind treating AI evals as the new PRD — you write down what "good" means before you let the machine chase it.
Stop rewarding volume. Lines of code shipped, tickets closed, PRDs produced, pages published — every one of those metrics is now trivially inflatable, which means every one of them is now a lie. Measure whether the thing actually worked: did the feature move the number, did the doc prevent the rework, did the support reply resolve the ticket. Output is free. Outcomes are the only honest currency left.
Decide explicitly who checks what. A reviewer is a role with teeth, not a rubber stamp at the end of a queue. Name the person accountable for verifying each class of AI-assisted work, and give them the time the METR study says the work actually takes. "The AI wrote it" is not a chain of custody.
The scarce skill is judgment
When generation becomes a commodity, the thing that's left — the thing that's suddenly worth a premium — is judgment. Knowing what "right" looks like. Being able to tell, quickly, whether an artifact is trustworthy or just confident. That's not a tooling problem you can buy your way out of; it's product taste, applied at speed, over and over.
So the question I'd ask your team this week isn't "how much more can we ship with AI?" You already know the answer is a lot. The better question is: how fast can we tell if what we shipped is any good? Close that gap, and the verification bottleneck stops being the thing that quietly slows you down — and becomes the moat nobody can copy by buying the same model you did.