AI Coding Tools Didn't Change My Output — My Workflow Did

I started taking AI tools seriously around August of last year. For the first month or two it was mostly frustrating. Sometimes the output was good; most of the time it wasn't. That friction is what eventually mattered — it forced me to stop optimizing the prompts and look at the surrounding process. That shift — from asking "how do I get better outputs" to asking "what does the process around it need to look like" — is where this article starts. That's where the interesting part starts.

The moment that made this concrete wasn't when AI did something impressive. It was when it did something mundane — at scale. Design docs for small features. Config notes for routine changes. Decomposition of problems I would have just powered through. Things I'd always known how to do but never bothered with, because the effort didn't justify the result. AI changed the economics, and the economics changed my process. That pattern repeats through everything I'll describe.

What follows is my experience over six months, trying to describe honestly what happened. I'll point to research where it exists and say so when I'm working from observation alone.

The Adoption Arc

I'd been circling AI for a while before I used it seriously. My university thesis dealt with demand-driven execution for TensorFlow, so when ChatGPT launched, I wasn't the person who needed convincing that the technology was real. I just hadn't figured out how to use it yet. The marketing was loud, I was skeptical, so I didn't give it much thought. I'd ask it to name a class, explain a function, recommend a movie. A curiosity. A toy with good manners.

That changed around August 2025, when I started using AI for actual coding. I didn't have a mental model for what it was. I was experimenting — asking it to name things, explain things, write things I could have written myself. The AI was doing the typing, I was doing the thinking.

September is when things accelerated. I subscribed to Claude, subscribed to ChatGPT, tried half a dozen others. I was burning through weekly usage caps within days, switching between services to keep momentum. The tools were rough: context windows too short, hallucinations in unfamiliar codebases, confident answers that were flatly wrong. But something was forming underneath the friction. Not in any single model's capability, but in the shape of how I was starting to work. Instead of "add error handling to this function," I was saying "here's what this module needs to do: draft a plan." I was writing design documents with AI, reviewing them, iterating before approving any code. Before plan mode existed as a feature, I was doing a manual version of it.

By early November, the personal projects felt different. The habits I'd been forcing on myself — writing docs before prompting, fixing the process instead of the output — had started working. The company I work for rolled out Claude Code to engineering around the same time. Near-unlimited access was the final piece: no more switching between services to stay under caps, no more momentum breaks. AI went from "a tool I use sometimes" to the primary interface for coding. The tools didn't get better. My approach did.

By January, my usage had spiked to the API equivalent of roughly two thousand dollars a month. At work, I was running five to ten concurrent AI sessions on any given day. For personal projects, two or three tasks across about five repositories. Projects that had sat on my mental backlog for years (a CLI for managing development workflows, a personal dashboard, this digital garden) started becoming things I actually used. In a month, three or four items moved from "someday" to "daily driver."

I want to be precise about what changed. It wasn't that I suddenly had more hours. The bottleneck shifted. Implementation used to be the constraint; now implementation was cheap. The new bottleneck was deciding what's worth building and verifying that what got built is actually correct. That shift, from creation to judgment, is the thread that runs through everything I've learned.

I was convinced AI had made me more productive. Then I looked at the research.

The Productivity Paradox

The story I wanted to tell was simple: I adopted AI tools, I got faster, end of section. But I owe it to the argument to engage with what the studies actually show, and what they show is a mess.

Start with the positive results. A controlled study found Copilot users completed a JavaScript task 55% faster (Peng et al., 2023); a Microsoft field study of 5,000 developers showed a 26% increase in weekly pull requests (Cui et al., 2024); Google reported 21% faster task completion in a randomized trial (Paradis et al., 2024). These are real numbers from real organizations, and they match the vibes.

Now the other side. The METR study recruited 16 experienced open-source developers to work on their own repositories — codebases they knew intimately — and found they were 19% slower with AI assistance (Becker et al., 2025); Uplevel tracked around 800 developers and found negligible cycle-time improvement alongside a 41% increase in bug rate (Uplevel, 2024); the DORA 2024 report found a modest productivity improvement paired with a decline in delivery stability (DORA, 2024); Faros AI analyzed over 10,000 developers and found that while individuals reported gains, none materialized at the organizational level (Faros, 2025).

Here's what makes this genuinely confusing rather than just contradictory: the pattern isn't random. Studies showing large gains tend to involve lab settings, greenfield tasks, relatively junior developers. Studies showing null or negative results tend to involve real-world codebases, mature systems, experienced engineers. The METR result sticks with me — those developers predicted they'd save 24% of their time. They lost 19% instead. If you've reorganized your workflow around these tools, that perception-reality gap is uncomfortable to sit with. It gave me pause.

And there's a measurement problem underneath all of it. The SPACE framework identifies five dimensions of developer productivity: Satisfaction, Performance, Activity, Communication, Efficiency (Forsgren et al., 2021). Most AI productivity studies measure only Activity: lines of code, pull requests merged, tasks completed. When I look at my own output, I can point to impressive numbers. But I genuinely don't know if that means I was more productive or just more prolific. Those aren't the same thing.

We've seen this before. In 1987, Robert Solow wrote a line that became famous: "You can see the computer age everywhere but in the productivity statistics." Companies had spent billions on PCs and enterprise systems, and the macro data showed nothing. Erik Brynjolfsson named this the "productivity paradox" in 1993 and argued that general-purpose technologies require massive unmeasured investment (workflow redesign, retraining, organizational restructuring) before they pay off.

Productivity doesn't just lag; it initially dips. His "J-Curve" model maps onto the pattern we're seeing now: individual gains that don't show up in aggregate metrics yet because the complementary investments haven't matured. The original IT paradox took roughly a decade to resolve. The resolution didn't come from better computers. It came from organizations learning to work differently.

This changes what the mixed research results mean. They aren't evidence that AI tools don't work. They're evidence that bolting AI onto existing workflows doesn't work — which is a very different claim. The evidence I've seen points toward process design as the strongest moderator. Developers who redesign their workflow see larger gains; those who treat AI as autocomplete-plus see marginal improvement at best. But I should be honest that this is my interpretive frame, not a proven ranking. I can't prove AI made me more productive. But I can describe what I actually changed — and it happens to match what the research says matters.

Workflow Evolution

The No Hands Rule

Mid-September 2025, I was bouncing between Claude and ChatGPT, burning through usage caps, when the AI got stuck on something I could have fixed in ten minutes. My fingers were already on the keyboard. Then I stopped. If I jump in and fix this, I learn nothing. The AI failed — that's diagnostic information. Patching the output hides a process failure.

So I made a rule: no hands on the code. Not "never touch a keyboard" — I still write docs, outlines, specs. But when the AI produces broken output, I don't fix the code. I fix whatever caused the AI to produce broken code. Was the context insufficient? Was I asking it to hold too many things in its head at once? Was my design unclear — not to the AI, but in general? The answer was almost always upstream. A vague spec, a task that needed decomposition, a missing document that I'd been carrying around in my head instead of writing down. Fix the cause, then let the AI try again. It's the difference between fixing defects and fixing the process that produces them.

This is not laziness. It's discipline, and it's genuinely unpleasant. Watching an AI spin its wheels on something you could solve in minutes requires a specific kind of patience that I don't naturally have. But the discomfort is the point. Every failure I refused to patch by hand forced a concrete improvement somewhere else in the workflow.

The downstream effects were immediate. The number one reason the AI got stuck was unclear intent, which meant I started writing more documentation, not because documentation is virtuous, but because I needed it to stop failing. Repeated failures on large tasks forced me to think seriously about problem decomposition, which eventually led to running agents in parallel. Failures I would have silently fixed became visible patterns — tunnel vision, context loss, verification gaps — and those patterns demanded structural solutions. Each of the workflow changes I'll describe next traces back to a specific class of failure that the no-hands rule made impossible to ignore.

There's a risk I haven't resolved. At some point, avoiding the keyboard might mean I've lost the instinct, not that I've transcended the need. I've been coding for about thirteen years, eight of them professional. The no-hands rule works partly because I can recognize what's wrong even when I choose not to fix it directly. I don't know how long that lasts if I stop practicing.

The Documentation Shift

Unclear intent was the first thing the no-hands rule surfaced. The AI got stuck most often not because the problem was hard, but because I hadn't explained what I actually wanted.

The clearest example is a dependency injection framework I built for a personal project. My initial design had a fundamental architectural flaw: the container handled both registry and lifecycle orchestration, which should have been separate concerns. I'd been working with AI agents on this for days, and they were faithfully executing my flawed design. Clean code, passing tests, confidently wrong.

It was only when I stepped back to document the system at a high level that the gap became visible. Writing forced me to make my assumptions explicit, and one of them was wrong. I externalized the bootstrap phase into a separate runtime, shifted from a monolithic processor to composable rules. Once the design was corrected, the rewrite took hours. The documentation didn't just help the AI — it helped me see what I'd been too close to see.

That experience changed how I start every significant piece of work. The friction from the no-hands rule kept pushing me toward document-first work — if I couldn't fix the code, I had to fix the context. What started with simple designs evolved into ADRs (Architecture Decision Records), then formal design documents, then roadmaps, phased delivery plans that kept work on track. By November, every significant feature started with a document.

LLMs respond to documentation the way they were trained: as readers of human-written context. When I started treating docs as the primary input to AI rather than code, output quality improved. And the persistent docs solved what I think of as session amnesia: without them, agents made different assumptions every session. I'd explain the same architectural decision repeatedly. Once documented, agents understood the full picture. Sometimes they caught things I'd missed, because the document forced me to be explicit about assumptions I'd been carrying around silently. Documentation is durable because it serves humans first — I'd want these artifacts even with perfect model memory. They happen to make models work better too, but that's a side effect.

I pushed this within my squad at work. The reception was positive: documentation with context on things LLMs or engineers would frequently ask about. Not just what exists, but why it exists that way. What makes documentation useful for AI turns out to overlap heavily with what makes it useful for humans.

Parallel Execution

Once the no-hands rule forced better problem decomposition, concurrency became a natural next step. I stopped working on one thing at a time.

At work, I typically run five to ten concurrent AI sessions. I built a CLI called Sylox to manage the infrastructure (git worktrees, Docker containers, isolated test environments) so each session gets its own sandbox. A deliberate architectural choice in Sylox: the model is a swappable leaf node. Claude Code recently shipped its own worktree management, but I don't like how they handle it. Tomorrow I might want Codex, Gemini, or DeepSeek. If your workflow depends on one vendor's tooling beyond the model itself, you can't move. The mechanism is straightforward. What it does to your attention is not.

I think of it as speed chessing — like playing multiple chess boards at once, each on a clock. You don't finish one game before starting the next. You move between boards, make a decision, move on.

The mental model that clarified this for me is CPU pipelining. A processor doesn't get faster by executing each instruction quicker — it gets more throughput by pushing multiple instructions through different stages simultaneously. Fetch, decode, execute, writeback: four instructions in flight at once, each at a different phase. My sessions work the same way. One is in init: I'm describing the problem, loading context, doing the brain dump that gets the agent oriented — sometimes literally talking through it with a voice-to-text tool I built. Another is mid-execution: I'm approving shell commands, glancing at intermediate output. A third is in review: I'm reading the finished diff, deciding what needs follow-up. Each stage demands a different kind of attention. Init is the bottleneck — it requires deep focus, and I can only do one at a time. Everything else can overlap.

The skill that developed over time — and I didn't expect this — is triage. Speed chess changes what kind of thinking you do: pattern recognition over deep calculation. I've learned when a session needs real attention (the agent is confused, or heading somewhere wrong) and when it needs a quick nod (a trivial error it will sort out on the next pass). It's similar to incident triage in software engineering, but at a frequency I'd never operated at before. Not a natural gift — a cognitive muscle I built through repetition.

Implementation takes thirty minutes to two hours. Review and merge takes one to two days. When you internalize that ratio, the picture clarifies: I'm the shared resource. The pipeline's throughput isn't limited by how fast the agents work. It's limited by how fast I can evaluate what they produce.

That creates an obvious downstream problem. Output increases dramatically, so review demand increases proportionally. The biggest productivity gains show up on experimental repositories where there's no review gate. On production code, the speed of implementation exposes the slowness of everything else: peer review, QA, staging, the pipeline to production. There are physical limits too — worktrees and containers devour disk and memory, and my machine freezes under concurrent test runs. But the binding constraint isn't hardware. It's me.

Composable Agents

LLMs have a default behavior I think of as tunnel vision. They pick one approach, commit fully, and optimize for it. If you don't intervene, you get a confident, coherent, and potentially wrong implementation that's hard to evaluate because it all hangs together internally.

I didn't figure out the fix on my own. Mitchell Hashimoto and others in the community were already describing the pattern: run multiple agents in parallel, compare results, pull the best parts together by hand. When I read that, it wasn't a revelation — it was recognition. I'd been bumping into the same problem during UI work. I'd describe what I wanted, and the agent would produce one design and optimize for it. The design might be fine, or it might have made assumptions I wouldn't have — but I had nothing to compare against. What changed was making it deliberate. Five approaches to the same problem, from different sessions. Then I'd combine elements, provide feedback, sometimes run another round. The results were consistently better than any single agent's output. The same logic applies adversarially: for big changes, my first step is having the AI interview me — ask questions, challenge assumptions, surface what I haven't considered. It's another way of refusing to let a single pass go uncontested.

This became a general principle: treat agents as composable boxes, each operating within a margin of error. A second perspective, even an artificial one, breaks the internal coherence that makes single-agent output hard to evaluate. I'm applying the same pattern to code review: one agent writes, another reviews, a third implements fixes, another reviews again. Two or three automated rounds before a human sees the PR. The reviewing agent isn't smarter — it's that having any second pass catches the things the first pass optimized around.

The idea goes further than I've been able to build. If each agent step is a stage in a pipeline, misalignment at any step amplifies error downstream — a probabilistic process where signal degrades without calibration. I think there's a full essay in treating multi-agent flows as algorithms with analyzable properties. For now I'm experimenting, not building theory.

I'm still running these flows ad hoc. I describe the process from scratch each time rather than formalizing it, partly because every tool and integration consumes context window, and I've seen what happens when agents drown in instructions. The tradeoff between capability and focus is real.

The Verification Burden

Every change I've described — the no-hands rule, documentation, concurrency, delegation — solved a real problem and moved the bottleneck to the same place: from creation to verification. The job didn't get faster. It moved to a different level of abstraction: not writing software, but verifying that what got written is actually correct.

Where Verification Fails

This manifests in a way I didn't expect. Before AI, if I was tired, sick, or having a bad day, I simply wouldn't write code. The barrier to creation was high enough that my own state acted as a quality filter — bad days meant low output, which meant less to go wrong. Now I can still prompt an LLM on a bad day, and it'll produce code that gets a job done. Not necessarily the job I want it to get done. My review quality drops, and the verification burden silently passes to my coworkers, or slips through to production. AI doesn't lower the quality bar for creation. It lowers the bar for when you choose to create, which is a different and more insidious problem.

There's a more subtle version of the same failure. I spent weeks on a reflection library in a domain I didn't know well. Without AI, I'd have made little progress by hand, realized early it wasn't worth pursuing, and dropped it. With AI, I got far enough, fast enough, with low enough effort that I kept going — progress felt real. The problem was I lacked the domain knowledge to judge how close I was to done. AI masked the "are we there yet?" signal. Eventually it became clear the code was garbage. I dropped it. The drop was easy precisely because my investment was low. But the weeks were still gone. When you can't verify quality, velocity is just a way of getting lost faster.

The Numbers Back This Up

This isn't just my experience being bad at reviews on tired afternoons. The data suggests verification is where AI-assisted development structurally breaks down. Faros AI found that teams with high AI usage merged 98% more pull requests — but review time per PR increased 91% (Faros, 2025). More output, slower verification. The bottleneck didn't disappear; it moved. And the quality of what's being reviewed is genuinely harder to assess: GitClear's analysis of millions of lines shows AI-generated code carries a 41% higher churn rate, meaning it gets rewritten or reversed at dramatically higher rates than human-written code, while meaningful refactoring dropped from 25% of changes to under 10% (GitClear, 2025). That's code that looks plausible on first read but doesn't survive contact with production.

The most striking finding is from CMU. They studied Cursor adoption across 807 open-source repositories: a 281% spike in lines of code in month one, collapsed back to baseline by month three. But the complexity it introduced — 30% more static analysis warnings, 41% higher code complexity — didn't revert (He et al., 2025). The velocity was self-defeating. More code, then more cleanup of that code, with lasting complexity baked in.

What makes verification particularly treacherous is that developers' own trust calibration is unreliable. The Stack Overflow 2025 survey found that stated trust in AI-generated code dropped from 40% to 29% year-over-year — while 80% of developers reported currently using AI tools (Stack Overflow, 2025). People are using it more and trusting it less, simultaneously. Sonar's data sharpens this into something harder to dismiss: 96% of developers say they don't fully trust AI code, but only 48% report always verifying it (Sonar, 2026). That gap — between knowing something needs checking and actually checking it — is the verification burden in its purest form. It's not a knowledge problem. Developers aren't naive about the risks. It's a capacity problem: review is cognitively expensive, the volume of generated code is high, and the path of least resistance is to skim, approve, and move on.

Tests as Specification

Testing is the first layer of defense — and not just as a safety net. A solid test suite makes the LLMs themselves more accurate. This isn't a hunch: a Microsoft Research study found that using validated tests to constrain the code generation space improved pass rates by 38% (Fakhoury et al., 2024). Tests narrow the space of acceptable implementations, so fixes and new features converge on correct solutions faster. The tests act as a specification the agent can check against, and the first iteration of that feedback loop captures most of the value (Olausson et al., 2024).

What I've found is that it's sometimes more important to review the tests than the implementation. LLM code tends to look right, especially for narrow changes. You read through a diff and think "yeah, that seems reasonable" because it's clean and coherent. But the tests reveal what wasn't considered. A concrete example: I was reviewing a change to an event consumer and noticed the test suite didn't cover a specific entity status — a state where, in theory, the entity would never receive that event. Real-world software is never that kind. I had Claude add a test for that combination, and the code failed miserably: inconsistent database state, no trace in logs. It would have been a nightmare to troubleshoot in production.

There's a paradox I'm sitting with. I'm getting better at reviewing AI code — I've developed two distinct modes, ruthless with AI output and socially calibrated with human colleagues. But I'm also starting to trust it more. Those two trends could cancel each other out — or worse, the trust could outpace the skill.

The answer isn't more discipline. It's better systems — automated checks, layered review, structured verification — that don't depend on me being sharp every single day. I haven't solved this. It's an open problem.

What I Don't Know

The METR developers thought they were faster. They weren't — they were 19% slower, while predicting 24% gains. I think I'm faster. But I have no controlled experiment, no baseline, and the strongest evidence says my own perception is the least reliable measure available. The same research that supports my workflow design also suggests I might be wrong about its results.

The pull to fix things by hand is fading. Early on, I'd constantly think "I could solve this in thirty seconds." Lately, I don't feel that pull. That might be growth — I've moved to a different level of abstraction, and the instinct isn't gone, just redirected. Or it might be erosion. The detection mechanism for skill atrophy is the skill itself, which means I'd be the last person to notice.

And underneath everything: the bias-confirmation problem. The AI builds on my assumptions. I caught a fundamental flaw in one design only because something felt off — not because the AI flagged it. My judgment was the detection mechanism, which is exactly the thing that might degrade over time. I'm operating within the limits of what occurs to me to question, and I have no way to see outside that boundary.

Conclusion

I'm moving fast, learning faster, and certain of very little. The tensions I've described — between speed and verification, between skill and dependency, between confidence and evidence — aren't problems to solve. They're the permanent condition of working with tools that are powerful enough to restructure your process and unreliable enough to require constant judgment.

That uncertainty isn't a weakness in the argument. It's the argument. The only thing I'm reasonably confident about is that the developers who treat this as a process design problem — not a tool adoption problem — will be the ones who come out ahead. Everything else, I'm still figuring out.