I spent a month deliberately trying every AI coding tool that crossed my path — editors, agents, review bots, doc generators, test writers. Twenty of them. Four are still installed, and the reasons the other sixteen failed are more useful than the four that worked.

What the demos don't show

Every demo starts with a greenfield project and ends with a working feature. Real work starts with a five-year-old codebase, inconsistent conventions, a test suite that needs a specific database seed, and a ticket that says “make the export faster.” A tool that can't read the surrounding code produces plausible code that doesn't fit, and you spend longer deleting than you would have spent writing.

The four that survived

One, an in-editor completion model that respects the current file's style closely enough that I stopped reading its diffs character by character. Two, a terminal agent for mechanical work: renaming, dependency bumps, writing the boring test cases I'd otherwise skip. Three, a summarizer that reads a large file and answers “where is this value actually set” — it replaced ten minutes of grep. Four, a plain-English shell helper that turns a half-remembered ffmpeg incantation into a correct one.

Notice what those have in common: none of them own the architecture. They're all local, mechanical, and verifiable in seconds.

Where AI still loses

Anything requiring a mental model of a system it can't hold. Cross-file refactors where the correct change depends on why the code looks weird. Performance work, where the bottleneck is never where the model guesses. And anything involving a real external system with state, because the model will confidently invent the API.

The failure mode that costs the most is not bad code, it's plausible confident code. You review it less carefully because it looks right, and the bug lands in production with your name on the commit.

How to evaluate any AI tool in a week

Don't try it on a toy. Give it the ugliest real ticket on your board and watch three things: can it find the relevant files itself, does its output match your conventions without a lecture, and how long does it take you to verify the result. If verification takes longer than writing, it's not a tool, it's a liability.

The honest conclusion

AI made me faster at the mechanical 30 percent of the job and no faster at the architectural 70 percent. That's still worth money. But the industry sales pitch is that it replaces the 70 percent, and a month of real work says otherwise.