AI-Generated Code Ships Fast. So Do Its Bugs.

I use Copilot and Claude every day, and I'm not going to pretend otherwise. They've genuinely made me faster. But last month one of them almost got me shipped into trouble, and the more I've paid attention since, the more I think the gap between "the code works" and "the code is safe" is the single most under-discussed problem with these tools.

Here's what happened.

The auth flow that compiled

I asked Claude to build a small JWT login flow — the kind of thing I'd normally write myself but wanted quickly for a side project. It returned clean, well-formatted code on the first try. Tests passed. The token round-tripped. Great.

Then I actually read it.

• The JWT secret was a constant at the top of the file, right above the function that uses it.
• It used HS256 without a single comment about when you'd want RS256 or asymmetric keys.
• Token expiry was checked, but sloppily — a malformed exp claim would sail straight through.
• The login endpoint had no rate limiting, so anyone could brute-force it in the background while I was busy admiring how tidy the code looked.

Was it broken? No. It did exactly what I asked. That's the trap. I asked for "a login endpoint," and the model gave me a login endpoint the way most tutorials write them — because that's what most tutorials in its training data do. Working and secure are different bars, and the model optimizes for the first one.

Why it keeps happening

This isn't malice or stupidity. It's the data.

The average open-source codebase is not a paragon of security practice. Stack Overflow answers optimize for "here's how to make it run," not "here's how to make it run without handing an attacker your session store." An LLM trained on that averages it out: it reproduces the common patterns, and the common patterns are full of cors({ origin: '*' }) and secrets sitting in source files.

Three specific things I've watched models do repeatedly:

They pick the most flexible option, not the safest one. Ask for a "complex query" and a helper that should use Prisma's typed API will quietly use $queryRaw with user input interpolated straight in, because raw SQL is the flexible option.

They treat placeholder values and real-looking values the same. I've lost count of the times generated code had your-api-key-here or an actual-looking sk_live_... sitting where a runtime secret should be loaded from. Both end up in diffs that someone might commit without thinking.

They mirror the security level of whatever example they're closest to. That's why you get dangerouslySetInnerHTML with user input, eval() on external data, and debug ports left open. The model doesn't know your threat model, so it defaults to none.

What that means for everyone shipping this stuff

Here's the part that actually worries me. These tools are lowering the barrier to shipping software, which is mostly great. But they're also lowering the barrier to shipping insecure software.

Five years ago, a beginner building their first auth system would spend a week reading docs and tutorials. That friction was annoying, but it was how people absorbed security context — why you salt hashes, what a timing attack is, why your error messages shouldn't tell attackers whether an email exists. Now the code just appears. It works. And the natural instinct is to trust it, because a machine that sounds confident wrote it.

Nobody has built the "review the AI's work" muscle into the default workflow yet. That's the gap.

What I actually do now

I still use these tools constantly. But I changed my habits after that login endpoint:

• Every AI output gets treated like a pull request from someone who means well but hasn't seen my codebase. I read it line by line, and I specifically hunt for the things models skip: input validation, rate limiting, error messages that leak internals, missing security headers.
• I prompt with the security constraints up front. "Write a login endpoint that uses bcrypt with a work factor of 12, rate-limits attempts, and returns the same error whether the email exists or not." That changes the output more than people expect.
• I run scanners on AI-written code. Semgrep and the ESLint security plugins catch a surprising amount of what the model misses. If the AI wrote it, the scanner reviews it — that's now just my rule.
• I keep a personal list of what LLMs get wrong. Auth flows, CORS, sanitization, error messages, CSP. Every time I review generated code I check that list first.

None of this is dramatic. It's just treating AI output like what it is: a very fast junior developer with no security instincts and infinite confidence. Review it like one.

The tools will get better — they already are. But right now, in 2026, the security responsibility still lands entirely on the person who hits merge. Skip the review step and you're not moving fast. You're just accumulating security debt at machine speed, and the bill always comes due later.