Skill Is a Dependency

Sometimes I catch myself handing a problem to the agent before I’ve thought about it at all. Nothing breaks when I do, which is exactly why it’s easy to keep doing.

That made me want to know whether AI is actually making engineers worse. The short answer is not across the board. But the research suggests it can weaken debugging and comprehension, especially when you’re learning something new, and we’re poor judges of whether it’s helping.

What the research says

Anthropic ran a randomized trial where developers learned an unfamiliar Python library, some with AI help and some without. The AI group scored about 17% lower on a quiz afterward, and the biggest gap was in debugging (paper). How people used the AI mattered a lot. Those who stayed engaged scored 65–86%. Those who handed everything over scored 24–39%.

METR followed experienced open-source developers on real tasks, reported here. They took 19% longer with AI. They had expected to be 24% faster, and afterward they still believed AI had sped them up.

Microsoft Research and Carnegie Mellon surveyed 319 knowledge workers. The more people trusted AI, the less critical thinking they said they applied to its output. The more they trusted themselves, the more they applied.

None of this proves that AI erodes skill over time:

• The Anthropic study measured learning a new library, not losing an existing skill, and it tested right after the tasks. It also comes from an AI vendor.
• METR had 16 developers using early-2025 tools. A separate trial of 96 Google engineers found AI cut time on a complex task by about 21%.
• The Microsoft survey is self-reported.

It points in a direction, and that’s all.

Why this is a reliability problem

In Ship Fast. Recover Faster. I argued that faster code generation doesn’t reduce production uncertainty. It moves the pressure onto review, and review depends on the engineer’s judgment. If that judgment weakens, the review still happens. It just catches less.

That’s the same shape as Synthetic Resilience: something that looks like safety but behaves wrongly when it matters.

What it looks like

This is a hypothetical, but every piece of it is a real failure mode.

A queue consumer falls behind at evening peak. Errors are flat, but p99 processing time has doubled. You paste the logs into the agent. It suggests raising the timeout, adding retries with backoff, and scaling out the consumer group. The diff is clean and the tests pass.

What the agent missed: connection pool wait time started climbing twenty minutes before the first error, and the lag sits on two partitions. More retries add load to a pool that’s already full. More consumers trigger a rebalance, and work that hadn’t been committed gets processed again, as covered in Ship Fast. Recover Faster.. That fix passes review and turns a slow evening into an outage.

The agent read the stack trace correctly. It didn’t understand the system, and you only catch that if you already have a picture of the system in your head.

The skills I’d protect

I don’t miss syntax recall or boilerplate. The skills worth keeping are the ones you only use when something is broken:

• Reading code you didn’t write. Agents write most of it now, and you still own it.
• Forming a guess before you ask. Know what you think the failure is first.
• Reading raw logs and traces before reading any summary of them.
• Knowing how the system fits together: dependencies, state, and failure modes.

You use these rarely, so average productivity numbers won’t show them fading.

Habits that help

Pilots practice manual landings because autopilot works almost all the time. Same idea here:

• Write the first draft of risky code yourself, especially anything touching idempotency, schemas, or offset commits.
• Write down your hypothesis before you prompt.
• Don’t accept a diff you can’t explain.
• Debug one incident a week without the agent, then use it to check your answer.
• When you’re learning a new system, start by hand. The Anthropic result was about exactly this.

I’m not suggesting you use agents less. The point is to keep the skill that lets you tell when they’re wrong.

Most of the time the agent is right and review is a formality. The week it’s wrong in a way that looks right, you’re the last check before production.