AI can produce answers instantly, but what happens to human judgment? New research explores how AI use may affect critical thinking, reasoning, and skill development.
Artificial Intelligence is moving faster than anyone predicted. Every week brings another announcement, another model, another story about how productivity has doubled somewhere. It is tempting to believe we have finally found a lever that removes friction from thinking itself. A recent research paper from Anthropic suggests we should think twice before believing that.
AI feels faster – but does it make us better?
In a controlled experiment developers completed identical coding tasks using a library none of them had worked with before. Half had access to an AI assistant, and the other half worked without one. Once the task was done, both groups were tested on conceptual understanding, code reading, and debugging. No AI usage was allowed during the test.
The results were striking. The developers who relied most heavily on AI scored lower across the board, with the biggest drop showing up in debugging. The group without AI made more mistakes along the way and took a bit longer, but those mistakes turned out to be the point.
Struggling with errors forced them to actually understand what they were building. AI made the task feel easier, but it did not produce a meaningful speed advantage overall. The most important takeaway is that the people who relied on it heavily understood less when the work was over.
How people actually use AI
The researchers identified six patterns of AI interaction. At one extreme, pure delegation: ask the model, paste the answer, move on. That group scored in the thirties. At the other extreme, a pattern they called generation then comprehension: the developer let AI produce something, then interrogated it, questioned it, and worked to understand why it functioned. That group scored in the eighties. The difference had nothing to do with access to technology. It had everything to do with whether the human stayed in the driver’s seat.
I have been in consulting long enough to recognize this pattern immediately. Right now, a lot of leaders are trying to figure out what to do with AI. They want to give their teams guidance on when to use it, how to use it, and how to make sure it does not become the crutch that eventually leads to their people forgetting how to walk. That is not a hypothetical concern. The Anthropic study shows it happening in real time, in a controlled setting, with experienced professionals.
This isn’t a new problem
When I read this study, I was struck by how familiar it felt. The terminology might be new, but the principle is not. For more than six decades, Kepner Tregoe has operated on a simple conviction: rational thinking is a discipline, not an instinct. It has to be practiced deliberately and applied consistently. We build capability through cognitive effort, especially in ambiguity and error. If AI removes that effort, skill formation erodes. If AI amplifies structured reasoning, skill formation can accelerate. The difference is not the tool; it is the process discipline around the tool. That is the kind of discipline KT has been helping clients build for decades.
In fact, the study’s findings line up naturally with KT’s frameworks. Situation Appraisal requires us to separate and clarify all concerns before acting. Many organizations fixate on one question: does AI accelerate task completion? Two other critical questions which people often regard as secondary: does AI inhibit skill formation, and are people retaining the capability to supervise AI output responsibly? If leaders focus mainly on speed, they may improve throughput while weakening the judgment and diagnostic skills the business depends on. If we let AI drive the process instead of support it, the diagnostic skills our operations depend on will slowly weaken.
Where the real skill gap shows up
The debugging findings are especially worth reflecting on. The largest gap between the two groups showed up in diagnosing and correcting errors. “Debugging” is essentially structured reasoning under uncertainty. It is the practical expression of KT Problem Analysis, distinguishing what is happening from what is not, examining deviations across what, where, when, and extent.
That discipline is built through repetition. When AI resolves errors instantly, the repetition disappears. The LLM produces an answer, the human accepts it, and their thinking muscle weakens.
Before deciding how to integrate AI into workflows, leaders owe it to their organizations to conduct a disciplined KT Decision Analysis. There are objectives that must be met, tradeoffs that must be understood, and risks that must be anticipated.
KT’s Potential Problem Analysis asks not only what could fail, but why it would fail and how you would detect early warning signs. The gradual erosion of diagnostic expertise through unreflective AI adoption is exactly the kind of slow-moving risk that never triggers an alarm until it is too late.
What high performers do differently
Of course there is an opportunity here too. The highest-performing participants in the study did not avoid AI. They used it as a thinking partner, generating output and then examining it, questioning it, testing assumptions. They stayed mentally engaged and kept doing the thinking. That is what Potential Opportunity Analysis cultivates: disciplined human judgment amplified rather than replaced.
The organizations that will do well in an AI-enabled economy will not necessarily be the ones that adopt new tools the fastest. They will be the ones that embed those tools within a culture of disciplined thinking, one that insists AI accelerates work without replacing judgment, and that protects the cognitive capabilities that good supervision requires.
The question worth asking is whether your organization has a structured framework for how to adopt AI, or whether it is simply happening. For over six decades, Kepner Tregoe has helped organizations think with rigor, particularly under pressure. The arrival of AI does not make that mission obsolete. It makes it urgent.
If this topic matters to you as a leader, the Anthropic paper How AI Impacts Skill Formation, published on 3 February 2026, is well worth your time.
https://arxiv.org/pdf/2601.20245
https://www.anthropic.com/research/AI-assistance-coding-skills
