There is a moment that is becoming familiar to executives across industries as organizations increasingly use AI to support decision making. A question that once required a multi-day analysis cycle now has a plausible, well-structured answer within seconds. The AI tool has synthesized the relevant data, organized the key considerations, and presented a recommendation with apparent confidence. The meeting moves on.\u00a0<\/p>\n\n\n\n
The question no one asked is whether the answer was right.<\/p>\n\n\n\n
This is not an argument against AI. The productivity gains from AI-assisted decision making & analysis are real, measurable, and accelerating. The tools are becoming more capable, more integrated, and more central to how organizations process information and generate recommendations. The concern is not the technology. The concern is what happens to organizational judgment when the challenge shifts from information access to answer evaluation. Most organizations have not yet built the capability to manage that shift.\u00a0<\/p>\n\n\n\n
For most of the modern organization’s history, the limiting factor in decision-making was information. Getting the right data, in the right form, to the right people, fast enough to act on it. Enormous organizational infrastructure was built around solving that problem: data warehouses (and even lakes), business intelligence platforms, analytics teams, reporting functions. The implicit assumption was that better information would produce better decisions.\u00a0<\/p>\n\n\n\n
Artificial Intelligence has largely solved the information access problem. A well-configured AI system can surface relevant data, identify patterns, generate summaries, and produce recommendations faster than any human analytical team. In many domains, it does this more accurately as well. The information problem, in other words, is increasingly solved.<\/p>\n\n\n\n
What AI has not solved is the judgment problem. It has, in fact, made it more acute. When information was scarce and slow, the limiting factor on bad decisions was partly the difficulty of generating a bad analysis fast enough to act on. When information is abundant and instant, the limiting factor on bad decisions is the quality of human judgment applied to AI outputs. That judgment, in most organizations, has not kept pace with the capability of the tools generating the outputs it is supposed to evaluate.<\/p>\n\n\n\n
\n\nAI has largely solved the information access problem in decision making. What it has made more acute is the judgment problem<\/p>\n<\/blockquote>\n<\/blockquote>\n\n\n\n
What does research say about AI and decision-making?<\/h2>\n\n\n\n
The most rigorous study of AI’s effect on the quality of knowledge workers’ decision making<\/a> to date was published by researchers from Harvard Business School, the Wharton School, and MIT in 2024. The study, which examined consultants at a major firm using AI assistance, found a striking pattern: AI tools significantly improved performance on tasks that fell within the AI’s capability boundary, but produced worse outcomes on tasks that fell outside it. Critically, workers using AI were less likely to recognize when they had crossed that boundary. The AI’s confident output suppressed the skepticism that would otherwise have prompted further investigation.\u00a0<\/p>\n\n\n\n
This finding, which the researchers called the ‘jagged frontier’ problem, has direct implications for executive decision-making. Artificial Intelligence systems are exceptionally capable within their effective domain and produce outputs that appear equally confident across their entire range. A human evaluator who cannot identify where the boundary lies will apply the same level of trust to both high-confidence and low-confidence AI outputs. The result is not that AI makes people worse at everything. It is that AI makes people selectively worse at the things they can no longer easily identify.\u00a0<\/p>\n\n\n\n
A complementary concern has been documented in research on automation bias<\/a>, the well-established tendency for humans to over-rely on automated or algorithmic recommendations, particularly under time pressure. Studies in aviation, medicine, and financial services consistently find that the presence of a system-generated recommendation reduces the probability that a human operator will independently evaluate the underlying evidence, even when the recommendation is incorrect. <\/p>\n\n\n\n
The compound effect of these two dynamics is significant. AI produces confident-sounding outputs across a range that includes both its strengths and its limitations. Humans under time pressure apply less independent scrutiny to system-generated recommendations. The result, in organizational decision-making contexts, is a systematic reduction in the quality of judgment applied to AI outputs at precisely the moments when better judgment is most needed.\u00a0<\/p>\n\n\n\n
\n\nThe presence of a system-generated recommendation reduces the probability that a human operator will independently evaluate the underlying evidence, even when the recommendation is wrong<\/p>\n<\/blockquote>\n<\/blockquote>\n\n\n\n
What are the 3 types of AI risk leaders should recognize?<\/h2>\n\n\n\n
Not all AI errors are alike, and understanding the different failure modes is the starting point for building better evaluation capability. Three categories are particularly relevant for executive decision making contexts.\u00a0<\/p>\n\n\n\n
Confident Fabrication <\/em>– AI generates confident but incorrect information<\/h3>\n\n\n\n
AI language models can generate plausible-sounding information (including citations, statistics, and precedents) that does not exist or is materially inaccurate. The output reads like reliable analysis. There is no internal signal, no hedging, no reduced confidence indicator, that distinguishes a fabricated fact from an accurate one. In high-stakes decisions where specific evidence is material to the conclusion, this failure mode can be consequential and difficult to detect without deliberate verification.<\/p>\n\n\n\n
Pattern extrapolation beyond valid range<\/em> – Why does AI struggle with new and unfamiliar situations?<\/h3>\n\n\n\n
AI systems learn from historical data and are effective at identifying patterns within that data. They are substantially less reliable when applied to unfamiliar conditions, structural breaks, or situations that differ from the training distribution in ways the system cannot detect. A recommendation generated from historical operational data may not account for a recent regulatory change, a new competitor dynamic, or a supply chain configuration that has no historical precedent. The AI does not know what it does not know, and its output will not reflect that limitation.<\/strong><\/p>\n\n\n\n
Framing lock<\/em> – What happens when AI Is asked the wrong question?<\/h3>\n\n\n\n
AI systems respond to the way a question is posed. A well-framed prompt produces a well-organized response to that framing. But if the framing itself is wrong, if the question being asked is not the right question, the AI will produce a thorough, well-organized answer to the wrong question. This is a more subtle failure mode than fabrication, and in many ways more dangerous: the analysis looks complete because it addresses the question that was asked, and there is no internal signal that the question itself was the problem.<\/p>\n\n\n\n
Why is critical thinking still important when using AI?<\/h2>\n\n\n\n
These failure modes share a common characteristic: they are not detectable from within the AI output itself. Identifying them requires the human evaluator to bring something to the interaction that the AI cannot supply: a structured framework for what a valid answer should look like, an understanding of the topic sufficient to recognize when an answer is implausible, and the discipline to apply independent scrutiny rather than accepting confident-looking output at face value.\u00a0<\/p>\n\n\n\n
This is exactly the capability that structured decision making frameworks develop. The discipline of clearly defining a problem before evaluating solutions, as described in Article 1<\/a> and Article 2<\/a> of this series, applies directly to AI output evaluation. An organization that cannot accurately diagnose the root cause of an operational failure without AI assistance is also an organization that cannot reliably evaluate whether an AI-generated diagnosis is correct.<\/p>\n\n\n\n
The same is true for AI-assisted decision-making<\/strong>. An organization that has not built the internal discipline to clarify objectives, evaluate alternatives systematically, surface assumptions, and examine tradeoffs will not suddenly acquire those capabilities by routing their decision-making through an AI tool. The AI may organize the information more efficiently. But the quality of the judgment applied to that information is determined by the human capability in the room, not by the capability of the tool generating the output.<\/p>\n\n\n\n
This is the central argument for the human edge in an AI world. The organizations that will consistently make better decisions are not the ones with the most sophisticated AI stack. They are the ones that combine capable tools with the structured human judgment and critical thinking to use those tools well. This involves knowing what questions to ask, how to recognize when an answer is incomplete or wrong, and how to apply the kind of independent critical thinking that AI cannot replicate.<\/p>\n\n\n\n
\n\nThe organizations that will consistently make better decisions are not the ones with the most sophisticated AI stack. They are the ones that combine capable tools with the structured human judgment to use those tools well<\/p>\n<\/blockquote>\n<\/blockquote>\n\n\n\n
How Can Organizations Prepare Their Workforce for AI Adoption?<\/h2>\n\n\n\n
The practical implication for executive teams is not to slow down AI adoption. The productivity and analytical benefits are too significant, and the competitive pressure to adopt is real. The implication is to invest in parallel: in the AI tools and in the human judgment capability required to use them well.<\/p>\n\n\n\n
A few specific practices that high-performing organizations are building into their AI-enabled workflows:\u00a0<\/p>\n\n\n\n
\n
- Pre-answer framing discipline<\/strong>: Before querying an AI system for analysis or recommendations on a significant decision, the team explicitly defines what a good answer would need to include, what assumptions it would need to validate, and what alternative conclusions should be considered. This is not about slowing down the AI query. It is about ensuring the humans evaluating the output have a structured basis for doing so.\u00a0<\/li>\n\n\n\n
- Assumption surfacing: <\/strong>For any significant AI-generated decision recommendation, the team identifies the key assumptions embedded in the output and evaluates each independently. What does this recommendation assume about market conditions, competitor behavior, internal capability, or regulatory environment? Which of those assumptions is most consequential if wrong?\u00a0<\/li>\n\n\n\n
- Boundary testing: <\/strong>When an AI output is used to support a high-stakes decision, the team explicitly asks: is this the kind of question where AI tools are likely to be reliable? Does this situation have historical precedent in the data the system was trained on? Are there structural features of this situation that might place it outside the AI’s effective range?\u00a0<\/li>\n\n\n\n
- Independent verification of material facts: <\/strong>Any specific statistic, precedent, or evidence that is material to a significant decision gets independently verified before the decision is finalized. This is not a reflection of distrust of the tool. It is a recognition that confident output and accurate output are not the same thing.\u00a0<\/li>\n<\/ul>\n\n\n\n
None of these practices require abandoning AI tools or slowing down workflows. They require building the structured thinking discipline that allows AI outputs to be evaluated rigorously rather than accepted reflexively. Organizations that build this discipline gain a compounding advantage: not only do they make better use of their AI investments, they also develop the organizational judgment that becomes harder to replicate as AI tools themselves become more commoditized.<\/p>\n\n\n\n
Why an effective AI implementation strategy requires more than technology <\/h2>\n\n\n\n
There is a version of the AI adoption story that is purely about tool capability: whoever has the most advanced tools wins. This version is compelling in the short term and almost certainly wrong in the medium term. AI tools are becoming more capable and more accessible simultaneously. The productivity advantages of any specific tool configuration erode as that configuration becomes available to everyone. <\/p>\n\n\n\n
The more durable competitive advantage is the organizational capability to use these tools well. <\/strong>That capability includes the technical infrastructure to deploy and maintain AI systems effectively. It also includes the human judgment infrastructure to evaluate AI outputs critically, to recognize the boundaries of AI reliability, and to apply structured thinking to the decisions that AI analysis is informing.\u00a0<\/p>\n\n\n\n
That second capability does not come with the software license. It is built through deliberate investment in critical thinking and decision making frameworks, diagnostic discipline, and the cultural expectation that confident-looking output, from any source, deserves scrutiny proportional to the stakes of the decision it is informing.\u00a0<\/p>\n\n\n\n
AI gives you answers fast. The question of whether they are the right ones is still, for now, a human responsibility. And organizations that take that responsibility seriously will consistently outperform those that do not.<\/p>\n\n\n\n
<\/div><\/div><\/div><\/div>\n\n\n\nAbout Kepner-Tregoe<\/h2>\n\n\n\n
For more than sixty years, Kepner-Tregoe has helped organizations solve problems, make decisions, manage risk, and build cultures of critical thinking. KT\u2019s structured methodologies – Situation Appraisal, Problem Analysis, Decision Analysis, Potential Problem Analysis, and Potential Opportunity Analysis – provide the thinking infrastructure that allows leaders and teams to perform at their best when it matters most.<\/p>\n\n\n\n
<\/div><\/div><\/div><\/div>\n\n\n\nSubscribe<\/h2>\n\n\n\n
Join our mailing list to stay up to date with our latest insights, blogs and articles, podcasts, webinars and events, upcoming class dates and more.<\/p>\n\n\n\n
\n\n<\/p>
<\/ul><\/div>\n