The Missing Layer
Thinking, Fast and Slow — Explaining Healthcare’s AI Adoption Problem
Automation with AI was never the hard part. Deciding who gets to decide — human, AI, or some negotiated combination of the two — is. That allocation of decision rights will become a primary determinant of organizational performance.
The Real Question
In speaking with health system executives, the common question I get is “how do we automate decisions?” The right question is “which decisions should be made by humans, which by AI, and which by human-AI teams?”
A growing and, and I would argue, an underappreciated line of organizational research — Puranam’s (2021) work on human-AI collaborative decision-making chief among it — suggests AI’s most consequential organizational impact may not be task automation at all, but the redesign of decision architecture along three dimensions: who makes decisions and how decision rights are allocated, how quickly decisions are made, and how decision quality is assured.
Adoption of AI in healthcare has lagged other sectors — a 2025 JAMA Health Forum analysis found healthcare AI use running at roughly a third the rate of information services and half the rate of financial services — and the gap is not primarily a technology problem; it is a decision-making, workforce, and operating-model transformation problem. The greatest constraint to value realization is not model capability, but a health system’s willingness to delegate decisions and work to AI in a safe, governed, and measurable way. Errors here have asymmetric consequences: clinical harm, regulatory exposure, reimbursement loss, malpractice liability, reputational damage, and erosion of patient confidence.
The allocation of decision rights between humans and AI will become a primary determinant of organizational performance — just as organizational structure and hierarchy were in previous eras.
The Missing Layer: Decision Process Architecture
Daniel Kahneman’s Thinking, Fast and Slow remains one of the most useful mental models available to anyone trying to understand how organizations actually make decisions. Kahneman, a Nobel laureate in behavioral economics, argued that the mind runs on two systems. System 1 is fast, automatic, intuitive, and largely unconscious — the machinery behind recognizing a face or steering down an empty road. System 2 is slow, deliberate, and effortful — the machinery we reach for when a problem is unfamiliar or the stakes are high.
The uncomfortable insight in the book isn’t that System 1 exists. It’s that System 1 runs the show far more often than we’d like to admit. It constantly generates impressions and snap judgments, and System 2 — lazy by design, always looking to conserve effort — tends to rubber-stamp them rather than interrogate them. We aren’t irrational because we’re unintelligent. We’re irrational because we’re cognitive misers: we lean on heuristics that work well on average and fail in predictable, specific ways.
Now overlay that framework on the modern healthcare enterprise. AI is rapidly automating what amounts to System 1 work — gathering data, synthesizing knowledge, generating options, surfacing insight (e.g. claims edits, appointment confirmations etc.). The analytical bottleneck that used to slow everything down is dissolving. What’s left, and what’s now exposed as the real constraint, is System 2: the deliberate, judgment-heavy work of actually deciding (e.g. clinical diagnosis, complex scheduling etc.). As the machinery for fast thinking gets faster, the natural question becomes what happens to the slow thinking that has to sit on top of it — and what tools and methods are available to strengthen it.
Kahneman’s own research is a caution against assuming expertise solves this. He found that even trained statisticians are poor intuitive statisticians — prone to overestimating what a small sample can tell them, prone to ignoring base rates, because the brain evolved to construct stories, not compute probabilities. Knowing about a bias doesn’t make you immune to it. What actually helps is designing the environment around the decision: checklists, pre-mortems, structured decision processes that force System 2 to engage before a call gets made — whether the context is scheduling a patient visit or setting institutional policy. It should be noted that the heuristics-and-biases research by Kahneman has serious critics (Gigerenzer and the “ecological rationality” school argue heuristics are often adaptive, not error-prone). This does not change the real work to be done. Governance features such as escalation criteria, audit frameworks, acceptance/override rates, trust telemetry etc. need to be addressed in mapping the full decision universe of a health system.
Mapping the Decision Universe — the New Hard Work
Labor shortages, cost pressures, and rising denial rates have made the question of “who should decide what” acutely operational. Mapping the full decision universe — decision rights, decision speed, decision quality, decision consequences — across an enterprise as complex as a hospital system is genuinely hard, in part because so much of the knowledge underneath these decisions is undocumented tribal knowledge. Denial and appeal logic in revenue cycle management, for instance, often lives in the heads of a handful of experienced staff rather than in any written standard operating procedure (SOP). Establishing the guardrails — decision rights, escalation paths, policy — isn’t a one-time exercise. It shifts across business process, time, organizational maturity, materiality, and risk.
Get Brian McCarthy’s stories in your inbox
Join Medium for free to get updates from this writer.
Figure 1 below is my own synthesis, mapping decision-rights and transaction-cost logic onto a two-axis frame — it is not a standard industry model, and any resemblance to other recent quadrant frameworks in this space reflects convergence on a fairly intuitive 2x2, not derivation. It plots decisions along two axes: work complexity — from routine transactions to higher stakes judgment-based work, and data complexity — from structured and stable to unstructured and volatile. Four zones fall out of that: routine, well-understood decisions (automate); high-stakes, novel decisions (human-led); and two harder zones in between.
Figure 1: Decision Mapping Framework for a Health System Revenue Cycle Domain
The bottom-left and top-right quadrants are largely self-evident — straightforward decisions to automate, and high-stakes decisions that will always be human-led. The interesting and difficult territory is the messy middle. This is where thousands of decisions get made every day, where the design choices about what AI owns, what a human owns, and what gets escalated actually matter — and where getting it wrong compounds fast, because AI doesn’t just automate a bad process, it accelerates it. Governing the boundary between these tiers requires escalation criteria, audit frameworks, and deliberate accounting for biases, delegation error, and explainability gaps. It also requires treating decision and autonomy rights as dynamic rather than fixed — they shift with complexity, stakes, and the quality of the data available at the moment of decision. This will be expanded upon in a subsequent article.
Agentic AI complicates this map further. The quadrant model above implicitly assumes a single decision, made once, by one actor. Agentic systems chain dozens of smaller decisions — retrieving a record, drafting an appeal letter, routing an escalation, scheduling a follow-up — into a single autonomous workflow, often without a discrete human checkpoint between steps. That changes the governance question from ‘should a human review this decision’ to ‘at which step in a multi-step chain does a human need to intervene, and how is that trigger point designed rather than discovered after a failure.’ Dynamic orchestration — where an AI system decides, in real time, whether to act autonomously, escalate, or hand off to another agent — is itself now a decision-architecture choice, not a technical implementation detail delegated to IT.
What Comes Next
None of this comes with an existing system of record. The rules governing how these decisions actually get made today are scattered across policy documents, SOPs, and tribal knowledge. Building that layer — defining the decisions themselves, the roles that own them (which are also evolving), decision rights, accountability, escalation paths, and workflow/agentic orchestration — is the trust infrastructure of the enterprise. It’s largely unaddressed by existing technology providers, and it’s the subject of the next piece in this series: why the most common safeguard against AI decision-making — keeping a human in the loop (HITL) — is far less effective than most organizations assume.
References
Kahneman, Daniel (2011). Thinking, Fast and Slow
Nguyen, T. D., Whaley, C. M., Simon, K., et al. (2025). Adoption of Artificial Intelligence in the Health Care Sector. JAMA Health Forum, 6(11), e255029. https://doi.org/10.1001/jamahealthforum.2025.5029
Puranam, P. (2021). Human–AI collaborative decision-making as an organization design problem. Journal of Organization Design, 10, 75–80. https://doi.org/10.1007/s41469-021-00095-2
Shrestha, Y. R., Ben-Menahem, S. M., & von Krogh, G. (2019). Organizational decision-making structures in the age of artificial intelligence. California Management Review, 61(4), 66–83. https://doi.org/10.1177/0008125619862257
Thanks for reading this article.
- Subscribe to my newsletter for free for more articles and posts related to AI transformation in healthcare, although much of this applies cross industry
- Let’s connect on LinkedIn
