Sitemap
Rob Gifford

Rob Gifford

Principal Designer @ athenahealth. Product + UX nerd

The Verification Tax

How to design AI products that earn trust and gain adoption

12 min read3 days ago

--

AI robot holding paper that a person is inspecting.

Most of us got into technology to make valuable tools that improve how people live and work. That value can be measured by one simple thing: Adoption. This sounds abstract, but adoption can be boiled down to a simple formula.

Users adopt a product when its value outweighs the cost of using it.

Press enter or click to view image in full size
Likelihood of Adoptions is a ratio of value over the cost of using it

For AI products, particularly in an enterprise setting, value usually comes from the work we’re taking off users’ plates. In the domain of healthcare specifically (where I work), that’s less time summarizing information, reading content, finding patterns in data or completing routine tasks.

Most conversations about AI and adoption tend to focus on increasing model or system performance. The assumption is that if we increase performance, more value will be provided and adoption will grow. This isn’t wrong, but its only one side of the equation.

There reality is that we’re in a period of transition. Users accustomed to deterministic, predictable tools are beginning to adopt powerful AI systems that can feel like a black box and produce unexpected results. The cost of using these tools includes not only the effort required to learn and interact with them, but also risk and uncertainty.

To create a confidence-inspiring experience with AI that leads to greater adoption, we need to provide users with the ability to pierce that “black box” and detect potential errors. For example, identifying if AI drew the right conclusions from a set of documents. Or AI interpreted and executed a complex task correctly.

We can think about the time and attention it takes for users to gain this confidence as a “Verification Tax”- something that needs to be accounted for, but minimized.

Quick note: while most of the following examples are healthcare related, where my work is focused, the same principles can be applied to other high-stakes domains like finance, law, or other enterprise settings.

Part 1: The Verification Tax

But why is the verification tax necessary and how does it relate to adoption?

Let’s go back to our Adoption equation. For users, the costs of adopting an AI product or feature boil down to two things:

  1. Effort: the time and mental energy required to use it and verify the output
  2. Risk: the potential consequences if an error slips through

So we can flesh out the equation like this:

Press enter or click to view image in full size
Likelihood of Adoption = Value / Cost (Effort Required + Risk)
  • Value = Work removed
  • Risk = Negative consequences if things go wrong

Verification plays two important and offsetting roles in this equation. It adds immediate effort, but it also lowers risk by decreasing the chance that an error will go unnoticed.

Press enter or click to view image in full size

The situation determines the verification burden

But not all errors are the same- the amount of confidence needed to act on an output varies greatly depending on the stakes of the scenario.

To illustrate the point, take two technically similar features:

A recommendation engine that suggests a television show and a clinical tool that suggests a possible diagnosis. Both use patterns in historical data to propose a recommendation that a user can accept or reject. But the risk involved in either scenario is wildly different. Adopting a poor television recommendation may cost the viewer a little bit of time while a poor clinical recommendation can lead to unnecessary treatment and delay in finding the correct care.

When we boil it down, two factors drive a greater need for verification or confidence in AI:

  1. the harm an undetected error could cause
  2. how difficult it is to detect an error

Low-consequence output that is easy to inspect may need little verification support. High-consequence output that is difficult to inspect benefits more from features that help users verify or develop confidence.

The following chart illustrates the dynamic:

Press enter or click to view image in full size
As the potential consequences of an AI error increase, the need for verification rises; a television recommendation is compared with a possible clinical diagnosis.

Verification supports appropriate trust

Our goal shouldn’t simply be to encourage users to develop blanket confidence in our AI features. Instead, we should shoot for appropriate reliance or “calibrated trust.” Users should rely on a system when its capability fits the task it is being as to perform. Too little trust wastes a useful automation. Too much trust allows errors to pass unnoticed.

A user’s level of trust also changes through experience. Before first use, people rely on signals such as reputation, validation, peer endorsement. After use begins, each success or failure they experience changes what they expect from the system. For example, a feature that works well nineteen times and fails badly on the twentieth can lose far more trust than it gained through the earlier successes. As the saying goes,

“Trust is won in drips and lost in buckets.”

Helping users identify errors is the right thing to do but it also promotes sustained usage of our features.

Verification is a (Necessary) tax

But the challenge is that verification consumes the same resources AI is supposed to save: a user’s time and mental effort.

If checking and correcting AI’s work takes nearly as long as doing the task manually, no value is created by an AI feature.

Let’s take an everyday task I ran into recently: using an LLM to order groceries. The agent found products and added them to my cart, but repeatedly interrupted me for approvals, requiring me to monitor its actions. When it was done, I then had to check quantities and substitutions in more detail in my cart. These review activities canceled out any potential time savings I might have gain from having AI order my groceries.

Press enter or click to view image in full size

The same tradeoff matters in clinical software. An AI chart summary may take seconds to generate, but a clinician still needs to assess whether it accurately represents the patient’s history. If conducting that assessment requires reopening each source note to confirm medications, dates, and diagnoses or check for omissions, the promised time saving disappears.

The central product problem is: Users need to be able to catch meaningful errors with significantly less effort than the work the AI removes.

Part 2: Designing Better Verification

So how do we help users build calibrated trust without driving up the verification tax? The answer is intentional, streamlined transparency. Transparency makes the sources, reasoning, and limitations behind AI’s output visible, allowing users to assess whether to rely on it or not. In other words, it lets users see inside the black box.

But effective transparency requires care in what we show and how we show it. Too much detail, or explanations that are hard to follow, can overwhelm users and lead them to tune out altogether.

The goal isn’t to show everything. It’s to show the right information, in the right form, at the moment users need it. Otherwise, transparency becomes noise. Instead of making verification more efficient, it adds effort and erodes the time savings AI is supposed to deliver.

To decide what information to show, it helps to anchor on how AI does its work. Most AI features move through three stages:

  1. the information AI uses
  2. the steps it carries out
  3. the results it produces

Errors and misinterpretation can creep in at any of these stages. So transparency should help users answer the following questions:

  • Did it use the right information?
  • Did it take an appropriate approach?
  • What uncertainty or limitations should affect how we use the result?

Transparency works when it helps users pinpoint where in this process something may have gone wrong, so they can focus their effort there instead of rechecking everything.

Press enter or click to view image in full size
Three AI workflow stages connect inputs to source transparency, process steps to process transparency, and outputs to confidence transparency.

1. Source transparency: Show where it came from

Everything the AI produces is downstream of the information it starts with. When AI summarizes documents, synthesizes research, or makes recommendations from a body of data, users need a direct path back to the underlying evidence. That path helps them answer the first question from our framework: “did the system use the right information in the first place?”. In practice, that means asking:

  • Did the AI select the right sources for the task?
  • If the platform feeds data to the AI, was that data structured correctly?
  • Is anything missing from what the AI received?

To answer these questions, users need to see which files the AI considered, the dates they cover, and any relevant information it couldn’t access.

Get Rob Gifford’s stories in your inbox

Join Medium for free to get updates from this writer.

A doctor I interviewed about an AI document summary described the behavior clearly:

“The AI is only as good as what’s been fed into it so I’m going to double-check the source until I have a better feel for how the system behaves.”

Put bluntly: “Garbage in. Garbage out.”

Designing Source Transparency

If users like this want to check sources, our job should be to make that check fast. The first step is placing citations directly alongside the claims they support. That lets users focus their verification on the statements that matter most or look unexpected, rather than reviewing everything.

Following the same principle, we can go a step further and highlight the exact passage in the source that the AI drew from, so users can pinpoint what to check without scanning the whole text.

Finally, moving between the output and its sources should feel effortless. Users need to jump back and forth quickly without losing their place in either.

Here’s how that might look:

Press enter or click to view image in full size
Animated clinical summary showing a contextual citation opening its source alongside the summary and highlighting the supporting passage.

Let’s take a real-world healthcare example.

OpenEvidence, an AI-powered clinical information and search tool, provides a great example of source transparency. Because OpenEvidence is used at the point of care to make clinical decisions, its adoption depends on earning clinicians’ trust and making supporting evidence quick and easy to evaluate.

OpenEvidence goes beyond contextual citations by giving users an “EvidenceGrade” that describes the quality of research support for each of its answers.

Press enter or click to view image in full size
OpenEvidence answer with an EvidenceGrade panel explaining the strength of evidence supporting the answer.

2. Process transparency: Show the steps

For complex tasks or agentic work, verification experiences need to go beyond visibility into sources. Users need information about the steps AI takes and the logic it used to produce its output.

This helps users identify whether the AI interpreted the task correctly, took the right approach and navigated the relevant information properly.

Complex AI tasks are like navigating a route. If the AI misunderstands the destination or takes a wrong turn, the steps that follow can carry it farther off course. Showing the planned route and progress along it helps users recognize when a correction is needed.

Press enter or click to view image in full size
Two maps show how choosing the wrong destination or taking a wrong turn can send an AI task off course.

Process transparency can support users at three moments: before the AI starts, while it works, and after it finishes. For lengthy or consequential tasks, we can show the plan the AI intends to follow and let users review and approve it before any work begins. Once users sign off on the plan, they can let the AI run on its own, without the constant check-ins that slowed down my grocery order in my prior example in this article.

Putting this all together can look like this:

  1. Before the AI takes action: show the plan and ask for approval
  2. While the task is in progress: allow users to monitor actions
  3. After the work is done: allow users to trace the steps it took
Press enter or click to view image in full size
Animated scheduling assistant showing a proposed appointment-rescheduling plan, review and approval, and progress through the task.

Again, it’s important to consider the potential for information overload here. Reviewing more information and taking more steps can slow users down and increase the verification tax.

Here are four questions that can help you prioritize what information to show:

  1. What meaningful mistake or misunderstanding does the user need to catch? Name a specific failure, such as misunderstanding the requested task, choosing the wrong comparison, or skipping a required step.
  2. What judgment or action will this information support? Be concrete: confirming the plan, redirecting the AI, stopping an action, or deciding to review the result more closely.
  3. Would removing this detail make that judgment materially harder? If the user can make the same judgment without the information, consider hiding it by default or not showing it at all.
  4. When does the user need this information? Some information belongs before an action, some during execution, and some in an activity history. To avoid information overload, only show the information when it’s actionable.

3. Confidence transparency: Show where attention is needed

AI can present uncertain claims with the same confident tone as well-supported ones.

Let’s consider a metaphor… Imagine a machine that bakes thousands of cookies in minutes. Most come out perfectly, but occasionally it mistakes salt for sugar, creating extremely salty cookies that are visually identical to the others. The machine has made baking effortless, but someone still has to figure out which cookies are worth serving. If they don’t, customers will receive an unpleasant surprise.

Press enter or click to view image in full size
A cookie machine accidentally uses salt instead of sugar; identical-looking cookies emerge while a taste tester reacts to a salty cookie.

AI creates a similar problem. A well-supported claim and an uncertain guess arrive in the same polished, authoritative tone. Without guidance on which is which, users either need to verify everything, which is slow, or review nothing, which is risky. To help save users time and improve the odds of catching an error, we can provide confidence signals that direct users’ attention to the parts of an output that warrant checking. Confidence flags can also highlight assumptions, unsupported claims, and missing information that warrant closer review.

Here’s an example:

Press enter or click to view image in full size
A medication evaluation report embeds Highly supported, Lower support, and Assumption flags, with a tooltip explaining the lower-support finding.

A hospital could use an AI-generated report to evaluate a medication before adding it to its formulary. The report flags conclusions as Highly supported, Lower support, or Assumption, and briefly explains why.

A benefit consistently demonstrated in trials might be highly supported, while a comparison based on limited evidence would receive a lower-support flag. If the AI assumes those findings apply to older adults poorly represented in the trials, it makes that assumption explicit. These signals help clinicians and pharmacists see where closer review is needed before making a decision.

A study by Eli Gibson and his colleagues at Siemens Health illustrates the potential here. Gibson and his team developed an AI system to help doctors detect intracranial hemorrhage on head CT scans and included statistical confidence scores for its predictions. In a simulation, researchers used those scores to prioritize scans likely to show hemorrhage first, followed by uncertain cases and then scans confidently classified as negative. Gibson found that providing these confidence scores reduced report turnaround time by 25–27% compared to an interface without confidence scores.

Press enter or click to view image in full size
A fictional clinical worklist groups AI hemorrhage predictions by confidence, with likely positive cases first, uncertain cases next, and confident negative cases last.

Worklist visual generating for this article to illustrate concept. Patients are fictional.

Just like other elements of transparency, confidence indicators work best when they are sparse and actionable. Flagging every sentence or claim creates another layer to review. Instead, signals should be focused on directing users toward the areas that need review such as uncertain claims, weak evidence, or missing source coverage.

Summary

AI products save time on drafting, searching, analysis, and execution, but they also require users to spend time checking their work, especially in high-stakes settings like healthcare.

To gain safe adoption of AI products, we need to accommodate that need to verify while retaining as much of the potential time-saving value as possible. In other words, we can’t eliminate the verification tax, but we can design experiences that make it worth paying.

References

Gibson, E., et al. (2022). “Artificial Intelligence with Statistical Confidence Scores for Detection of Acute or Subacute Hemorrhage on Noncontrast CT Head Scans.” Radiology: Artificial Intelligence, 4(3), e210115. doi:10.1148/ryai.210115.

Hoff, K. A., and Bashir, M. (2015). “Trust in Automation: Integrating Empirical Evidence on Factors That Influence Trust.” Human Factors, 57(3), 407–434. doi:10.1177/0018720814547570.

Lee, J. D., and See, K. A. (2004). “Trust in Automation: Designing for Appropriate Reliance.” Human Factors, 46(1), 50–80. doi:10.1518/hfes.46.1.50_30392.

Rader, E., Cotter, K., and Cho, J. (2018). “Explanations as Mechanisms for Supporting Algorithmic Transparency.” Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, Paper 103. doi:10.1145/3173574.3173677.

https://www.fiercehealthcare.com/ai-and-machine-learning/openevidence-launches-medical-ai-copilot-feature-grades-medical-evidence

https://www.eleken.co/blog-posts/agentic-ux-examples

Amershi, S., et al. (2019). “Guidelines for Human-AI Interaction.” Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems.

Google PAIR. People + AI Guidebook. “Explainability + Trust”; “Feedback + Control”; “Errors + Graceful Failure”.

Microsoft. Human-AI eXperience (HAX) Toolkit. “Guidelines for Human-AI Interaction” and “HAX Design Library”.

--

--

Rob Gifford
Rob Gifford

Written by Rob Gifford

Principal Designer @ athenahealth. Product + UX nerd