The Machine That Asks, “Are You Sure?”
How recursive AI is starting to catch the diagnoses doctors miss
Part 1 of 4 — The Bounded Machine: Recursive AI in Clinical Medicine
By TABARC-Studio
A radiologist studies a scan, narrows their eyes, and calls it benign. It’s a judgement made thousands of times a day, correctly, by people who are very good at their jobs and also, by hour eight of a shift, tired.
Now picture the same scan passed through a system built specifically to flag when it disagrees. Not loudly. Not with a diagnosis of its own pushed to the front. Just a note: this region’s uncertainty score is high enough that someone should look again.
That’s roughly what a framework out of Radboud University Medical Center in the Netherlands actually does. It’s called Uncertainty-Guided Annotation, and researchers validated it on Camelyon16 and Camelyon17 — the public datasets behind the well-known 2017 study where an algorithm matched pathologists at spotting breast cancer that had spread to lymph nodes.
The system doesn’t try to diagnose alone. It works out where its own confidence is thin, hands those specific regions to a clinician for correction, and folds the correction back in.
That’s a real, published mechanism, not a marketing pitch. And it’s a decent way into what this series is actually about: Recursive Self-Improvement, or RSI. The idea that a system can inspect its own performance, work out where it’s falling short, and adjust accordingly, case by case, rather than sitting frozen at whatever level it reached in training.
None of the foundational papers behind that idea were written with medicine in mind. Worth being upfront about that. But the theory they establish is exactly what’s now being tested, cautiously, on hospital wards.
Where the idea actually comes from
In 2018, a researcher named Wenyi Wang published a paper with a title that undersells how strange the result is: A Formulation of Recursive Self-Improvement and Its Possible Efficiency. It gave RSI a mathematical shape for the first time, rather than leaving it as a philosophical idea people had “dreamed of since the early days of computer science,” in the paper’s own words.
Working with a deliberately restricted version of the problem, Wang showed such a system is not just possible but efficient, running with logarithmic complexity in simulation. That word, restricted, will matter again in Part Two.
A few years later, a framework called Gödel Agent pushed the same idea closer to practice: a system that inspects and rewrites its own runtime logic through a loop of mutual recursion, sharpening its own coding and reasoning as it goes. Again, nothing to do with hospitals. This is software learning to edit itself.
Take that instinct and point it at a biopsy slide, though, and the calculation changes completely. Nobody sane wants a diagnostic model quietly rewriting its own logic overnight because a handful of new cases came in unsupervised.
Which is exactly why the field distinguishes, sharply, between letting a system change anything about itself and letting it improve inside a fence. A 2026 survey out of UC Riverside, AlphaAvatar and the Illinois Institute of Technology, which reviewed 1,250 arXiv papers from 2024 to 2026, gives this distinction real teeth: bounded self-refinement on one side, open-ended RSI on the other.
Medical AI belongs, almost without exception, to the bounded camp.
What the loop actually does
The same 2026 survey sorts these systems by how closed their feedback loop is, and the vocabulary is useful because it maps almost exactly onto how doctors already talk about human-in-the-loop medicine.
Three settings: a human reviews every single change before it happens; a human audits outcomes and gates deployment while the system generates its own improvement signal automatically; or the loop closes with no human review at all. Nearly everything currently running in a hospital sits in the first two categories, deliberately.
In practice, for something like the Radboud pathology framework, the loop runs roughly like this. The model reads a slide and produces both a call and a confidence score. Where that confidence drops below a set threshold, or resembles a pattern the model has previously got wrong, the case gets flagged rather than signed off.
A clinician reviews specifically the flagged region, not the whole slide again from scratch. Their correction goes back into the model. The next similar case is read a little more sharply.
Get TABARC-Social’s stories in your inbox
Join Medium for free to get updates from this writer.
This is a meaningfully different design from a system that diagnoses alone and asks for approval afterwards. It has learned, mathematically, how much it doesn’t know, and treats that gap as its cue to ask.
The clock that actually matters
None of this is abstract if you’re the patient behind the scan.
A model trained on a fixed set of historical slides will, sooner or later, hit a tumour that doesn’t resemble anything in its training data, and call it benign. On a locked, static system, that’s the end of it: the same mistake repeats until a developer manually retrains the model, which might be years away.
A system built to flag its own uncertainty behaves differently in exactly the case that matters. It doesn’t push a wrong answer with false confidence. It pushes an honest one: I’m not sure, please look. The correction that follows doesn’t just fix one patient’s case. It changes how the next similar case gets read.
Wearable monitoring is a rougher environment for the same idea, because motion is very good at faking arrhythmia. This is also where the failure mode on the other side becomes real. Alert fatigue in clinical decision support is not a hypothetical risk; it’s one of the best-documented problems in health informatics.
Across drug-interaction and monitoring alerts, override rates of 49 to 96 percent have been reported for years, and non-actionable alarms have been measured above 70 percent in some inpatient monitoring settings. A wearable that queries its wearer every time a sensor jitters will train people to ignore it within a week, which defeats the entire point of the loop.
The honest conclusion isn’t that these systems don’t work. It’s that getting the trigger threshold right is most of the engineering, and getting it wrong is the fastest way to burn the trust the whole approach depends on.
The regulatory door that actually opened
For years, the theory ran well ahead of what any regulator would sign off. The FDA’s long-standing position was that AI in a medical device had to be locked at approval, frozen, precisely because a model that kept changing itself didn’t fit any existing framework for what counts as “the same device.”
That changed, formally, on 4 December 2024, when the FDA published final guidance on Predetermined Change Control Plans, or PCCPs, for AI-enabled device software. A PCCP is, in effect, a document a manufacturer files in advance, specifying exactly what the model is allowed to learn from and how, so later updates don’t each need a fresh submission.
It builds on five guiding principles the FDA published jointly with Health Canada and the UK’s MHRA back in October 2023, covering things like keeping any change bounded, risk-based, and transparent to the people actually using the device. For a UK reader, that MHRA co-authorship is worth noticing. This isn’t purely an American regulatory story.
Where this leaves the doctor
It’s tempting to write this up as the slow-motion automation of diagnosis. What’s actually taking shape is narrower and, honestly, more interesting than that.
The published research behind it, the Radboud framework, the FDA’s own change-control paperwork… all describe something considerably less dramatic than a machine that thinks for itself: a machine that has been taught, carefully, when to stop.
Part Two of this series looks at what happens when that machine gets it wrong anyway, and the specific engineering built to catch it.
References
- Wang, W. (2018). A Formulation of Recursive Self-Improvement and Its Possible Efficiency. arXiv:1805.06610.
- Gödel Agent: A Self-Referential Agent Framework for Recursive Self-Improvement. arXiv:2410.04444.
- Chen, M., Wang, L., & Qu, B. (2026). Recursive Self-Improvement in AI: From Bounded Self-Refinement to Autonomous Research Loops. arXiv:2607.07663 (accepted, TMLR 2026).
- Khalili, N., Spronck, J., Ciompi, F., van der Laak, J., & Litjens, G. (2024). Uncertainty-guided annotation enhances segmentation with the human-in-the-loop. arXiv:2404.07208.
- US FDA (2024). Marketing Submission Recommendations for a Predetermined Change Control Plan for Artificial Intelligence-Enabled Device Software Functions. Final guidance, 4 December 2024.
- FDA, Health Canada & MHRA (2023). Predetermined Change Control Plans for Machine Learning-Enabled Medical Devices: Guiding Principles.
- Van der Sijs, H., et al. (2009); Sendelbach & Funk (2013); Drew et al. (2014) — on clinical alert override and alarm fatigue rates.
