<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Responsible Healthcare AI - Medium]]></title>
        <description><![CDATA[Ethical, Patient-centric, and Governance-first approaches to generative AI in healthcare and life-sciences applications. We encourage the sharing of research, thought leadership, case studies, theoretical exploration, and sociotechnical critique. - Medium]]></description>
        <link>https://medium.com/responsible-healthcare-ai?source=rss----fea69b1bd593---4</link>
        <image>
            <url>https://cdn-images-1.medium.com/proxy/1*TGH72Nnw24QL3iV9IOm4VA.png</url>
            <title>Responsible Healthcare AI - Medium</title>
            <link>https://medium.com/responsible-healthcare-ai?source=rss----fea69b1bd593---4</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Fri, 20 Nov 2026 18:22:44 GMT</lastBuildDate>
        <atom:link href="https://medium.com/feed/responsible-healthcare-ai" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Beyond the Hype: Training Clinicians for AI Ethics, Governance, and Patient Trust]]></title>
            <link>https://medium.com/responsible-healthcare-ai/beyond-the-hype-training-clinicians-for-ai-ethics-governance-and-patient-trust-f9b8cb2360de?source=rss----fea69b1bd593---4</link>
            <guid isPermaLink="false">https://medium.com/p/f9b8cb2360de</guid>
            <category><![CDATA[medical-school]]></category>
            <category><![CDATA[healthcare-technology]]></category>
            <category><![CDATA[healthcare]]></category>
            <category><![CDATA[generative-ai-tools]]></category>
            <category><![CDATA[learning]]></category>
            <dc:creator><![CDATA[Brian M. Green]]></dc:creator>
            <pubDate>Tue, 26 Aug 2025 22:27:22 GMT</pubDate>
            <atom:updated>2025-08-26T22:27:22.709Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="Four figures, one with a tablet and two are healthcare providers, with a symbol for AI trust and security and scale for balance in the background." src="https://cdn-images-1.medium.com/max/1024/1*SuqfrxIeP7Wkr6B9YCf-EA.png" /></figure><p><em>By Brian M. Green, MS, Founder, HealthRoundsAI by Envision-Health.AI, with Laura Offutt, MD, FACP, Chief Medical Officer, and Quincy Greene, MPH, Chief Data Officer, Envision-HealthAI.</em></p><p>It seems like AI in healthcare is everywhere right now, at conferences, in journals, integrated in hospital systems, and in the media. AI in healthcare is being touted as the next revolution and a panacea for many of the problems healthcare faces today.</p><p>But as people who have seen inside the world of MedAI education, here’s the contrarian truth: Many trainees get little to no education about AI’s roles in healthcare. Those that do receive AI training experience programs focused on familiarizing teams with the <strong>technology</strong>, which is great, but misses what is really at stake for AI in healthcare, <strong>AI ethics and governance, patient-centered care, and provider/patient communication.</strong></p><h3>Where Most AI Training Misses the Mark</h3><p>Most MedAI courses or workshops focus on technical aspects and system features, with a typical curriculum structure that breaks down into predictable components. When examining the educational landscape for healthcare AI training, you’ll notice these programs generally follow a pattern that emphasizes technological understanding over practical application. The typical structure of these educational offerings can be characterized as follows:</p><ul><li>Technical walkthroughs (“Here’s how the algorithm works…”)</li><li>Explanations of AI features integrated into EHR systems</li><li>Demos of clinical decision support tools</li><li>Sometimes, a slide or two on data privacy or ethics</li></ul><p>Yet most programs fail to provide meaningful training for clinicians, students, or support staff on the <strong>real-world ethical issues, AI governance,</strong> and <strong>human dilemmas</strong> that lie at the heart of AI in medicine. For example:</p><ul><li>How should you proceed when AI-generated suggestions contradict a patient’s own description of their symptoms, pain, and daily life impacts?</li><li>How to understand how the AI tool has been trained; for example, does the training data reflect the patient population for whom it is supposed to serve?</li><li>Who bears responsibility when an LLM’s “black box” algorithm misses a rare but relevant disease?</li><li>How can you build patient trust and ensure they feel safe when unfamiliar algorithms or data sources are involved?</li></ul><h3>Why Patient-Centricity Must Be at the Core</h3><p>Every week, we hear from doctors, nurses, and patients who feel that most AI tools are designed to replace human judgment or prioritize system efficiency, not person-centered care. If you ask patients what they want, it isn’t a flawless algorithm. Instead, they will ask for clear communication, context, empathy, and a sense that technology is there <strong>to enhance their care</strong>, not just to optimize throughput.</p><p>Years ago, I listened to dozens of stories from families affected by rare diseases. The lesson was clear: what matters most is feeling seen, heard, and empowered in decision-making, not simply being a data point for a predictive model. MedAI training must equip care teams to use technology as a tool for better understanding patient needs and enhancing communication, not just for automating workflows and handoffs.</p><p>Far from displacing clinicians, responsible AI can provide significant contributions to <strong>the care team</strong>, amplifying human expertise and extending resources rather than replacing them. Patients still require guidance from a human healthcare provider for enhanced explainability, to weigh care trade-offs, and make decisions that reflect their unique circumstances and personal values.</p><h3>Ethical AI as a Business Advantage</h3><p>Investing in ethical AI isn’t just the right thing to do; it’s also <strong>good business</strong>. Organizations that prioritize <strong>patient-centricity</strong>, <strong>equity</strong>, and <strong>trustworthiness</strong> see measurable returns across multiple dimensions:</p><ul><li><strong>Enhanced patient engagement:</strong> Ethical AI fosters transparency and trust, resulting in improved patient engagement, satisfaction, and outcomes.</li><li><strong>Market differentiation:</strong> Ethical safeguards increasingly serve as a <strong>competitive advantage</strong> in a crowded health‑tech landscape, and can lead to clear metrics for calculating the risk-adjusted ROI of AI integrations.</li></ul><p>By embedding ethics into <strong>design</strong>, <strong>development</strong>, and <strong>governance</strong> from the outset, we not only mitigate risk but also <strong>create enduring value</strong> for patients, providers, investors, and society as a whole.</p><h3>Three Essentials for “Real World” Healthcare AI Training</h3><h4><strong>Governance Before Gadgetry</strong></h4><ul><li>Every clinician should understand the boundaries of algorithmic decision-making, including consent and accountability, as well as model limitations, bias prevention, and mitigation.</li><li>Training must include case studies of ethical and governance dilemmas, error reporting, and the importance of human-in-the-loop oversight and escalation.</li></ul><h4><strong>Patient-Centric Frameworks</strong></h4><ul><li>AI recommendations are just one input; patient narrative, lived experience, and cultural context aren’t optional data points; they are <em>core to ethical practice</em> and should be integrated into model outputs at inference and orchestration layers.</li><li>Providers need to be trained and provided with point-of-care coaching support to build skills for explaining AI outcomes (recommendations) in plain language. This should include best practices in shared decision-making with patients and caregivers, especially for diverse populations and complex cases.</li></ul><h4><strong>Evidence-Based Clinical Context Simulation and Feedback</strong></h4><ul><li>Lectures alone won’t cut it. Scenario-based simulations and peer-driven feedback are crucial for developing Day-1 clinical readiness.</li><li>Integrate Entrustable Professional Activities (EPA) models (adapted for AI competencies), or equivalent for allied health professionals, into MedAI training, so learners master skills like documentation, communication, handoffs, and governance in context, not as afterthoughts when prompted by EHRs.</li></ul><h3>HealthRoundsAI: Our Evidence-Based Approach</h3><p>At HealthRoundsAI, we are designing every course and simulation around patient realities, governance best practices, and measurable outcomes. Every learning experience is mapped to practice-based clinical competencies, not tech adoption milestones. Our participants learn to:</p><ul><li>Navigate the complexities of trust, transparency, and contextual and role-based explainability in AI-supported care workflows.</li><li>Ethically use AI to strengthen patient-clinician communication and foster more effective care partnerships.</li><li>Implement continuous, evidence-based improvement by adapting AI systems to healthcare team workflows. This ensures that patient safety, empathy, effective communication, and technical advancements are seamlessly incorporated into new features and learning pathways.</li></ul><blockquote>“There is increasing urgency for teaching medical AI competencies in medical education, a setting rife with organizational resistance to updating medical curricula…Technology and innovation will proceed with or without us. This is our call to action — if medical people are concerned at all about how healthcare organizations, providers and patients will use AI in the near and distant future — we must evolve what is taught to medical students and in graduate medical education.”<em> -Laura Offutt. M.D., FACP</em></blockquote><h3>Key Insights and Conclusions</h3><p>AI in healthcare has the potential to be a powerful tool that can improve patient outcomes, but only when clinicians and care teams are trained for the messy, human realities that technology alone can’t solve. Healthcare professionals need comprehensive education that addresses the nuanced challenges that arise when implementing AI in clinical settings.</p><p>These challenges include navigating ambiguous diagnostic scenarios, managing conflicting data sources, and maintaining the human connection that forms the foundation of effective healthcare. If your AI training skips AI ethics and governance, patient-centricity, and evidence-based practice, it’s not just inadequate; it’s potentially harmful and just part of the AI hype cycle.</p><p>The future of healthcare depends on more than adopting cutting-edge AI technology. We need healthcare professionals who possess both AI literacy and practical competencies, skills that adapt to evolving contexts, prioritize patient needs, and keep pace with technological advancements. This means developing a workforce that can critically evaluate AI outputs, integrate them appropriately into clinical decision-making, and communicate effectively with patients about how AI influences their care. Without these fundamental skills, even the most sophisticated AI tools will fail to deliver on their promise of transforming healthcare for the better.</p><h4>Ready to move beyond the hype?</h4><p>Join us at <strong>HealthRoundsAI</strong> to experience MedAI education that truly prepares you for the challenges and opportunities of patient-centered AI innovation. Coming Soon!</p><p>For more information or to join the <a href="https://envision-health.ai/healthroundsai">HealthRounds.AI</a> waiting list, connect with us today.</p><p>This is Article 1 in a 10-part Article Series: <strong>“HealthRoundsAI™: Bridging the MedAI Education Gap.”</strong></p><figure><img alt="A graduation cap on the left side, illustrating healthcare education, and an AI symbol on the right side with a jagged line between them, illustrating the current gap between them." src="https://cdn-images-1.medium.com/max/886/1*e8GesYgqebDh_vdpJ02HKw.png" /><figcaption>HealthRoundsAI, Bridging the MedAI Education Gap Series</figcaption></figure><h3>References</h3><p>Sun L, Yin C, Xu Q, Zhao W. Artificial intelligence for healthcare and medical education: a systematic review. Am J Transl Res. 2023 Jul 15;15(7):4820–4828. PMID: 37560249; PMCID: PMC10408516.</p><p><a href="https://pmc.ncbi.nlm.nih.gov/articles/PMC10408516/">Artificial intelligence for healthcare and medical education: a systematic review</a></p><p>Blanco MA, Nelson SW, Ramesh S, Callahan CE, Josephs KA, Jacque B, Baecher-Lind LE. Integrating artificial intelligence into medical education: a roadmap informed by a survey of faculty and students. Med Educ Online. 2025 Dec;30(1):2531177. doi: 10.1080/10872981.2025.2531177. Epub 2025 Jul 14. PMID: 40660466; PMCID: PMC12265092.</p><p><a href="https://pmc.ncbi.nlm.nih.gov/articles/PMC12265092/">Integrating artificial intelligence into medical education: a roadmap informed by a survey of faculty and students</a></p><p><em>AMA J Ethics.</em> 2019;21(2):E138–145. doi: 10.1001/amajethics.2019.138.</p><ul><li><a href="https://pubmed.ncbi.nlm.nih.gov/30794123/">How Should Clinicians Communicate With Patients About the Roles of Artificially Intelligent Team Members? - PubMed</a></li><li><a href="https://jmai.amegroups.org/article/view/6664/html">Factors influencing trust in medical artificial intelligence for healthcare professionals: a narrative review</a></li><li><a href="https://medium.com/responsible-healthcare-ai/the-ethical-use-of-generative-ai-in-healthcare-4fab1f045363">The Ethical Use of Generative AI in Healthcare</a></li></ul><p>Kimiafar K, Sarbaz M, Tabatabaei SM, Ghaddaripouri K, Mousavi AS, Raei Mehneh M, Mousavi Baigi SF. Artificial intelligence literacy among healthcare professionals and students: A systematic review. Front Health Inform. 2023; 12: 168. DOI: 10.30699/fhi.v12i0.52</p><ul><li><a href="https://medium.com/responsible-healthcare-ai/governance-first-navigating-the-spectrum-of-ai-concerns-and-risks-in-healthcare-12bb6bce2500">Governance First: Navigating the Spectrum of AI Concerns and Risks in Healthcare</a></li><li><a href="https://medium.com/responsible-healthcare-ai/reimagining-family-centered-care-evaluation-a-consumer-driven-approach-for-children-and-youth-with-70842ffee136">Reimagining Family-Centered Care Evaluation: A Consumer-Driven Approach for Children and Youth with…</a></li></ul><p>Knopp M, Warm E, Weber D, Kelleher M, Kinnear B, Schumacher D, Santen S, Mendonça E, Turner L. AI-Enabled Medical Education: Threads of Change, Promising Futures, and Risky Realities Across Four Potential Future Worlds. JMIR Med Educ 2023;9:e50373. DOI: 10.2196/50373</p><ul><li><a href="https://mededu.jmir.org/2023/1/e50373/">AI-Enabled Medical Education: Threads of Change, Promising Futures, and Risky Realities Across Four Potential Future Worlds</a></li><li><a href="https://medium.com/responsible-healthcare-ai/why-an-ethical-evaluation-framework-for-ai-agents-is-essential-in-the-era-of-multi-agent-solutions-be93c7b5c5ce">Why an Ethical Evaluation Framework for AI Agents Is Essential in the Era of Multi-Agent Solutions</a></li><li><a href="https://generativeai.pub/the-transparency-containment-paradox-balancing-trust-and-security-in-healthcare-ai-b2f93d8db737">The Transparency‑Containment Paradox™: Balancing Trust and Security in Healthcare AI</a></li><li><a href="https://www.enterprisedb.com/news/data-observability-governance/pilot-practice-scaling-ai-healthcare-ethical-oversight-brian-m">From pilot to practice: scaling AI in healthcare with ethical oversight</a></li><li><a href="https://arxiv.org/abs/2407.18939">Promoting AI Competencies for Medical Students: A Scoping Review on Frameworks, Programs, and Tools</a></li><li><a href="https://arxiv.org/abs/2505.02062">Ethical AI in the Healthcare Sector: Investigating Key Drivers of Adoption through the Multi-Dimensional Ethical AI Adoption Model (MEAAM)</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f9b8cb2360de" width="1" height="1" alt=""><hr><p><a href="https://medium.com/responsible-healthcare-ai/beyond-the-hype-training-clinicians-for-ai-ethics-governance-and-patient-trust-f9b8cb2360de">Beyond the Hype: Training Clinicians for AI Ethics, Governance, and Patient Trust</a> was originally published in <a href="https://medium.com/responsible-healthcare-ai">Responsible Healthcare AI</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Reimagining Family-Centered Care Evaluation: A Consumer-Driven Approach for Children and Youth with…]]></title>
            <link>https://medium.com/responsible-healthcare-ai/reimagining-family-centered-care-evaluation-a-consumer-driven-approach-for-children-and-youth-with-70842ffee136?source=rss----fea69b1bd593---4</link>
            <guid isPermaLink="false">https://medium.com/p/70842ffee136</guid>
            <category><![CDATA[evaluation]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[healthcare]]></category>
            <category><![CDATA[family]]></category>
            <category><![CDATA[generative-ai-tools]]></category>
            <dc:creator><![CDATA[Brian M. Green]]></dc:creator>
            <pubDate>Tue, 29 Jul 2025 11:54:51 GMT</pubDate>
            <atom:updated>2024-06-26T23:19:04.491Z</atom:updated>
            <content:encoded><![CDATA[<h3><strong>Reimagining Family-Centered Care Evaluation: A Consumer-Driven Approach for Children and Youth with Chronic Health Conditions</strong></h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*P0SUJ-VwL4ZIV5IdgWO7AA.png" /><figcaption>Image generated by the author using MS copilot, DALL-E 3</figcaption></figure><p>There is a growing emphasis on patient-centered care in today&#39;s healthcare landscape. However, it&#39;s important to remember that family-centered care (FCC) is not just a critical, but an integral aspect of service delivery, particularly for children and adolescents with chronic health conditions. It&#39;s not a matter of one or the other, but rather a combination of both that truly benefits the patient.</p><p>Chronic conditions such as cystic fibrosis, diabetes, or <a href="https://brian-m-green.com/blog/f/unveiling-the-potential-digital-twins-for-personalized-medicine">rare diseases present unique challenges</a> for families. Managing medications, navigating complex treatment plans, and coping with the emotional and social impacts of these conditions all require a supportive family environment. FCC, with its emphasis on informed decision-making, fosters collaboration between healthcare professionals, patients, and families, ensuring care plans are tailored to individual needs.</p><p>As AI technology becomes increasingly integrated into healthcare, ensuring that FCC principles are embedded within these advancements is even more critical. The use of AI in developing diagnostic tools, designing interventions, triaging, and providing decision support for healthcare providers (HCPs) necessitates a family-centered approach to maximize patient outcomes and satisfaction.</p><h3>Understanding Family-Centered Care</h3><p>Family-centered care is a model of care that emphasizes the <a href="https://brian-m-green.com/blog/f/unveiling-the-potential-digital-twins-for-personalized-medicine">partnership between families and healthcare</a> providers. It is built on four key principles that have implications for the care, services, and evaluation design:</p><ol><li><strong>Dignity and Respect</strong>: Listening to and honoring the perspectives and choices of patients and families. This includes understanding how people define their family, family strengths, and cultural values.</li><li><strong>Information Sharing</strong>: Providing complete and unbiased information to patients and families in affirming and useful ways. Consent to include multiple family members in a patient record requires respect for patient and family privacy and adherence to HIPAA and other relevant regulations. Data systems should be designed to incorporate FCC principles, including developing data structures that link family members to the index patient.</li><li><strong>Participation</strong>: Encouraging and supporting patients and families to participate in care and decision-making at their chosen level. Healthcare systems should enhance data collection practices to include comprehensive family information. This can involve updating electronic health records (EHRs) to better capture family relationships and interactions.</li><li><strong>Collaboration</strong>: Collaborating with patients, families, and providers in policy and program development, implementation, and evaluation. Collaborative efforts between program planners, families, and healthcare professionals can ensure that programs and services are tailored to capture the complexities of family dynamics and their impact on health outcomes.</li></ol><h3>The Impact of FCC on Health Outcomes</h3><p>Studies have shown a clear link between FCC and positive health outcomes. For example, a meta-analysis by Gibson et al. (2010) found that FCC interventions improved patient health outcomes, increased patient satisfaction, and better adherence to treatment plans. This makes FCC essential for promoting the well-being of children and adolescents with chronic conditions.</p><h3>Challenges in Evaluating FCC</h3><p>Traditional evaluation methods for FCC often rely heavily on quantitative data, focusing on process measures like service utilization or appointment attendance. While these data points offer some insights into program reach and efficiency, they fail to capture families&#39; lived experiences adequately. These experiences encompass a range of factors, including emotional well-being, caregiver burden, communication between families and healthcare providers, and access to necessary resources.</p><p>Most healthcare databases focus on individual patient data and lack robust mechanisms for linking family members to an index patient. This gap can lead to incomplete or inaccurate data representation for FCC. Without proper linking, the nuanced dynamics of family interactions and their impact on health outcomes are lost, making it difficult to develop AI models that truly reflect FCC principles.</p><p>The lack of data systems that recognize family connections and dynamics means that traditional evaluation and assessments will produce outputs that do not adequately consider the family context. The result is that we are likely to see program or service recommendations or decisions that are not family-centered.</p><h3>Limitations of Traditional Evaluation Methods</h3><p>The focus on process measures can be particularly problematic in the context of rare diseases. Due to the low prevalence of these conditions, sample sizes are often small, making it difficult to draw statistically significant conclusions from traditional quantitative evaluations. Additionally, the unique nature of rare diseases means that “one-size-fits-all” metrics may not adequately capture the specific needs and challenges families face.</p><h3>Reimagining FCC Evaluation: A Consumer-Driven Approach</h3><p>There is a critical need for a more nuanced and comprehensive approach to FCC evaluation. A consumer-driven approach centered on the voices of families can offer valuable insights that traditional methods often miss.</p><p>A consumer-driven approach involves engaging families throughout the evaluation process. This can be achieved through participatory design, technology-aided data collection, mixed-methods approaches, and longitudinal studies. Here’s how this approach can be implemented:</p><ol><li><strong>Participatory Design</strong>: Move beyond simply including families as passive participants in the evaluation. Instead, actively engage them throughout the process — from defining success to developing evaluation tools. Focus groups, surveys, and individual interviews can be used to gather qualitative data on what “family-centered care” truly means to them and how it impacts their lives. Understanding their perspectives on what constitutes high-quality FCC is crucial for crafting meaningful evaluation measures.</li><li><strong>Technology-Aided Data Collection</strong>: Leverage the power of technology to collect data efficiently and conveniently for families. This could involve online surveys, mobile apps specifically designed for data collection. Cloud resources can also be used to for collecting qualitative data like interview transcripts or journaling entries using video ethnography or other techniques. Technology can also facilitate participation from geographically dispersed families, ensuring a wider evaluation representation of family experiences.</li><li><strong>Mixed-Methods Approach</strong>: <a href="https://www.jmir.org/2020/7/e17338/">Combine quantitative and qualitative data</a> collection methods. Quantitative data can offer a broader picture of program reach and utilization, while qualitative data can provide rich insights into the impact of FCC on families’ emotional well-being, decision-making processes, and overall perceptions of care.</li><li><strong>Longitudinal Studies</strong>: Static snapshots of data provide limited insight. Longitudinal studies that track families over time offer a more holistic understanding of the long-term benefits of FCC programs. This can reveal how FCC empowers families to adapt and manage their child’s condition over time, informing ongoing program development and improvement efforts.</li><li><strong>Integrating Video Ethnography</strong>: Video ethnography can capture the complexities of caregiving and family dynamics, particularly for rare diseases. This method provides detailed documentation, enhances communication between families and healthcare providers, and conveys the emotional impact of caregiving.</li></ol><h3>Case Studies in Family-Centered Care</h3><p>A Scoping Review by Kokorelias et al., 2019, underscores the importance of FCC in improving healthcare outcomes. By identifying both universal and illness-specific components of FCC models, the review provides a comprehensive framework for implementing and evaluating FCC across different healthcare settings.</p><p>Common desired outcomes of FCC models include enhanced feelings of mastery and empowerment among family members, improved family engagement in care planning, and better health outcomes, including increased adherence to treatment plans and improved quality of life for patients and families.</p><p>While many FCC components were found to be universal, certain aspects were tailored to specific illnesses, such as pediatric HIV, cancer, and mental health conditions. These adaptations addressed the unique needs of different patient populations and care contexts.</p><p>To illustrate the specific impact of FCC, let’s explore a few case studies:</p><ol><li><strong>Neonatal Intensive Care Units (NICUs)</strong>: In NICUs, involving parents in caring for their infants has shown remarkable benefits (Cooper, et al., 2007). Parents participating in rounds, kangaroo care, and decision-making processes reported feeling more empowered and connected to their child’s care. This involvement also fosters better communication and trust between families and healthcare providers.</li><li><strong>Pediatric Oncology</strong>: FCC in pediatric oncology settings ensures that the patient&#39;s and family&#39;s emotional and psychological needs are addressed (Smith, et al., 2018). Family advisory councils, sibling support programs, and family-centered rounds are some of the strategies employed to support families during the challenging journey of cancer treatment.</li></ol><p>As we look to modernize FCC evaluation, integrating technology and consumer feedback becomes crucial. These elements can enhance the accuracy and relevance of evaluation processes, ensuring they remain aligned with contemporary healthcare practices.</p><h3>Conclusion</h3><p>Family-centered care represents a fundamental shift in how most approach healthcare. By recognizing the critical role of families and fostering a collaborative environment, we can improve health outcomes, enhance satisfaction, and create a more responsive healthcare system. As we continue to refine and expand FCC practices, staying committed to the core principles of dignity, respect, information sharing, participation, and collaboration is essential. Through these efforts, we can ensure that patients and their families are at the heart of healthcare.</p><p>The current trend towards developing large language models (LLMs) and multimodal large language models (MLLMs) for generative AI applications presents unique challenges for advancing Family Centered Care (FCC) principles. These challenges stem from the limitations of existing data sources, which often do not adequately link family members to an index patient. As a result, FCC data may be underrepresented or misrepresented in these models unless explicitly designed with FCC considerations in mind.</p><p>Modernizing FCC evaluation involves adopting innovative approaches and leveraging technology to ensure comprehensive and accurate assessments. By integrating consumer feedback and utilizing tools like video ethnography, healthcare providers can enhance the effectiveness of FCC programs, ultimately improving health outcomes for families.</p><p>The advancement of <a href="https://medium.com/@BrianMGreen/responsible-use-of-ai-in-healthcare-strategies-for-balancing-opportunities-and-risk-in-a-b29ab27259b4">LLMs and AI to guide healthcare</a> decision tools represents an opportunity for family-centered care. By designing AI systems that explicitly incorporate FCC principles, improving data collection practices, and establishing strong ethical frameworks, we can ensure that generative AI applications support and enhance family-centered care.</p><h3>References</h3><p>Callery, P., et al. (2017). Using video-ethnography to explore the experiences of families living with rare diseases. <em>Journal of Clinical Nursing</em>, 26(23–24), 4777–4788.</p><p>Cooper, L. G., Gooding, J. S., Gallagher, J., Sternesky, L., Ledsky, R., &amp; Berns, S. D. (2007). Impact of a family-centered care initiative on NICU care, staff and families. <em>Journal of Perinatology, 27</em>(2), S32-S37.</p><p>Gooding, J. S., Cooper, L. G., Blaine, A. I., Franck, L. S., Howse, J. L., &amp; Berns, S. D. (2011). Family support and family-centered care in the neonatal intensive care unit: origins, advances, impact. <em>Seminars in Perinatology</em>, 35(1), 20–28.</p><p>Green, B.M. (2003). Quality Improvement Initiatives with Families Affected by HIV. <em>Advances in Family-Centered Care</em>. Bethesda, MD: Institute for Family-Centered Care.</p><p>Green, B.M. (2024). Unveiling the Potential: Digital Twins for Personalized Medicine. <a href="https://brian-m-green.com/blog/f/unveiling-the-potential-digital-twins-for-personalized-medicine">https://brian-m-green.com/blog/f/unveiling-the-potential-digital-twins-for-personalized-medicine</a></p><p>Green, B.M. (2024). Prioritizing Patient-Centricity in GenAI Healthcare Solutions. <a href="https://brian-m-green.com/blog/f/prioritizing-patient-centricity-in-genai-healthcare-solutions">https://brian-m-green.com/blog/f/prioritizing-patient-centricity-in-genai-healthcare-solutions</a></p><p>Green, B.M. (2024). Responsible Use of AI in Healthcare — Strategies for Balancing Opportunities and Risk in a Speculative Environment. <a href="https://medium.com/@BrianMGreen/responsible-use-of-ai-in-healthcare-strategies-for-balancing-opportunities-and-risk-in-a-b29ab27259b4">https://medium.com/@BrianMGreen/responsible-use-of-ai-in-healthcare-strategies-for-balancing-opportunities-and-risk-in-a-b29ab27259b4</a></p><p>Green, B.M, Van Horn, K.T., et al. (2020). Assessment of Adaptive Engagement and Support Model for People With Chronic Health Conditions in Online Health Communities: Combined Content Analysis<br>J Med Internet Res 2020;22(7):e17338. doi: 10.2196/17338</p><p>Kokorelias, K.M., Gignac, M.A.M., Naglie, G. <em>et al.</em> Towards a universal model of family centered care: a scoping review. <em>BMC Health Serv Res</em> <strong>19</strong>, 564 (2019). <a href="https://doi.org/10.1186/s12913-019-4394-5">https://doi.org/10.1186/s12913-019-4394-5</a></p><p>Smith, L., Cheater, F., Bekker, H., &amp; Warner, J. (2018). Parents’ experiences of living with a child with cancer: a systematic review and synthesis of qualitative research. <em>European Journal of Cancer Care</em>, 27(6), e12917.</p><p>Watson, M., et al. (2004). Using video-reflexive ethnography to explore the experiences of families of children with cerebral palsy. <em>Journal of Child Health Care</em>, 8(4), 285–298.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=70842ffee136" width="1" height="1" alt=""><hr><p><a href="https://medium.com/responsible-healthcare-ai/reimagining-family-centered-care-evaluation-a-consumer-driven-approach-for-children-and-youth-with-70842ffee136">Reimagining Family-Centered Care Evaluation: A Consumer-Driven Approach for Children and Youth with…</a> was originally published in <a href="https://medium.com/responsible-healthcare-ai">Responsible Healthcare AI</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Health IT Regulatory Crossroads: Implications for AI-Driven Innovation]]></title>
            <link>https://medium.com/responsible-healthcare-ai/health-it-regulatory-crossroads-implications-for-ai-driven-innovation-b93ec4aa2c4e?source=rss----fea69b1bd593---4</link>
            <guid isPermaLink="false">https://medium.com/p/b93ec4aa2c4e</guid>
            <category><![CDATA[regulation]]></category>
            <category><![CDATA[healthtech]]></category>
            <category><![CDATA[policy]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[ai]]></category>
            <dc:creator><![CDATA[Brian M. Green]]></dc:creator>
            <pubDate>Wed, 23 Jul 2025 05:38:39 GMT</pubDate>
            <atom:updated>2025-07-23T05:38:39.474Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="A doctor holding a tablet standing at a forked road with roadsigns saying “regulation” and “innovation” with AI and medical symbols in the background." src="https://cdn-images-1.medium.com/max/1024/1*0XEXAm7mjSHKQ8SqprUiQA.png" /><figcaption>Image created by author using DALL-E3 in ChatGPT4o</figcaption></figure><p><em>For HealthTech executives, investors, and policymakers navigating the evolving AI Health regulatory landscape</em></p><h3>A Watershed Moment for Health IT Regulation</h3><p>The U.S. Department of Health and Human Services (DHHS) recently concluded its Request for Information (RFI) to identify outdated or misaligned regulations “stifling innovation” in healthcare technology. This effort, framed by Executive Orders 14219 and 14192, represents the current administration’s deregulatory push and the largest in DHHS history.</p><p>With responses from major Health IT stakeholders — including the Electronic Health Record Association (EHRA), National Committee for Quality Assurance (NCQA), and AdvaMed (as well as my own submission) — the debate over <strong>clinical decision support (CDS) tools</strong>, <strong>FDA-ONC jurisdictional conflicts</strong>, and <strong>public health infrastructure</strong> has reached a critical juncture.</p><blockquote>For executives and investors, the stakes are clear: How HHS responds to these submissions will reshape compliance budgets, product development timelines, and market entry strategies for AI-powered health tools over the next decade.</blockquote><h3>The EHRA’s Case for Regulatory Realignment</h3><p>The HIMSS Electronic Health Record Association (EHRA) response to the RFI, submitted on behalf of nearly 30 major health IT developers, serves as a clarion call for modernizing oversight frameworks.</p><h3>Concerns with the FDA’s 2022 CDS Guidance:</h3><p>At the heart of the EHRA’s rationale is the assertion that the FDA’s 2022 CDS guidance <strong>oversteps statutory authority</strong> by introducing unauthorized terms like “time-criticality” and “automation bias” absent from the 21st Century Cures Act. EHRA argues that these create legal ambiguity, forcing developers to navigate redundant regulatory processes. For AI-powered clinical decision-support (CDS) tools embedded in EHRs (e.g., predictive analytics for chronic disease management), this 2022 CDS framework imposes:</p><ul><li><strong>Duplicative oversight</strong>: FDA certification, layered atop ONC’s Health IT Certification Program, increases compliance costs by over $ 24,000 per 510(k) submission. According to EHRA, this creates a chilling effect that discourages risk-taking and incentivizes risk-averse development cycles.</li><li><strong>Innovation barriers</strong>: AI-driven CDS tools that rely on real-world data (RWD) can face delayed deployment due to the FDA’s prescriptive safety requirements, which often conflict with the ONC’s interoperability focus.</li></ul><p>The EHRA advocates <strong>reverting to the risk-based 2019 draft guidance</strong>, which excluded CDS tools in ONC-certified EHRs from FDA oversight (it is unclear whether the 2019 guidance is sufficient for AI CDS). This reform could have the effect of redirecting compliance resources toward R&amp;D, which is critical for advancing predictive analytics in oncology, neurology, and population health management.</p><h3>Modernizing Electronic Case Reporting (eCR) for Public Health AI</h3><p>While the EHRA emphasizes <strong>retaining FDA’s 2022 guidance</strong>, it urges HHS to stabilize electronic case reporting (eCR) frameworks as a counterbalance. Chronic disease tracking, enabled through electronic clinical records (eCR) systems, relies on scalable AI solutions to identify trends in diabetes, hypertension, and respiratory conditions. However, fragmented state/local coordination, as well as historically underfunded CDC infrastructure, currently hinder development.</p><p><strong>Proposed actions include</strong>:</p><ul><li>Establishing national eCR standards to reduce developer rework.</li><li>Increasing CDC technical assistance funding to support STLTs (states, tribes, local jurisdictions) in adopting automated reporting tools.</li></ul><p><strong>For investors</strong>, this presents an opportunity to support firms specializing in data interoperability and AI-driven public health analytics, which are crucial for fulfilling Executive Order 14212’s mandate to prioritize the alleviation of chronic diseases.</p><h3>Overhauling Legacy Compliance Mechanisms</h3><p>Three additional EHRA priorities are focused on the DHHS systemic inefficiencies that may affect AI adoption in healthcare:</p><p><strong>Table 1: Legacy Regulations and Proposed Changes</strong></p><figure><img alt="Table showing 3 proposed regulatory changes, and their implications for Health IT" src="https://cdn-images-1.medium.com/max/1024/1*qzBqySAsl84IbPsQIHM0iA.png" /><figcaption>Legacy Regulations and EHRA Proposed Changes</figcaption></figure><p>These reforms would enable agile AI development cycles, allowing for iterative model training without the need for repeated regulatory submissions, something that industry, investors, and insurance payors would like to see.</p><h3>Broader Stakeholder Concerns</h3><p>While the EHRA’s submission may dominate headlines, other major industry players submitted equally impactful feedback to DHHS. These submissions reveal <strong>cross-cutting priorities</strong> spanning FDA modernization, telehealth expansion, and patient-centered AI development. Below is my synthesis of key themes and recommendations that extend beyond the core concerns of EHR developers.</p><p><strong>Table 2: Summary of Eight Stakeholder Organizations&#39; Recommendations</strong></p><figure><img alt="Table showing Eight stakeholder organizations top 2 focus areas in Health IT and their recommended policy changes." src="https://cdn-images-1.medium.com/max/1024/1*uIFI7mtFvpEpXxVDbqt3XQ.png" /><figcaption>Summary of Stakeholder Organization Submissions to DHHS</figcaption></figure><h3>Quality Measurement and AI Alignment</h3><p><strong>The National Committee for Quality Assurance (NCQA)</strong> emerged as a vocal advocate for <strong>FHIR-based quality reporting</strong>, urging the retirement of legacy QDM/QRDA standards (HL7 Clinical Document Architecture and Quality Reporting Document Architecture). Their focus mirrors broader demands to integrate AI tools into value-based care analytics in two primary areas: data quality management and aligned patient-centered tools.</p><p><strong>Digital Quality Measurement </strong>recommendations include:</p><ul><li>Retire outdated data frameworks to prioritize <strong>FHIR/CQL interoperability</strong>.</li><li>Pilot <strong>Bulk FHIR Quality Coalitions</strong> to test real-world AI-driven reporting.</li></ul><p><strong>Patient-Centered Tools </strong>should be prioritized to:</p><ul><li>Develop CMS-sponsored directories for certified digital health tools to reduce fragmentation.</li><li>Support AI models aligned with Medicare Star Ratings and HEDIS.</li></ul><p>These efforts address a systemic challenge: <strong>legacy frameworks, </strong>such as QDM/QRDA, create silos, forcing AI developers to map proprietary data models to outdated standards. By standardizing on FHIR/CQL, NCQA aims to <strong>unify quality metrics</strong> across Medicare Advantage, the Medicare Merit-based Incentive Payment System (MIPS), and other programs.</p><p><strong>Investment Angle</strong>: Startups utilizing FHIR-compliant AI for population health analytics may be able to capture a growing share of Medicare Advantage innovation funds tied to chronic disease management.</p><h3>FDA Regulatory Modernization</h3><p><strong>AdvaMed</strong> (Advanced Medical Technology Association) and the <strong>American Heart Association (AHA)</strong> aligned on <strong>AI-driven device regulation</strong> concerns in two key areas:</p><p><strong>AI/ML Guidance</strong>:</p><ul><li>Reduce 510(k) fees (from $24,000 to $6,000) for small firms developing AI diagnostic tools and devices.</li><li>Update labeling rules to permit digital formats, thereby accelerating the deployment of AI tools.</li></ul><p><strong>Global Leadership</strong>:</p><ul><li>Warnings against FDA staffing cuts that could delay approvals are a critical issue for the AI-driven medtech industry.</li></ul><p><strong>Telehealth Synergy</strong>: While AdvaMed focuses on <strong>device classification</strong>, the American Telemedicine Association (ATA) emphasizes <strong>platform-level HIPAA compliance</strong> to streamline the integration of AI into virtual care workflows, such as remote-patient monitoring.</p><p>FDA regulatory modernization is less about replacing oversight and more about targeting high-impact efficiency gains. While stakeholders push for fee cuts and clearer pathways for AI, the core challenge remains balancing innovation velocity with patient safety.</p><p><strong>Investors</strong> following health AI developments should monitor the HHS’s implementation of the EHRA’s proposed <strong>CDS carve-outs</strong> and AdvaMed’s <strong>AI-specific guidance</strong> recommendations, which may be pivotal for de-risking portfolio companies.</p><h3>Telehealth Expansion</h3><p><strong>AHA, ATA, AAMC, and </strong><a href="http://health-vision.ai/"><strong>Health-Vision.AI</strong></a><strong>, LLC</strong> all prioritized <strong>permanent telehealth</strong> flexibilities, a critical piece of the infrastructure needed for scaling AI-driven remote care. Key policy recommendations here include:</p><p><strong>Table 3: Telehealth Policy Recommendations</strong></p><figure><img alt="A table showing 3 telehealth policy recommendations including Interstate Licensure, Behavioral Health, and Prior Authorization actions." src="https://cdn-images-1.medium.com/max/1024/1*w26rrQVpBufRvfSO2NTGlQ.png" /><figcaption>Key Telehealth Policy Recommendations</figcaption></figure><p><strong>Regulatory Conflict</strong>: The push for <strong>administrative relief</strong> clashes with the FDA’s overlapping oversight, underscoring the need for agency alignment. The push for <strong>permanent telehealth expansions,</strong> a dominant theme in stakeholder submissions to HHS’ RFI, reflects an industry imperative.</p><p>For <strong>business executives and investors</strong>, the stakes are clear: without legislatively entrenched telehealth policies, interoperability advancements, and AI adoption, the risk of stalling is significant. This underscores the need to lock in pandemic-era innovations to scale AI-driven remote care, behavioral health access, and chronic disease management.</p><h3>Research Infrastructure</h3><p>Both the <strong>AMA</strong> and <strong>AAMC</strong> highlighted systemic gaps in <strong>clinical trial support </strong>and included recommendations for:</p><ul><li><strong>Decentralized Clinical Trials (DCTs)</strong>: Expand telehealth authorizations for remote patient monitoring.</li><li><strong>Community Sites</strong>: Streamline training/liability frameworks for local clinicians deploying AI tools.</li><li><strong>IRB Harmonization</strong>: Align conflict-of-interest rules across agencies to reduce redundant Institutional Review Board (IRB) submissions, particularly for DCTs.</li></ul><p>Stakeholders advocate eliminating bureaucratic redundancies through <strong>IRB harmonization</strong> and <strong>FHIR mandates</strong> for trial data exchange, while urging funding for decentralized clinical trials (DCTs) and community clinician training to accelerate AI validation.</p><p>Proposals include streamlining prior authorization for telehealth-monitored studies and retiring outdated reporting requirements to reduce legal friction. These reforms aim to counter geographic and socioeconomic biases in AI models by expanding access to underrepresented populations while enabling faster adoption of predictive analytics.</p><p><strong>AI Implications</strong>: Simplified trial protocols could accelerate validation of predictive analytics in oncology, rare diseases, and population health.</p><p><strong>Business Executives and Investors</strong> may be particularly interested in platforms that enable AI-driven <strong>remote patient monitoring</strong> and <strong>predictive analytics</strong> for decentralized clinical trials involving drugs and diagnostics. As well, AI systems and tools that prioritize <strong>FHIR Adoption</strong> and <strong>TEFCA/eCR Alignment</strong> are likely to receive future regulatory favor and incentives in the marketplace.</p><h3>Patient-Centered AI and Regulatory Clarity</h3><p>Patient-centered AI represents a pivotal shift in healthcare innovation, prioritizing <strong>transparency, consent, and equitable access</strong> in algorithmic decision-making. However, current regulatory frameworks often lag behind these ethical imperatives, creating legal and operational ambiguity.</p><p>Key stakeholders, including <a href="https://health-vision.ai/"><strong>Health-Vision.AI</strong></a><strong>, LLC,</strong> and the <a href="https://connectedhi.com/">Connected Health Initiative</a>, advocate for recognition of critical gaps that help address <strong>the trust deficit</strong> among patients and <strong>commercial uncertainty,</strong> including:</p><ul><li><strong>Patient-Centered Tools</strong>: require regulatory alignment on <strong>data guidelines</strong> that ensure patient ownership, dynamic consent granularity, and equitable algorithm design. Additionally, clarity regarding <strong>AI device/non-device classification</strong> is needed from regulatory bodies.</li><li><strong>Sector Alignment</strong>: recommendations that complement NCQA’s digital quality push with <strong>transparent AI implementation</strong> rules. We echo EHRA/AdvaMed’s call for <strong>CDS oversight reform</strong>, but expand it to include patient-facing tools such as symptom checkers and health and wellness coaching tools.</li></ul><p><strong>Business Executives and Investors</strong> should prioritize <strong>Consent-Driven AI architectures.</strong> This involves developing tools that enable granular patient control over data sharing, including FHIR-enabled APIs. This can reduce patient data privacy risks and build trust in platforms, such as AI-assisted mental health or chronic disease coaches, if classified by the FDA as non-devices.</p><h3>The FDA-ONC Conflict: Structural Roots and AI Implications</h3><p>At the core of health IT regulation lies a <strong>jurisdictional tug-of-war</strong> between the FDA and ONC, exacerbated by AI’s dual role as both a potential clinical tool and a data infrastructure. Below is an analysis of statutory foundations and paths forward.</p><p><strong>Table 4: Current Statutory Authority, FDA and ONC</strong></p><figure><img alt="Table showing the statutory authority for FDA and ONC along with their core mandates." src="https://cdn-images-1.medium.com/max/1024/1*i_CZVZsP3V5546mUegLHWw.png" /><figcaption>Statutory Authority for FDA and ONC and Core Mandates</figcaption></figure><p>The FDA’s statutory power to regulate “medical devices” has the potential to conflict with ONC’s focus on interoperability. AI tools that <em>generate actionable recommendations</em> (e.g., sepsis prediction models) represent a grey area as features of the AI tools and their aligned clincial workflows vacillate between these two regulatory schemas.</p><h3>Paths to Resolution of Authority</h3><p>Among the various stakeholders who submitted comments, many common themes surfaced, including:</p><ol><li><strong>Reinforcing Cures Act Carve-Outs</strong>: Explicitly excluding non-device CDS tools from FDA oversight.</li><li><strong>Outcome-Based ONC Certification</strong>: Aligning ONC’s Real World Testing (RWT) with actionable <strong>interoperability metrics</strong> (e.g., FHIR adoption rates and other key metrics).</li><li><strong>Risk Stratification Frameworks</strong>: Treating AI models as devices only when directly automating clinical decision pathways (e.g., insulin dosing algorithms).</li></ol><h3>Investment and Executive Considerations</h3><p>For health IT leaders and investors, the HHS RFI response window marks a strategic inflection point. Business policy advisors and Executives must assess whether:</p><ul><li><strong>AI tools</strong> fall under the FDA’s purview (e.g., high-risk diagnostics) or the ONC’s scope of interoperability (e.g., population health analytics).</li><li><strong>PCCP and 513(g) reforms</strong> could lower costs for iterative AI model development for SaMD, a growing pain point amid RWD integration.</li></ul><blockquote><strong>PCCP</strong> (Predetermined Change Control Plans): These plans enable AI/ML medical device developers (SaMD) to make predetermined changes to their software without requiring a new FDA submission for each update. Current regulations prohibit changes to intended use without new FDA submissions, and the proposed fix is to “Permit updates via PCCPs to accelerate AI model iteration.”</blockquote><blockquote><strong>513(g)</strong>: This refers to Section 513(g) of the Federal Food, Drug, and Cosmetic Act, which provides a pathway for developers to request the FDA’s feedback on how their product would be classified (as a device or non-device). Reform in this area would help de-risk iterative AI development and provide clarity on the delineation between high-risk and low-risk tools.</blockquote><h4>Industry-Wide Trends</h4><p>These submissions reflect a <strong>sector-wide push</strong> to modernize health IT regulation. While EHRA and others focused on FDA/ONC jurisdictional conflicts, many stakeholders emphasized telehealth, quality measurement, and medtech innovation, as seen in AdvaMed’s FDA reform proposals and AHA’s chronic disease focus.</p><p><strong>Table 5: Stakeholder Pain Points and Common Recommendations</strong></p><figure><img alt="Business Stakeholder pain points include regulatory costs, interoperability, and small business realities, common recommended HealthIT policy solutions are outlined." src="https://cdn-images-1.medium.com/max/1024/1*G-8kJ7WncMyGEtixTfBSAg.png" /><figcaption>Business Stakeholder Pain Points and Common HealthIT Policy Recommendations</figcaption></figure><h4>eCR and Public Health AI</h4><p>Firms specializing in <strong>automated disease surveillance systems</strong> should monitor National eCR standards to avoid fragmented developer rework. Additionally, monitoring CDC funding allocations for STLT infrastructure is important because it may identify areas where applicants and their partners could unlock <strong>Medicaid rebates</strong> for compliant tools.</p><h4>Value-Based Care &amp; AI Quality Measurement</h4><p>With NCQA leading FHIR adoption, startups that align their AI models with <strong>digital quality standards</strong> may be well-positioned to capture Medicare reimbursement benefits.</p><h4>Telehealth AI: Scalability Post-PHE</h4><p>Investors eyeing behavioral health or remote patient monitoring AI will likely continue to advocate for <strong>Platform-level compliance frameworks</strong> to reduce HIPAA burdens. Another telehealth policy priority is <strong>Interstate licensure parity, </strong>which aims to expand market reach.</p><h3>Conclusion: Forging a New Regulatory Framework</h3><p>These submissions collectively highlight three critical insights about the current health IT regulatory landscape. First, the FDA and ONC’s overlapping jurisdictions create duplicative oversight, particularly for clinical decision support tools, resulting in delayed AI deployment and innovation. Second, permanent telehealth flexibilities are now essential for developing and scaling AI-driven remote care solutions. Third, without robust enforcement of data transparency requirements, patient distrust of AI technologies will continue to hinder adoption and effectiveness.</p><p>The path forward requires <strong>collaboration among agencies,</strong> not siloed rulemaking. HHS’s response to these submissions will determine whether the U.S. maintains its global leadership in health AI innovation. As stakeholders await HHS’s implementation plan, executives, investors, and AI policy advocates should:</p><ul><li><strong>Engage proactively with PCCP 513(g) reforms</strong> to mitigate risks associated with iterative AI development.</li><li><strong>Prioritize patient-centered, FHIR-compliant tools</strong> that align with NCQA/ONC standards.</li><li><strong>Monitor the FDA’s AI guidance</strong> for clarity on the distinction between high-risk and low-risk AI tools.</li></ul><p>Moving forward requires balancing innovation with safety — a challenge best addressed through <strong>sector-specific solutions,</strong> such as EHRA’s proposed CDS carve-outs and <a href="https://health-vision.ai/">Health-Vision.AI</a>’s recommendations. For health IT leaders, the coming months will determine whether the U.S. remains a global hub for health AI or cedes ground to other countries as our regulatory frameworks continue to lack the necessary clarity.</p><h3>References</h3><ul><li><a href="https://www.ropesgray.com/en/insights/alerts/2014/04/fda-onc-and-fcc-issue-draft-report-on-the-regulation-and-oversight-of-health-it-technologies">FDA, ONC, and FCC Issue Report on the Regulation and Oversight of Health IT Technologies | Insights | Ropes &amp; Gray LLP</a></li><li><a href="https://www.thefdalawblog.com/2022/10/the-incredible-shrinking-exemption-fda-final-cds-guidance-would-significantly-narrow-the-scope-of-exempt-clinical-decision-support-software-under-the-cures-act/">The Incredible Shrinking Exemption: FDA Final CDS Guidance Would Significantly Narrow the Scope of Exempt Clinical Decision Support Software Under the Cures Act</a></li></ul><p><a href="https://www.ehra.org/sites/ehra.org/files/EHR%20Association%20Response%20to%20HHS%20RFI%E2%80%8A-%E2%80%8AJuly%202025_0.pdf">EHRA Response to DHHS, July 14, 2025</a></p><p><a href="https://www.healthcareitnews.com/news/ehra-asks-hhs-improve-regulatory-alignment-rfi-response">EHRA asks HHS to improve regulatory alignment in RFI response</a></p><p><a href="https://advocacy.sba.gov/wp-content/uploads/2025/07/Comment-Letter_Advocacy-Comments-to-HHS-Deregulatory-RFI.pdf">U.S. Small Business Administration response to DHHS, July 14, 2025</a></p><p><a href="https://www.americantelemed.org/wp-content/uploads/2025/07/ATA-Action-HHS-FDA-RFI-Response-Final-7.14.25.pdf">American Telemedicine Association response to DHHS, July 14, 2025</a></p><p><a href="https://www.advamed.org/wp-content/uploads/2025/07/AdvaMed-Comments-to-HHS-Deregulation-RFI-7-14-2025.pdf">AVAMed response to DHSS, July 14, 2025</a></p><ul><li><a href="https://www.aha.org/lettercomment/2025-07-14-aha-comments-hhs-rfi-maha-initiative">AHA Comments on HHS RFI on MAHA Initiative | AHA</a></li><li><a href="https://www.ncqa.org/comment-letter/ncqa-responds-to-hhs-health-technology-ecosystem-rfi/">NCQA Responds to HHS Health Technology Ecosystem RFI - NCQA</a></li></ul><p><a href="https://essentialhospitals.org/wp-content/uploads/2025/07/FINAL-HHS-Deregulatory-RFI-July-2025-Unsig.pdf">America’s Essential Hospitals Letter to DHHS, July 14, 2025</a></p><p><a href="https://www.aamc.org/media/84956/download?attachment">Association of American Medical Colleges, Letter to DHHS, July 2025​</a></p><p><a href="https://connectedhi.com/chi-healthcare-and-artificial-intelligence-sign-on-letter-to-u-s-secretary-of-health-and-human-services-robert-f-kennedy-jr/">Connected Health Initiative, Letter to DHHS, July 14, 2025</a></p><p><a href="https://medium.com/responsible-healthcare-ai/why-disruptive-ai-wont-work-in-healthcare-db484f9d4071">Why “Disruptive” AI Won’t Work in Healthcare</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b93ec4aa2c4e" width="1" height="1" alt=""><hr><p><a href="https://medium.com/responsible-healthcare-ai/health-it-regulatory-crossroads-implications-for-ai-driven-innovation-b93ec4aa2c4e">Health IT Regulatory Crossroads: Implications for AI-Driven Innovation</a> was originally published in <a href="https://medium.com/responsible-healthcare-ai">Responsible Healthcare AI</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Why an Ethical Evaluation Framework for AI Agents Is Essential in the Era of Multi-Agent Solutions]]></title>
            <link>https://medium.com/responsible-healthcare-ai/why-an-ethical-evaluation-framework-for-ai-agents-is-essential-in-the-era-of-multi-agent-solutions-be93c7b5c5ce?source=rss----fea69b1bd593---4</link>
            <guid isPermaLink="false">https://medium.com/p/be93c7b5c5ce</guid>
            <category><![CDATA[ai-ethics]]></category>
            <category><![CDATA[digital-health]]></category>
            <category><![CDATA[agentic-ai]]></category>
            <category><![CDATA[generative-ai-tools]]></category>
            <category><![CDATA[ai-governance]]></category>
            <dc:creator><![CDATA[Brian M. Green]]></dc:creator>
            <pubDate>Wed, 16 Jul 2025 16:23:10 GMT</pubDate>
            <atom:updated>2025-07-16T02:17:41.842Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="Person reviewing a large screen monitor that shows several alerts shown with red flags and descriptions of the alerts." src="https://cdn-images-1.medium.com/max/1024/1*HZUBsSU3yzxHOHp5gPPcIg.png" /><figcaption>Image created by author using DALL-E3</figcaption></figure><p>As AI agents become more autonomous, ensuring they behave ethically and transparently is crucial for building public and stakeholder trust. Multi-agentic AI systems and multi-model agents expand the potential risks, the threat matrix, and the attack surface, making the need for ethical and robust governance all the more apparent.</p><p>Our proposed <strong><em>Ethical Evaluation Rubric for AI Agents</em></strong> directly addresses this challenge by providing a structured way to systematically assess the trustworthiness, safety, and societal impact of AI agents throughout their lifecycle.</p><p>Multi-agent AI systems introduce distinct ethical complexities that differ from those of single-agent or conventional AI tools. They involve multiple autonomous agents interacting, collaborating, or competing, which creates new forms of risk, emergent behavior, and accountability concerns. The rubric we have developed is designed to address these multi-layered challenges by operationalizing ethical principles for design-time, testing, and continuous assessment.</p><h3>The Need for a Purpose-Built Ethical Rubric</h3><p><strong>Agentic AI</strong> and multi-agent solutions introduce unique complexities that traditional (legacy) evaluation methods do not fully address:</p><ul><li><strong>Autonomy and Unpredictability</strong>: Modern AI agents are designed to make independent decisions, execute tasks, and adapt in unexpected ways.</li><li><strong>Emergent Behaviors</strong>: Multi-agent systems often exhibit collective behaviors with ethical implications that are difficult to foresee in isolated testing environments.</li><li><strong>Continuous Learning and Deployment</strong>: Many AI agents adapt or degrade relative to their guardrails post-deployment, raising the stakes for ongoing monitoring and ethical assurance.</li></ul><p>Without a nuanced, well-defined rubric, organizations are left to ad hoc approaches that may overlook crucial risks or fail to provide actionable insights.</p><h3>How Our Ethical AI Evaluation Framework Adds Value</h3><h4>Pre-Deployment &amp; Integration Assessment</h4><ul><li><strong>Purpose-Built Ethics</strong>: Guiding teams to embed consent, fairness, privacy, and transparency considerations from the earliest design phases.</li><li><strong>Risk-Benefit Analysis</strong>: Structured scoring and weighting help prioritize mitigations according to context, domain, and risk tolerance.</li><li><strong>Regulatory Mapping</strong>: Ensures alignment with frameworks like the EU AI Act, ISO 42001, NIST AI-RMF, and healthcare-specific requirements through explicit checkpoints.</li></ul><h4>Model Testing &amp; Evaluation</h4><ul><li><strong>Bias and Performance Audits</strong>: Disaggregated metrics and fairness checks can be executed to ensure equitable outcomes across various demographics or group segments.</li><li><strong>Transparency and Explainability</strong>: Calls for source citations, input-saliency, and contextual and layered explanations make AI agents more accountable and trustworthy.</li><li><strong>Safety Triggers</strong>: Provides a checklist for harm mitigation and simulation testing before real-world exposure.</li></ul><h4>Post-Deployment Continuous Evaluation</h4><ul><li><strong>Monitoring and Iterative Improvement</strong>: Dashboards and audit logs track real-world performance, surfacing drift or emergent issues for automated or HITL action depending upon risk scoring.</li><li><strong>Incident Response</strong>: Regular reviews and clear escalation paths ensure that problems are identified early and addressed appropriately.</li><li><strong>Sustainability and Societal Impact</strong>: Ensures impact reporting to keep teams accountable to internal stakeholders, users, as well as the broader community and regulators.</li></ul><figure><img alt="Table presenting ethical evaluation framework for AI Agents. Includes 10 ethical principles, description of each item, and recommended example evidence for each principle. Health-VisionAI copyrighted evaluation rubric." src="https://cdn-images-1.medium.com/max/1024/1*aLryE5UsArg5HKOkKaj68A.png" /><figcaption>AI Ethical Evaluation Rubric, ©2025, Health-Vision.AI, LLC</figcaption></figure><h3>Empowering Responsible AI at Every Stage</h3><p>Adopting a per-item <strong>scoring key (1–5)</strong> and flexible weighting schema brings rigor and adaptability to our ethical AI evaluation rubric. Whether your team is integrating a new agent, testing a collaborative AI Agentic workflow, or monitoring live deployments, our framework enables:</p><ul><li>Evidence-based decision-making and prioritization</li><li>Clear communication of risk and readiness to both technical and non-technical audiences</li><li>Iterative refinement to build public trust and deliver measurable, beneficial AI solutions</li></ul><p>By approaching ethical evaluation as a dynamic, multi-stage discipline, rather than a one-and-done checklist, this framework helps you position your organization to lead responsibly as you develop and deploy agentic AI solutions.</p><h3>Using the Ethical Evaluation Rubric for AI Agent Evaluation</h3><h4>Step 1: Define Evaluation Scenarios</h4><p><strong>Identify Relevant Scenarios</strong>: Determine the specific applications or tasks for which the AI agent will be used. For example, healthcare diagnosis, financial forecasting, or customer service. For a multi-agentic workflow, isolate the task-specific and desired executive functions within the CI/CD pipeline. Agent-specific metrics may include: intent matching to the user request, accuracy of tool calling, and task adherence.</p><p><strong>Create Representative Test Cases</strong>: Develop a set of test cases that cover a variety of scenarios, including edge cases and potential adversarial inputs. Ensure that these test cases accurately represent real-world usage.</p><h4>Step 2: Apply the Ethical Evaluation Rubric</h4><p>For each test case, evaluate the AI agent’s performance against the 10 principles outlined in the Health-Vision.AI rubric (we show the first 3 out of 10 below).</p><p><strong>Respect for Persons / Autonomy</strong>:</p><ul><li>Assess whether the AI respects user autonomy and decisions. For example, does it provide clear choice paths or allow for HITL overrides?</li></ul><p><strong>Beneficence and Non-Maleficence</strong>:</p><ul><li>Evaluate the AI’s potential to cause harm or benefit. Consider both direct impacts (e.g., medical diagnosis accuracy) and indirect impacts (e.g., psychological effects of interactions).</li></ul><p><strong>Justice and Fairness</strong>:</p><ul><li>Check for biases in the AI’s outputs across different demographic groups or defined user segments. Use fairness metrics and diverse test datasets with documented lineage.</li></ul><h4>Step 3: Gather Evaluation Data</h4><p><strong>Quantitative Data</strong>: Use metrics and automated tests to gather data on the AI’s performance against the rubric principles. For example, accuracy rates, fairness metrics, or privacy compliance checks.</p><p><strong>Qualitative Data</strong>: Collect qualitative feedback from users or experts regarding the AI’s performance, especially on aspects that are difficult to quantify and require SME for context, such as respect for autonomy or transparency.</p><h4>Step 4: Analyze and Interpret Results</h4><p><strong>Compare Against Thresholds</strong>: Compare the gathered data against predefined thresholds or benchmarks for each principle, for example, your baseline scores.</p><p><strong>Identify Areas for Improvement</strong>: Highlight areas where the AI agent does not meet or scores lower than the desired ethical standard metrics.</p><p><strong>Prioritize Improvements</strong>: Based on the analysis, prioritize areas for improvement, considering both the severity of the issue and the potential impact of addressing it, while leveraging input from stakeholder groups.</p><h4>Step 5: Implement Changes and Monitor</h4><p><strong>Adjust the AI System and Agent Tasking</strong>: Make necessary adjustments to the AI system based on the evaluation results. This might involve retraining the model, adjusting algorithms, or implementing additional safety measures.</p><p><strong>Continuous Monitoring</strong>: Regularly re-evaluate the AI system against the ethical evaluation rubric as AI model changes are implemented, AI technology and datasets evolve, to ensure ongoing compliance with ethical principles.</p><h4>Step 6: Integrate with RLHF Methodologies</h4><p><strong>Reinforcement Learning from Human Feedback (RLHF)</strong>: Use the insights gained from the ethical evaluation to inform the RLHF process. This can involve incorporating ethical considerations into the reward function or using the evaluation rubric to score the level of risk and prioritize to guide the selection of escalation for human feedback.</p><p><strong>Iterative Improvement</strong>: Continuously iterate on the AI system, using the ethical evaluation rubric to guide improvements and ensure that the system remains aligned with ethical principles as the AI system, AI agents, and regulatory landscape evolve.</p><h3>Conclusion:</h3><p>By systematically applying this ethical evaluation rubric across the AI development lifecycle, you can comprehensively assess AI agents and guide their development towards more ethical and responsible AI systems.</p><p>The rubric demands <strong><em>traceable decisions, data lineage, and clear model updates</em></strong>, ensuring users and auditors can understand not only what decisions were made but also why and how they were reached.</p><p>Requiring <em>m</em><strong><em>odel cards, source citations, and explainable outputs</em></strong> helps users understand the reasoning behind agent actions, thereby strengthening their confidence in the agent’s reliability.</p><p>By featuring items such as <strong><em>consent-flow UX, opt-out logs, and data-use disclosures</em></strong>, the rubric ensures that users are empowered to make informed choices and that agents honor those choices in real-time. This enables users to withdraw or refuse consent at any point, directly supporting agency and fostering respectful, trustworthy interactions.</p><p>The rubric’s structure, scoring, weighting, and evidence standards bring clarity, rigor, and accountability to the ethical assessment of multi-agent AI. It empowers developers, integrators, business users, and auditors to systematically surface, monitor, and correct the specific risks that arise from complex agentic systems, moving beyond checklists to ongoing, <strong>Risk-Adaptive Governance</strong>(TM).</p><p>By embedding these safeguards and evidence requirements, this rubric can be used to transform trust and accountability from abstract goals into operationalized, measurable outcomes for Agentic AI systems.</p><h3>References:</h3><ul><li><a href="https://medium.com/responsible-healthcare-ai/governance-first-navigating-the-spectrum-of-ai-concerns-and-risks-in-healthcare-12bb6bce2500">Governance First: Navigating the Spectrum of AI Concerns and Risks in Healthcare</a></li><li><a href="https://pmc.ncbi.nlm.nih.gov/articles/PMC10097940/">Ethical content in artificial intelligence systems: A demand explained in three critical points</a></li><li><a href="https://pmc.ncbi.nlm.nih.gov/articles/PMC10879008/">Ethical and regulatory challenges of AI technologies in healthcare: A narrative review</a></li><li><a href="https://medium.com/responsible-healthcare-ai/responsible-use-of-ai-in-healthcare-strategies-for-balancing-opportunities-and-risk-in-a-b29ab27259b4">Responsible Use of AI in Healthcare — Strategies for Balancing Opportunities and Risk in a…</a></li><li><a href="https://generativeai.pub/turn-up-the-p-or-turn-up-the-k-optimizing-genai-model-outputs-but-for-what-03ba721dfaed">Turn up the P or Turn up the K? Optimizing GenAI Model Outputs, but for What?</a></li><li><a href="https://stars.library.ucf.edu/hut2024/212/">Reinforcement Learning From Human Feedback For Ethically Robust Ai Decision-Making</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=be93c7b5c5ce" width="1" height="1" alt=""><hr><p><a href="https://medium.com/responsible-healthcare-ai/why-an-ethical-evaluation-framework-for-ai-agents-is-essential-in-the-era-of-multi-agent-solutions-be93c7b5c5ce">Why an Ethical Evaluation Framework for AI Agents Is Essential in the Era of Multi-Agent Solutions</a> was originally published in <a href="https://medium.com/responsible-healthcare-ai">Responsible Healthcare AI</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[From Isolation to Inspiration: A Guide to Online Health Communities]]></title>
            <link>https://medium.com/responsible-healthcare-ai/from-isolation-to-inspiration-a-guide-to-online-health-communities-aa1a1847a28d?source=rss----fea69b1bd593---4</link>
            <guid isPermaLink="false">https://medium.com/p/aa1a1847a28d</guid>
            <category><![CDATA[social-media]]></category>
            <category><![CDATA[community-engagement]]></category>
            <category><![CDATA[digital-health]]></category>
            <category><![CDATA[online-community]]></category>
            <category><![CDATA[community-management]]></category>
            <dc:creator><![CDATA[Brian M. Green]]></dc:creator>
            <pubDate>Thu, 09 Jan 2025 05:48:10 GMT</pubDate>
            <atom:updated>2024-07-28T04:09:31.982Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="people in various settings with phones demonstrating connection and healthy communities" src="https://cdn-images-1.medium.com/max/1024/1*1HnMJcH-8hektZaw5XS-bQ.png" /><figcaption>Image created by author using MS Copilot, DALL-E 3</figcaption></figure><p>Have you ever found yourself overwhelmed by medical terminology, uncertain about a new diagnosis, or simply yearning for a sense of community with others facing a similar health challenge?</p><p><strong>If so, you are not alone.</strong></p><p>The internet has revolutionized how we connect and engage with others, and online health communities (OHCs) stand as a testament to the power of virtual support systems.</p><p>In this article, I provide some helpful information about OHCs and how they are not all created equal. I also outline the evolution of OHCs over time and the impact that technologies like generative AI may have on their future.</p><blockquote><strong><em>Q: With so many OHCs available, where do you start? A: Selecting the right OHC for you is crucial, so here are some tips!</em></strong></blockquote><h3>How Do I Find An Online Health Community?</h3><p>Finding the right online health community can be challenging, and knowing what you are looking for is one of the first things to consider.</p><p>You’ll want to consider your privacy needs and whether you want a community that is open to the public or “closed” to members only. Paid or free membership is often a consideration. But before you pay, you should know that you can find high-quality free communities online.</p><ul><li><strong>What Are You Looking For in an OHC?</strong>: Determine whether you seek information on a specific health condition, a particular part of your healthcare journey, emotional support and interaction with your peers, or a combination of these.</li><li><strong>Explore different platforms:</strong> OHCs exist on various platforms, including social media groups, dedicated websites, and online forums. Explore different formats to find one that aligns with your preferences. As I discuss below, there may be an integrated OHC for you that combines all of these features.</li><li><strong>Assess the community atmosphere:</strong> Look for communities with a positive and respectful environment moderated by a team of moderators who are not anonymous and where moderators ensure reliable information exchange and respect for community rules.</li></ul><p>While OHCs offer valuable insights and support, consulting qualified healthcare providers for diagnosis, treatment plans, and medical recommendations remains essential. Good OHCs will have links to vetted resources that do not have a “pay to get” or solicit personal information beyond what you would have to submit to be a verified community member.</p><blockquote><strong><em>Remember: OHCs are not a substitute for professional medical advice.</em></strong></blockquote><p>Now that you know how to find an OHC that meets your needs, how did we get here? You may have a local in-person health group that meets your needs, or you may want the flexibility and convenience of an online group.</p><h3>What Are Online Health Communities?</h3><p>Simply stated, Online Health Communities serve as online spaces where individuals with shared health interests exchange information, provide emotional support, and connect with others facing similar challenges.</p><p>These communities may cater to a wide range of participants, including individuals with specific health conditions, caregivers, and even healthcare professionals who are sometimes on the same platform. OHCs can be disease-specific, focusing on conditions like cancer or chronic illnesses, or, more general, encompassing overall wellness and a range of mental health topics.</p><p>While some companies view OHCs as extensions of their brands, their value should transcend content marketing and brand promotion goals. If you think an OHC is too strongly associated with a single product, like a branded drug or medical device, that may not be the right community for you. There is most likely an OHC that meets your needs, and you can try one out and try another one if one doesn’t work out.</p><h3>The Power of Connection in OHCs</h3><p>There are many different <a href="https://www.jmir.org/2020/7/e17338/">benefits of engaging</a> with Online Health Communities. By very definition, these platforms are designed to provide a space for members to:</p><ul><li><strong>Learn about health conditions</strong>: Medically vetted information should be a core offering of the OHC. But you may be looking for stories and experiences from people like you who live with a specific health condition. Sharing personal experiences, treatment insights, and reliable resources can deepen the OHC experience, and these should be features of a well-managed Community you find online.</li><li><strong>Find emotional support</strong>: Connecting with individuals who empathize with your struggles and victories can alleviate feelings of isolation and empower you on your health journey. Research has shown that most people first come to an OHC to find specific information, often treatment or medication information, but <a href="https://www.mdpi.com/1660-4601/18/23/12743">stay engaged long-term</a> because of the support they find in a well-managed OHC.</li><li><strong>Exchange practical tips</strong>: From managing symptoms to navigating healthcare appointments and other aspects of your journey, OHC members can offer valuable advice based on their firsthand experiences. A well-managed OHC will ensure these are moderated, not spammy, or promoting “fake cures.”</li></ul><h3>What Makes a Community? Key Elements of OHCs</h3><p>OHCs are designed to “mirror” the features of in-person communities or larger-scale social groups. Their key features encompass social support, engagement features that can be adapted to different situations and contexts, and active moderation.</p><p>Each of these features helps create a comprehensive support system for members. These elements are often intertwined, creating a structure that reinforces the community’s functionality and impact. These core community functions and impact include:</p><ul><li><strong>Social Identity: </strong>OHCs cultivate a sense of social identity by fostering a shared purpose and belonging among members. Connecting based on common health conditions or experiences is particularly meaningful for individuals with rare diseases or unique health challenges that come with many chronic health conditions.</li><li><strong>Social Relationships:</strong> Within OHCs, social relationships are important, providing a unique and safe space for connection and support. Shared struggles can foster empathy and camaraderie among members, building robust emotional support networks that significantly influence well-being.</li><li><strong>Emotional Connection: </strong>The emotional connection forms the foundation of thriving OHCs, fueling engagement and nurturing a sense of community. Shared experiences, empathy, and mutual support drive this connection, creating a supportive environment for all members. For this reason, moderators can start by validating members&#39; concerns and feelings, which often goes far in diffusing any tensions or misunderstandings that can occur online.</li><li><strong>Interdependence: </strong>Interdependence within OHCs signifies that members rely on each other for information, support, and maintaining the community’s norms and dynamics. Community Rules, for example, can be a formal mechanism that supports interdependence, but many of these rules are self-enforcing. This mutual reliance fosters a sustainable, diverse, and collectively knowledgeable space for all participants.</li></ul><figure><img alt="Timeline showing the evolution of online health communities from online bulletin boards, social media, online communities with community managers, and AI-assisted OHCs." src="https://cdn-images-1.medium.com/proxy/0*CoHBYkRtr5mCrnyY" /></figure><h3>The Evolution of Online Health Communities</h3><h4>Pre-Social Media: Bulletin Boards and Early Forums</h4><p>For those of us who remember pre-social media community spaces, these often were text-based forums and bulletin board-style threaded chats with limited interaction capabilities outside of threaded conversations.</p><p>Early bulletin board systems (BBS) for online communities faced several challenges that hindered their effectiveness and user experience. One significant issue was the limited accessibility and connectivity, as users often needed specialized equipment or slow dial-up modem-based connections to access BBS platforms. This restricted the reach and participation of community members, leading to smaller and less diverse user bases.</p><p>The lack of standardized interfaces and user-friendly designs made navigation and interaction cumbersome for BBS users. Limited tools and mechanisms to regulate discussions led to potential conflicts, misinformation, and inappropriate behavior among community members. These challenges underscored BBS&#39;s early struggles in fostering inclusive and engaging online communities, highlighting the importance of evolving technologies and practices to overcome these limitations.</p><p>Despite their constraints, these communities offered crucial peer support to individuals with rare diseases or socially stigmatized conditions. Some of the first of these were for HIV-positive or “AIDS activist” communities, LGBTQ people seeking affirming care, and cancer support communities.</p><h4>The Rise of Social Media</h4><p>The advent of social media platforms like Myspace marked a significant milestone in the evolution of Online Communities. Myspace, one of the earliest social networking sites, allowed users to create personalized profiles, connect with others based on shared interests, and engage in discussions on various topics, including health and wellness.</p><p>However, Myspace lacked robust moderation mechanisms, leading to challenges maintaining a safe and supportive environment for health-related discussions. The platform’s open nature allowed for diverse interactions but raised concerns about the quality and accuracy of health information shared within the community.</p><p>Newer social media platforms like Facebook and Reddit began with limited moderation as a key feature. Reddit, however, utilizes volunteer mods that have limited moderation controls within a sub-community they create.</p><p>Regardless, the “explosion” of these and other social media platforms with mobile-first designs revolutionized OHCs, making them more accessible and interactive. Enhanced features such as multimedia sharing, chat functions, passive “like” or “reaction” buttons, and private messaging facilitated deeper connections and richer interactions among members.</p><h4>The Role of the Community Manager</h4><p>As Online Health Communities evolved, the role of the community manager emerged as pivotal in ensuring the smooth functioning and growth of these digital spaces.</p><p>Community managers are crucial in introducing new discussion topics, fostering engagement, moderating discussions, resolving conflicts, and upholding community guidelines. Depending on the company or platform, Community Managers may also serve as content marketing specialists, social media managers, membership managers, and other roles.</p><p>Community Managers act as platform facilitators, ensuring members feel heard, supported, and valued. By nurturing a positive and inclusive environment, community managers can contribute significantly to the overall success, growth, and sustainability of Online Health Communities.</p><h4>Cross-Platform Connections and Challenges</h4><p>As social media platforms continued to proliferate, the issue of cross-platform connections emerged as a notable challenge for Online Health Communities. While platforms like Reddit or Facebook offer in-depth discussions and support groups for specific health conditions, the question of sharing such content seamlessly across platforms like TikTok or Instagram posed logistical and content adaptation challenges.</p><p>The dynamic landscape of social media presents opportunities and obstacles for mature Online Health Communities. Community managers and members must adapt to changing trends, explore new engagement strategies, and navigate the complexities of cross-platform communication to maximize the reach and effectiveness of health-related discussions and support networks.</p><p>The differences in format, audience demographics, and engagement styles across social media platforms necessitated innovative strategies to ensure that valuable health-related content could reach a broader audience without losing its relevance or impact.</p><p>Each social media’s proprietary algorithm frequently changes and requires constant adjustment of organic engagement and paid tactics. At times of rapid change or when multiple channels roll out updates simultaneously, smaller OHCs that do not have budgets or other resources may be unable to adjust and maintain thriving community spaces.</p><figure><img alt="Diverse people shown engaging with each other using phones online with healthcare symbols and icons in the background." src="https://cdn-images-1.medium.com/max/1024/1*8PAXq46Z6SKcjIi9_r1I5w.png" /><figcaption>Image created by author using MS Copilot, DALL-E 3</figcaption></figure><h3>Integrated Community Ecosystems</h3><p>Health Union is a leading company that develops and manages over 46 OHCs and provides an example of an <strong>integrated community ecosystems model</strong>. This <a href="https://www.jmir.org/2020/7/e17338/">model exemplifies the evolution</a> of Online Health Communities towards comprehensive and interconnected platforms that blend expert content, user-generated content (UGC), community engagement features, and robust moderation.</p><p>For example, platforms like <a href="https://multiplesclerosis.net/">MultipleSclerosis.net</a> and <a href="https://raredisease.net/">RareDisease.net</a> within the <a href="https://health-union.com/communities/">Health Union network of OHCs</a> offer a multifaceted approach to community building and support. Key features of this integrated community ecosystem include:</p><ul><li><strong>Expert Content and UGC Integration</strong>: Integrated community ecosystems combine professionally written and curated content with user-generated contributions, fostering a rich and diverse knowledge base that caters to community members&#39; varied needs and interests. This blend of expertise and personal experiences enhances the depth and relevance of information shared within the community.</li><li><strong>Community Engagement Features</strong>: Platforms like MultipleSclerosis.net incorporate interactive features that mirror real-world interactions, such as forums, chat functions, and supportive sub-groups (tailored segments). These engagement tools facilitate meaningful connections, peer support, and knowledge exchange among community members, fostering a sense of belonging and empowerment.</li><li><strong>Community Moderation</strong>: Effective moderation is a cornerstone of integrated community ecosystems, ensuring that discussions remain respectful, informative, and supportive. Community moderators play a vital role in upholding community guidelines, resolving conflicts, and creating a safe space for all participants to share their experiences and insights.</li><li><strong>Integration with Social Media Channels: </strong>Integrated community ecosystems extend their reach and impact by integrating with popular social media channels like Facebook, TikTok, etc. For instance, a Multiple Sclerosis community page on Facebook extends the main community platform, allowing for seamless organic cross-platform engagement and content sharing. Community members can post links to their own social media channels, share relevant content both on and off the platform, and amplify the community’s presence across various digital spaces.</li></ul><blockquote><strong><em>By embracing integrated community ecosystems, OHCs like those within the Health Union network create holistic and interconnected platforms that combine expert knowledge, user contributions, community engagement features, and social media integration to foster a supportive, informative, and inclusive environment for individuals navigating health challenges.</em></strong></blockquote><h3>The Potential Impact of Generative AI for OHCs</h3><p>GenAI, a fusion of tailored large language models and learning algorithms, can <a href="https://www.linkedin.com/pulse/leveraging-ai-online-health-communities-enhancing-patient-green-n79ee/?trackingId=%2F1nhix9UQYeq7tsFFTy8sQ%3D%3D">enhance OHCs by optimizing community management</a> through data-driven insights, personalized content delivery, and user engagement.</p><p>For example, AI chatbots could offer introductory support and question-based moderation triage, with human moderators jumping in when specific questions are marked priority or to tailor a response. AI-driven content curation could ensure the accuracy and reliability of UGC-shared information.</p><p>GenAI can empower community managers to adapt community engagement features more quickly, provide tailored content, and offer targeted interventions to effectively meet community members&#39; evolving needs. Integrating GenAI technologies in OHCs can enhance the user experience, foster community engagement, and create more personalized resources for individuals navigating health challenges.</p><h3>Challenges and Considerations</h3><p>Ethical considerations such as <a href="https://medium.com/@BrianMGreen/responsible-use-of-ai-in-healthcare-strategies-for-balancing-opportunities-and-risk-in-a-b29ab27259b4">data privacy, bias mitigation, and fairness</a> in AI algorithms are paramount, just as they are in OHC management. Developing AI models with diverse health datasets is essential to prevent the exclusion of underserved communities.</p><p>As stated at the beginning of this article, with the plethora of OHCs online, finding the right one may be challenging. When looking at OHCs with AI enhancements, consider the Privacy Policy, the transparency and explainability of AI features, and other community guidelines when deciding.</p><p>Online Health Communities have redefined how individuals interact, share knowledge, and support each other within the healthcare domain. These virtual spaces facilitate discussions on health concerns, information exchange, and mutual encouragement among patients, caregivers, and healthcare professionals.</p><p>Being an informed and responsible community member is essential for thriving online health communities. Whether you are seeking support or sharing your experiences, active participation in OHCs can be immensely beneficial. <a href="https://health-union.com/blog/conversations-reduce-stigma-mental-health/">Emotional health is linked to your physica</a>l health and well-being. By contributing your thoughts on credible information and your story, fostering empathy, and nurturing a community spirit, you can help create a supportive and enriching online space for others.</p><h3>References and resources</h3><ul><li><a href="https://www.jmir.org/2020/7/e17338/">Assessment of Adaptive Engagement and Support Model for People With Chronic Health Conditions in Online Health Communities: Combined Content Analysis</a></li><li><a href="https://www.mdpi.com/1660-4601/18/23/12743">Come for Information, Stay for Support: Harnessing the Power of Online Health Communities for Social Connectedness during the COVID-19 Pandemic</a></li><li><a href="https://brian-m-green.com/blog/f/prioritizing-patient-centricity-in-genai-healthcare-solutions">Prioritizing Patient-Centricity in GenAI Healthcare Solutions</a></li><li><a href="https://bmchealthservres.biomedcentral.com/articles/10.1186/s12913-023-09469-6">What drives the adoption of online health communities? An empirical study from patient-centric perspective - BMC Health Services Research</a></li><li><a href="https://www.linkedin.com/pulse/leveraging-ai-online-health-communities-enhancing-patient-green-n79ee/?trackingId=%2F1nhix9UQYeq7tsFFTy8sQ%3D%3D">Leveraging AI in Online Health Communities: Considerations for Enhancing Patient Engagement and Digital Marketing Strategy</a></li><li><a href="https://medium.com/@BrianMGreen/responsible-use-of-ai-in-healthcare-strategies-for-balancing-opportunities-and-risk-in-a-b29ab27259b4">Responsible Use of AI in Healthcare — Strategies for Balancing Opportunities and Risk in a…</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=aa1a1847a28d" width="1" height="1" alt=""><hr><p><a href="https://medium.com/responsible-healthcare-ai/from-isolation-to-inspiration-a-guide-to-online-health-communities-aa1a1847a28d">From Isolation to Inspiration: A Guide to Online Health Communities</a> was originally published in <a href="https://medium.com/responsible-healthcare-ai">Responsible Healthcare AI</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Why “Disruptive” AI Won’t Work in Healthcare]]></title>
            <link>https://medium.com/responsible-healthcare-ai/why-disruptive-ai-wont-work-in-healthcare-db484f9d4071?source=rss----fea69b1bd593---4</link>
            <guid isPermaLink="false">https://medium.com/p/db484f9d4071</guid>
            <category><![CDATA[innovation]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[healthcare]]></category>
            <category><![CDATA[2025]]></category>
            <category><![CDATA[disruption]]></category>
            <dc:creator><![CDATA[Brian M. Green]]></dc:creator>
            <pubDate>Sun, 05 Jan 2025 21:52:25 GMT</pubDate>
            <atom:updated>2025-01-06T00:46:48.750Z</atom:updated>
            <content:encoded><![CDATA[<h4>Disruption vs. Strategic Innovation in Guiding Healthcare AI Applications</h4><figure><img alt="Art Deco stylized image of medical symbols with computer chip or tech circuitry in the background." src="https://cdn-images-1.medium.com/max/1024/1*lD2usyxiN9MahZzgaePHFg.png" /><figcaption>Created by author using DAL-E3</figcaption></figure><p>Healthcare and health tech industries have been abuzz with talk of “disruption” and the transformative potential of artificial intelligence (AI), particularly generative AI. While these technologies undoubtedly hold immense promise, the rhetoric surrounding their implementation often oversimplifies the complex realities of healthcare systems.</p><p>This article explores the nuanced landscape of healthcare innovation, arguing for a more measured, strategic approach to integrating AI and other advanced technologies into medical practice.</p><h3>The Allure of Disruption</h3><p>The term “disruption” has become a buzzword in tech circles, often used to describe innovations that radically alter existing markets or create entirely new ones. In healthcare, proponents of disruptive technologies promise revolutionary changes that will dramatically improve efficiency, reduce costs, and enhance patient outcomes. Generative AI, in particular, has been hailed as a potential game-changer. Its ability to analyze vast amounts of unstructured data, generate human-like text, and assist in complex decision-making processes has captured the imagination of healthcare leaders and investors alike.</p><p>The excitement is understandable, given the potential applications across various aspects of healthcare, from administrative tasks to clinical decision support and drug discovery. However, the enthusiasm for disruption often overlooks the unique challenges and responsibilities inherent in healthcare delivery. Unlike many other industries, healthcare deals directly with human lives and well-being, operating under strict regulatory frameworks and ethical guidelines. The cost of failure or unintended consequences in this field can be catastrophic.</p><h3>The Kuhnian Perspective on Scientific Progress</h3><p>To better understand the nature of change in healthcare, it’s useful to consider Thomas Kuhn’s concept of scientific revolutions. Kuhn, a philosopher of science, in 1962 argued that scientific progress occurs through periods of “normal science” punctuated by paradigm shifts or “revolutions.” While Kuhn used the term “revolutions,” his description of scientific progress aligns more closely with what we might call strategic transformations:</p><ol><li><strong>Normal Science:</strong> Most scientific work occurs within established paradigms, making incremental progress.</li><li><strong>Puzzle-Solving</strong>: Scientists work within the established framework to solve specific problems.</li><li><strong>Accumulation of Anomalies</strong>: Scientific revolutions often result from the gradual accumulation of anomalies that the current paradigm struggles to explain.</li><li><strong>Paradigm Shift</strong>: The actual “revolution” is more of a shift to a new paradigm that better explains observed phenomena rather than a sudden, complete overhaul.</li></ol><p>Kuhn’s perspective suggests that even significant changes in scientific understanding are often the result of gradual accumulation of evidence and incremental shifts in thinking rather than sudden, disruptive breaks with the past. This view aligns well with the complex nature of healthcare systems, where change often requires careful maneuvering within existing structures.</p><h3>Fitting AI Into the Puzzle of Healthcare</h3><p>You may have a memory of a 15-tile puzzle game from when you were a kid or similar digital versions that exist today. This classic sliding puzzle is an apt metaphor for the complexity of healthcare innovation, particularly when implementing artificial intelligence (AI) and other advanced technologies.</p><figure><img alt="Sliding puzzle, with 15 tiles in a 4x4 matrix, featuing medical symbols" src="https://cdn-images-1.medium.com/max/1024/1*42SjY2xSJJTOGpVyhYjrdw.png" /><figcaption>Created by author using DALL-E3</figcaption></figure><p>In healthcare, like in this sliding puzzle game, leaders face a fixed frame with limited space to maneuver. Healthcare organizations often operate within tight constraints of resources, regulations, and established workflows, restricting their ability to implement sweeping changes. Each “move” in the innovation process must be carefully considered, as every decision affects the entire system. Just as solving a sliding puzzle requires a methodical approach, healthcare innovation demands a strategic, step-by-step implementation <a href="https://brian-m-green.com/blog/f/is-healthcare-ready-for-a-genai-close-up">rather than abrupt, disruptive changes</a>.</p><p>The interconnected nature of healthcare systems adds another layer of complexity to this puzzle. Implementing new AI technologies impacts various aspects of the organization, from workflows to staff roles and patient care. Leaders must maintain a clear vision of the end goal while remaining adaptable to evolving challenges. This multidimensional approach requires engaging across technological, strategic, operational, and organizational domains, much like considering both horizontal and vertical movements in a sliding puzzle. Ultimately, solving this healthcare innovation puzzle demands patience and persistence, with realistic timelines often spanning 3–5 years due to the complex interplay of stakeholders and systems.</p><h3>The Case for Strategic Innovation in Healthcare</h3><p>Drawing on Kuhn’s insights, we argue that healthcare innovation, particularly in the realm of AI application development and integration, should be approached as a process of <strong>Strategic Transformation</strong> rather than disruptive upheaval. This approach recognizes the complexity of healthcare systems and the need for careful, deliberate, and ethical integration of new technologies.</p><p>Kuhn’s framework highlights that transformation in complex systems, like healthcare, is rarely a rapid, all-encompassing disruption. Instead, meaningful progress often unfolds as incremental advancements within an existing paradigm, punctuated by occasional shifts when innovations prove their utility and alignment with systemic needs. This nuanced understanding of change aligns well with the challenges of implementing AI in healthcare, where each innovation must be carefully integrated into the existing “puzzle” of healthcare delivery.</p><p>Using the sliding puzzle game analogy, technological disruption in healthcare would be akin to pulling all the puzzle pieces out and shoving them back into the frame to get the “final solution” rather than incrementally sliding them into place with the correct order.</p><h3>Bridging Theory and Practice: A Framework for Healthcare AI Integration</h3><p>The potential for disruption is not uniform across technologies. Instead, it varies based on the complexity of the technology, the infrastructure it requires, and the level of change it introduces to existing workflows. To provide a practical perspective, we propose a 2D matrix that categorizes healthcare AI technologies based on their degree of disruption and the complexity of healthcare infrastructure and technology needs.</p><p>This matrix can serve as a strategic guide for evaluating which technologies are ready for integration, which require more foundational groundwork, and how organizations can prioritize investments to balance innovation with sustainability.</p><h3>Healthcare AI Technology, Complexity and Potential for Disruption</h3><p>Here, we explore a variety of envisioned ML/AI applications and integration initiatives in healthcare. As part of the strategy for planning potential innovation projects, we propose an assessment process that is use-case and sensitive to the dynamics of healthcare systems.</p><figure><img alt="Scatterplot showing 15 AI and ML technologies, plotted on X-axis representing degree of disruption potential and Y-axis representing relative difficulty of implementation." src="https://cdn-images-1.medium.com/max/1024/1*3Y8zXZk5Q9YvtZIYLjIPlQ.png" /><figcaption>Scatterplot created by Author using GoogleSheets graphing</figcaption></figure><p><strong><em>Figure 1</em></strong> highlights the varying demands placed on healthcare organizations based on the degree of likely technological disruption and infrastructure complexity from each listed AI technology integration. This scatter plot provides a two-dimensional representation of healthcare AI technologies, focusing on the relationship between their degree of disruption and the complexity of healthcare infrastructure and technology needs required for their deployment.</p><p>This matrix can be viewed as a representation of the sliding puzzle of healthcare innovation, where each technology represents a piece that must be carefully maneuvered into place within the existing healthcare landscape. Implementing these technologies should be approached strategically, considering their potential impact and the resources required for successful integration.</p><h3>Low Potential for Disruption, Lower Complexity</h3><p><strong>RPA (Robotic Process Automation)</strong> has low disruptive potential as it typically automates routine, rule-based tasks without significantly altering existing workflows. It’s relatively simple to implement, often integrating well with existing systems and tech infrastructure.</p><p><strong>Basic ML/Analytics</strong> tools have low disruptive potential as they often enhance existing processes rather than replace them. They’re generally straightforward to implement, leveraging existing data infrastructure.</p><h3>Medium Potential for Disruption, Moderate Complexity</h3><p><strong>Robust AI Governance</strong>: While not directly disruptive to clinical practices, implementing <a href="https://medium.com/responsible-healthcare-ai/responsible-use-of-ai-in-healthcare-strategies-for-balancing-opportunities-and-risk-in-a-b29ab27259b4">robust AI governance</a> can significantly impact organizational processes and decision-making. It requires careful planning and integration across various departments.</p><p><strong>Predictive AI</strong> has moderate disruptive potential as it can significantly enhance decision-making processes. Implementation complexity is moderate, requiring integration with existing systems and workflows.</p><p><strong>Basic AI integrations with Electronic Health Records</strong> (EHRs) have moderate disruptive potential, enhancing existing systems rather than completely overhauling them. Implementation complexity is moderate, requiring careful integration with existing EHR systems.</p><p><strong>RAG (Retrieval-Augmented Generation)</strong> systems have moderate disruptive potential, enhancing information retrieval and generation capabilities. Implementation complexity is moderate, requiring integration with existing knowledge bases and systems.</p><p><strong>Intermediate AI solutions</strong>, falling between basic and advanced implementations, have moderate disruptive potential and complexity, depending on the specific application.</p><p><strong>Cybersecurity Enhancements for AI.</strong> While not directly disruptive to clinical practices, implementing <a href="https://www.linkedin.com/pulse/ai-crosshairs-understanding-genais-impact-brian-m-green-zspxe/">AI-enhanced cybersecurity</a> can significantly impact organizational processes. It requires careful integration with existing security infrastructure.</p><p><strong>IoT Integrations</strong> have moderate disruptive potential, introducing new data sources and monitoring capabilities. Implementation complexity is moderate, requiring integration with existing systems and new device management.</p><p>Many healthcare organizations, like hospitals or ambulatory care centers, already use predictive AI for specific functions and have IoT integrations for some devices. Enterprise-wide integrations and expansion of predictive AI with advanced generative AI outputs, like triage systems, will push the complexity assessment and disruption potential further up the scales.</p><h3>High Potential for Disruption, High Complexity</h3><p><strong>GenAI Applications</strong> have high disruptive potential, transforming various aspects of healthcare from diagnosis to treatment planning. <a href="https://medium.com/responsible-healthcare-ai/governance-first-navigating-the-spectrum-of-ai-concerns-and-risks-in-healthcare-12bb6bce2500">Implementation complexity is high</a>, requiring significant infrastructure and workflow changes.</p><p><strong>Agentic AI</strong> (Multi-agent) has high disruptive potential, potentially revolutionizing complex decision-making processes in healthcare. Implementation complexity is high, requiring sophisticated infrastructure and integration.</p><p><strong>Advanced AI integrations with EHRs</strong> have high disruptive potential, potentially transforming how healthcare providers interact with patient data and make clinical decisions. Implementation complexity is high, requiring significant changes to existing EHR systems and workflows.</p><p>While these technologies may appear to be fairly straightforward in the pilot stage, getting them to scale requires significant infrastructure investments and will have an increased disruptive potential with enterprise-wide implementation.</p><h3>Deployment Models and Healthcare Contextual Factors</h3><p>Another consideration is the deployment models that would ideally be implemented when integrating these specific technologies into a healthcare system. Most healthcare organizations already operate with a mix of on-premise, cloud-based, and hybrid tech stacks and data solutions. New AI deployments and integrations will necessitate reevaluating this mix to <a href="https://www.linkedin.com/pulse/taming-cost-beast-cost-containment-strategies-genai-healthcare-green-q8qme/">determine the optimal balance of resources</a> required.</p><p>It is important to note that this visualization is based on assumptions about how healthcare organizations, on average, might deploy these technological solutions and integrations. For larger and more complex healthcare organizations — such as those with multiple campuses or geographically dispersed sites — the distribution of technologies across this matrix may shift. For example:</p><ul><li><strong>Infrastructure and Deployment Variability</strong>: Larger organizations may lean more heavily on hybrid models to accommodate diverse IT environments and ensure compliance with local data regulations.</li><li><strong>Adoption of High-Disruption Technologies</strong>: These organizations might prioritize investments in cloud-based solutions for scalability while also maintaining on-premise components for critical systems, databases, and backups.</li></ul><p>These considerations further illustrate the complexity of the healthcare innovation puzzle, where each piece must fit within the organization and broader patient population mix, staffing resources, and regulatory and technological landscapes.</p><h3>Gradual Paradigm Shifts vs. Disruption</h3><p>The concept of gradual paradigm shifts is particularly relevant to healthcare, where abrupt changes can pose significant risks. The healthcare sector, with its complex regulatory environment, established practices, and critical focus on patient safety, is not conducive to sudden, disruptive changes. Also, for good reason, healthcare providers are often resistant to changes that involve adopting new technologies or externally-developed systems.</p><p>For generative AI to be successfully integrated into healthcare, we should approach it as a strategic innovation rather than a disruptive force. This approach involves:</p><ol><li><strong>Integration with Existing Systems</strong>: AI applications must work seamlessly with current electronic health records (EHRs) and other healthcare technologies and workflows.</li><li><strong>Ethics and Patient-Centricity</strong>: AI tool development must have an ethical foundation and guardrails, and their implementation must adhere to regulatory guidelines and <a href="https://brian-m-green.com/blog/f/prioritizing-patient-centricity-in-genai-healthcare-solutions">prioritize patient needs</a>, privacy, and safety.</li><li><strong>Incremental Deployment</strong>: A phased approach allows for careful model evaluation, iteration, and adjustment of data inputs at each stage across the AI lifecycle.</li></ol><p>Not surprisingly, this strategic approach also aligns with the sliding puzzle metaphor, where each innovation must be carefully positioned within the existing healthcare framework.</p><h3>The Case for Strategic AI Innovations in Healthcare</h3><p>Adopting a strategic transformation approach to AI implementation offers a nuanced and sustainable path forward. Rather than chasing the latest AI technology for its own sake, this approach focuses on aligning AI initiatives with organizational needs, operating environment, and capabilities.</p><h4>Key Advantages of the Strategic Transformation Approach</h4><ul><li><strong>Mitigated Risks</strong>: Incremental improvements and iterative learning reduce the potential for costly AI missteps.</li><li><strong>Greater Agility</strong>: Building a culture of adaptability positions organizations to navigate evolving market and AI regulatory landscapes.</li><li><strong>Stronger Stakeholder Alignment</strong>: Clear articulation of AI’s business value fosters buy-in from executives, staff, and patients.</li><li><strong>Sustainable Progress</strong>: Strategic innovation ensures long-term growth, avoiding the pitfalls of short-term, reactive implementations.</li></ul><p>By emphasizing deliberate, incremental advancements over disruptive upheaval, healthcare organizations can responsibly harness AI&#39;s transformative potential. In turn, healthcare organizations can drive meaningful, lasting change while safeguarding the well-being of patients and the integrity of the system.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*yd-tigWUcOwGMuwCusqnHQ.png" /></figure><h3>Timeframe for Implementation</h3><p>Realistic expectations for significant changes in healthcare often span 3–5 years because of the complex interplay of provider, institutional, payor, regulator, and patient-driven imperatives. While there’s excitement about GenAI’s potential in healthcare, there’s also a necessary caution. We’ve previously written about how <a href="https://www.linkedin.com/pulse/navigating-vision-to-action-gap-organizational-success-brian-m-green-kz6fe/?trackingId=zeAd6FdgRFe2XIUlVyW7Dg%3D%3D">AI strategy mapping</a> and use-case prioritization are essential first steps. Developing AI strategy and evaluation across the entire lifecycle allows for robust governance frameworks while also ensuring flexibility for:</p><p><strong>Extensive Testing and Validation</strong>: Ensuring the reliability and safety of AI systems in clinical settings and with context-driven and evolving workflows.</p><p><strong>Stakeholder Adaptation</strong>: Giving healthcare professionals time to train, adapt, and adjust to new AI-assisted workflows.</p><p><strong>Regulatory Alignment</strong>: Allowing regulatory frameworks to evolve alongside technological advancements.</p><h3>Healthcare Strategic Innovations for 2025- “Predictions”</h3><p>The strategic innovations driving AI evolution in 2025 focus on building a solid evidence base for practical, industry-specific applications. This is particularly crucial in regulated industries like healthcare, finance, manufacturing, and beyond. The “trends” likely to dominate in 2025 include:</p><p><strong>Localized AI Models</strong>: Advancements in small language models (SLMs) and multimodal large language models (MMLMs) tailored to specific industries. These models will integrate local data and context to enhance relevance and accuracy, particularly in compliance-heavy sectors like healthcare and finance.</p><p><strong>Agentic AI and Real-Time Adaptation</strong>: Integrating retrieval-augmented generation (RAG) and agentic frameworks to allow AI systems to pull from real-time data sources. This ensures recommendations and insights remain up-to-date and contextually appropriate, a critical requirement for most healthcare applications.</p><p><strong>Human-in-the-Loop (HITL) Optimization</strong>: Evolving HITL processes to enable adaptive feedback loops where AI-generated outputs are refined based on domain expert input. This makes AI more reliable and actionable in high-stakes decision-making, from clinical trials to crisis response.</p><p><strong>Cross-Industry Benchmarks</strong>: Establishing standardized frameworks to test AI systems across multiple domains. For instance, MMLMs can be tested in both healthcare diagnostics and legal document analysis to uncover shared capabilities and identify domain-specific adjustments.</p><p><strong>Ethics-Driven Innovation</strong>: Creating tools and methodologies for AI risk assessment and governance to ensure compliance, safety, and societal acceptance. AI systems will increasingly embed ethical and regulatory guardrails, especially as they scale into public-facing roles.</p><p>These innovations represent new pieces in the healthcare sliding puzzle, each requiring careful integration into existing systems and workflows.</p><h3>The Role of Leadership in Strategic Innovation</h3><p>Leaders in healthcare AI are not just decision-makers but the architects of a new era in patient care. Imagine a hospital leadership navigating the complex sliding puzzle, where each piece represents a critical component of the healthcare system — from existing workflows to new AI technologies. Like in the 15-square puzzle, leaders must strategically shift these pieces within a confined space, carefully maneuvering each element to create a cohesive picture of enhanced patient care and operational efficiency.</p><p>This sliding puzzle metaphor aptly captures the challenges healthcare leaders face. With limited room for maneuvering and numerous interconnected pieces already in place, they must make calculated moves to integrate AI innovations without disrupting the entire system. Each shift requires foresight and precision, as a misplaced piece could throw the entire puzzle into disarray.</p><p>Leaders must articulate a clear vision for how AI will enhance patient outcomes and organizational performance, all while working within the constraints of existing structures. They are tasked with allocating resources judiciously, making high-stakes decisions on technology investments, data infrastructure upgrades, and comprehensive staff training programs. Like a puzzle solver, they must balance the immediate need to create space for new elements with the long-term goal of achieving a harmonious, complete picture.</p><p>By fostering a culture that embraces AI while maintaining an unwavering focus on compassionate, human-centered care, healthcare leaders can transform their organizations into models of strategic innovation.</p><blockquote><em>The question remains: can today’s healthcare leaders solve this intricate puzzle, methodically sliding each piece into place to reveal a transformed, AI-enhanced healthcare landscape?</em></blockquote><h3>Conclusion: The Path Forward</h3><p>Integrating AI into healthcare represents significant opportunities to enhance patient care, improve operational efficiency, and drive medical innovation. However, realizing this potential requires a strategic, measured approach that respects the complexities and responsibilities of healthcare delivery.</p><p>By moving beyond the rhetoric of technological disruption and embracing a model of strategic innovation, healthcare organizations can harness the power of AI while maintaining their commitment to patient safety, ethical practice, and high-quality care. This approach, grounded in careful planning, incremental implementation, and ongoing evaluation, offers a more sustainable and responsible path to healthcare transformation.</p><p>Drawing from Kuhn’s framework and the sliding puzzle metaphor, we recognize that meaningful progress in healthcare AI integration will likely occur through a series of gradual paradigm shifts rather than abrupt disruptions. This approach aligns with the complex nature of healthcare systems and the critical importance of maintaining patient safety and trust throughout the innovation process.</p><p>As we move forward, it’s essential to view AI implementation not as a race to disrupt but as a carefully orchestrated journey of strategic transformation that respects the unique challenges and responsibilities of the healthcare sector.</p><blockquote><strong>AI startups, developers, and healthcare organizations can minimize the disruptive potential of AI integrations through adequate AI strategic planning and a robust governance framework.</strong></blockquote><p>Find out how <a href="https://health-vision.ai/">Health-Vision.AI, LLC</a> can help guide you.</p><h3>References</h3><p><a href="https://www.lri.fr/~mbl/Stanford/CS477/papers/Kuhn-SSR-2ndEd.pdf">https://www.lri.fr/~mbl/Stanford/CS477/papers/Kuhn-SSR-2ndEd.pdf</a></p><p><a href="https://www2.deloitte.com/us/en/pages/life-sciences-and-health-care/articles/generative-ai-in-healthcare.html">https://www2.deloitte.com/us/en/pages/life-sciences-and-health-care/articles/generative-ai-in-healthcare.html</a></p><p><a href="https://pubmed.ncbi.nlm.nih.gov/38491544/">https://pubmed.ncbi.nlm.nih.gov/38491544/</a></p><p><a href="https://www.eclinicalworks.com/blog/generative-ai-integration-into-the-ehr-welcome-to-the-healthcare-revolution/">https://www.eclinicalworks.com/blog/generative-ai-integration-into-the-ehr-welcome-to-the-healthcare-revolution/</a></p><p><a href="https://implementationscience.biomedcentral.com/articles/10.1186/s13012-024-01357-9">https://implementationscience.biomedcentral.com/articles/10.1186/s13012-024-01357-9</a></p><p><a href="https://www.goldmansachs.com/insights/articles/how-artificial-intelligence-is-accelerating-innovation-in-healthcare">https://www.goldmansachs.com/insights/articles/how-artificial-intelligence-is-accelerating-innovation-in-healthcare</a></p><p><a href="https://www.mckinsey.com/industries/healthcare/our-insights/tackling-healthcares-biggest-burdens-with-generative-ai">https://www.mckinsey.com/industries/healthcare/our-insights/tackling-healthcares-biggest-burdens-with-generative-ai</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=db484f9d4071" width="1" height="1" alt=""><hr><p><a href="https://medium.com/responsible-healthcare-ai/why-disruptive-ai-wont-work-in-healthcare-db484f9d4071">Why “Disruptive” AI Won’t Work in Healthcare</a> was originally published in <a href="https://medium.com/responsible-healthcare-ai">Responsible Healthcare AI</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Governance First: Navigating the Spectrum of AI Concerns and Risks in Healthcare]]></title>
            <link>https://medium.com/responsible-healthcare-ai/governance-first-navigating-the-spectrum-of-ai-concerns-and-risks-in-healthcare-12bb6bce2500?source=rss----fea69b1bd593---4</link>
            <guid isPermaLink="false">https://medium.com/p/12bb6bce2500</guid>
            <category><![CDATA[responsible-ai]]></category>
            <category><![CDATA[ai-ethics]]></category>
            <category><![CDATA[generative-ai-tools]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[healthcare-technology]]></category>
            <dc:creator><![CDATA[Brian M. Green]]></dc:creator>
            <pubDate>Mon, 23 Sep 2024 14:27:57 GMT</pubDate>
            <atom:updated>2024-09-23T14:25:14.915Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="A figure of a person jumping amongst blocks with various medical symbols representing risks of AI" src="https://cdn-images-1.medium.com/max/1024/1*ZkOUhKeubJSNoDj3oIEr_g.png" /><figcaption>Image created using MS Designer, DALL-E 3</figcaption></figure><p><a href="https://medium.com/u/fcac4707ae16">Matt Konwiser</a> and <a href="https://medium.com/u/82389cbc52ab">Brian M. Green</a> are co-authors of this article.</p><h4>Why Do We Need a Governance-First Framework?</h4><p>Healthcare and the Health Tech industry are increasingly turning to AI-driven solutions. At the same time, research has begun to expose how AI’s transformative potential is accompanied by notable risks, especially within generative AI applications.</p><blockquote>Adopting a “<strong>Governance First</strong>” AI strategy is critical as the industry catches up with these technological innovations.</blockquote><p><strong>Healthcare</strong> is a multifaceted system encompassing preventative measures, diagnostic procedures, and treatment interventions to maintain and improve individual and public health. We are using this term to include:</p><ul><li>Traditional medical practices and services provided by healthcare professionals in clinical settings.</li><li>Preventative care strategies designed to promote wellness and reduce the risk of disease, including public health initiatives and population health management programs.</li><li>Diagnostic tools and techniques used to identify health issues and inform treatment decisions.</li><li>Therapeutic interventions ranging from medication and surgical procedures to medical devices.</li><li>Health technologies and digital systems that facilitate patient engagement, data management, and remote care delivery.</li><li>Communication platforms and processes that enhance information exchange between healthcare providers and patients.</li></ul><p>This holistic approach to healthcare aims to optimize patient outcomes, improve quality of life, and enhance health services’ overall efficiency and effectiveness by integrating traditional medical practices with innovative technologies and patient-centric communication strategies.</p><p>In this article, we identify some of the intricacies of addressing AI-related errors, ranging from minor inaccuracies to widespread systemic issues, and advocate for a robust, comprehensive governance framework designed for the healthcare industry.</p><h3>The Spectrum of AI Concerns, Understanding the Landscape</h3><p>Before exploring governance strategies, it’s essential to understand the <a href="https://medium.com/@BrianMGreen/no-one-wants-a-counterfeit-golden-ticket-a-lexicon-for-the-spectrum-of-generative-ai-errors-3dc84e379774">spectrum of AI concerns</a> that can arise in healthcare applications. These can range from <strong>localized errors</strong> — such as factual inaccuracies due to incorrect data — to <strong>metanarrative inconsistencies</strong>, where systemic biases or flawed logic can lead to broader issues in patient care.</p><p>AI concerns in healthcare can be categorized along a spectrum that includes biases, risks, and model limitations. These elements are often intertwined, affecting both the technical and sociotechnical aspects of AI deployment:</p><ul><li><strong>Bias</strong>: A stereotype or disproportionate performance skew for some subpopulations, often resulting from historical data that reflect societal inequalities or data representing historical medical training practices that are not evidence-based.</li><li><strong>Risk</strong>: A socially relevant issue that the model might cause, such as exacerbating health disparities or undermining trust in medical institutions.</li><li><strong>Limitation</strong>: This likely failure mode can be addressed by following recommended mitigations, often tied to the model’s technical constraints or specialization of the model’s use case.</li></ul><p>AI errors must be carefully managed, particularly in healthcare, where they can directly impact patient outcomes. The consequences of these errors in healthcare are not just theoretical — they can directly impact patient safety, equity, and the overall quality of care.</p><blockquote>A thorough understanding of these concepts is essential for any sociotechnical professional — a strategist skilled at analyzing the long-term interaction of technology and society. This sociotechnical role is increasingly crucial in AI governance.</blockquote><h3>The Five Layers of AI Risk in Healthcare</h3><p>To effectively address the spectrum of AI errors, it’s helpful to categorize them within the broader framework of AI risks. <a href="https://www.linkedin.com/posts/mattkonwiser_generativeai-governance-ethicalai-activity-7223658533098000385-tYVH/?/">Matt Konwiser’s five-layer</a> Model of AI Risk provides a comprehensive way to understand and mitigate these risks within healthcare.</p><figure><img alt="Figure shows 5 layers of AI risk and provides examples of each category. These 5 include latent, intrinsic, accidental, malicious, and personal." src="https://cdn-images-1.medium.com/max/720/0*QV0vtpjvLofqu0y0" /></figure><h4><strong>Latent Risks</strong></h4><ul><li><strong>Model Collapse</strong>: The <a href="https://www.forbes.com/sites/bernardmarr/2024/08/19/why-ai-models-are-collapsing-and-what-it-means-for-the-future-of-technology/">degradation of AI model performance</a> when one model uses another model’s outputs as training data can lead to compounded inaccuracies, particularly in predictive diagnostics.</li><li><strong>Discrimination in Healthcare Outcomes</strong>: Biases in training data can lead to discriminatory outputs, such as unequal treatment recommendations across different patient demographics, which can compound over time.</li><li><strong>Armageddon Decisions</strong>: In critical care, AI might suggest extreme or overly conservative diagnostic decisions or treatment options, potentially endangering patient lives. In large-scale emergencies, AI might default to a crisis management mode and respond with inhumane “who to save” triage decisions.</li><li><strong>Biased Treatment Pathways</strong>: AI could inadvertently create or reinforce biased treatment protocols that affect patient outcomes based on race, gender, or socioeconomic status, replicating or deepening current health disparities.</li></ul><h4><strong>Intrinsic Risks</strong></h4><ul><li><strong>Poor Fit for Clinical Use</strong>: AI models trained in non-clinical settings or a different disease state may not perform well in actual healthcare environments, particularly with rare or specific subtypes of diseases, leading to misdiagnoses or ineffective treatments.</li><li><strong>Excessive Automation in Patient Care</strong>: Over-automation can result in losing the human touch, which is crucial for patient trust and satisfaction.</li><li><strong>Over-reliance on AI in Critical Care</strong>: In emergency or emergent care situations, excessive dependence on AI can lead to delayed or inappropriate care decisions.</li><li><strong>Misaligned Diagnostic Outputs</strong>: Inaccurate AI interpretations can lead to incorrect diagnoses, affecting patient treatment plans.</li></ul><h4><strong>Accidental Risks</strong></h4><ul><li><strong>Regurgitation of Patient Data</strong>: AI systems trained on sensitive patient information may inadvertently expose that data, violating privacy and trust.</li><li><strong>Improper Training Leading to Mismanagement</strong>: Training AI on incorrect or incomplete data can result in harmful recommendations or actions, directly affecting patient care.</li><li><strong>Plagiarism</strong>: Without proper governance, AI might reproduce content or proprietary data exposed during training, raising intellectual property and data ownership issues in healthcare systems. AI might also incorrectly cite research or link to published research erroneously.</li><li><strong>Implicit Trust in AI Recommendations</strong>: Clinicians might place too much trust in AI outputs, neglecting to verify the accuracy and relevance of the recommendations.</li></ul><h4><strong>Malicious Risks</strong></h4><ul><li><strong>Ablation Attacks on Medical Devices</strong>: Malicious actors could reverse anti-bias measures in AI, leading to biased or harmful medical outcomes. AI could be directed to depart from approved dosing recommendations and ignore safety precautions, drug interactions, or age-related therapeutic guidelines.</li><li><strong>Poisoning of Diagnostic Algorithms</strong>: Deliberately introducing bad data to tamper with AI diagnostics can lead to widespread healthcare mismanagement.</li><li><strong>Disinformation in Medical Outputs</strong>: AI systems could be manipulated to spread false or misleading medical information, endangering public health.</li><li><strong>Existential Threats to Patient Safety</strong>: Sophisticated attacks could expose core AI logic, potentially leading to system-wide failures or manipulations in critical healthcare applications.</li></ul><h4><strong>Personal Risks</strong></h4><ul><li><strong>Overestimation of AI in Healthcare</strong>: Healthcare providers might overestimate AI’s capabilities, reducing critical human oversight and introducing multiple HITL points of failure.</li><li><strong>Isolation in Patient Interaction</strong>: AI could replace meaningful human interaction in care settings, negatively impacting patient experience and outcomes. For example, human contact and emotional support are critical factors in recovery outcomes for patients experiencing long-term recovery.</li><li><strong>AI Dependency among Healthcare Providers</strong>: Easy access and reliance on AI tools might lead to dependency, particularly in time and cost-cutting administrative environments, reducing the development of clinical judgment and critical thinking.</li><li><strong>Humanization of AI in Care Settings</strong>: Patients might begin to perceive AI as more human-like, leading to inappropriate levels of trust or emotional attachment and more skepticism of medical advice from healthcare providers.</li></ul><h3>Case Study: AI-Assisted Diagnostic Tool “CardiacAssistAI”</h3><p>To illustrate applying a “Governance First” approach in healthcare, we provide an example of a hypothetical AI-assisted diagnostic tool, <strong>CardiacAssistAI,</strong> designed to provide healthcare providers with a “second opinion” for a range of cardiovascular diagnoses.</p><h4>Use-case summary</h4><p>Primary care doctors can use generative AI tools utilizing multimodal LLMs (MLLMs) and trained on specific differential diagnoses to enhance diagnostic accuracy. These tools aim to assist healthcare providers in making more informed decisions, especially in complex, rare cases and cases that need quick triage after diagnosis. The <strong>CardiacAssistAI</strong> is designed to provide an AI-assisted “second opinion” and provide initial screening and next steps recommendations before handing off to cardiology or the appropriate department.</p><p>In a real-life appointment, <strong><em>Dr. Smythe</em></strong> reviews their patient, <strong><em>Ms. Johnston</em></strong>’s symptoms using CardiacAssistAI. The AI suggests considering a rare cardiovascular condition Dr. Smythe had not initially considered. Dr. Smythe explains to Ms. Johnston that they used an AI tool to help guide the diagnosis, emphasizing that it is designed to supplement their medical expertise.</p><p>The AI’s interface allows Dr. Smythe to easily show Ms. Johnston how her symptoms align with the suggested diagnosis and what additional tests might be needed by a cardiologist, facilitating a transparent and informative discussion about the next steps. CardiacAssistAI also provides this information more quickly and references more research and evidence-based guidelines than Dr. Smythe could have provided on their own within a single healthcare visit.</p><p>In this use case, Dr. Smythe’s use of an AI-assisted diagnostic tool to guide their consultation with Ms. Johnston illustrates the potential benefits of AI in healthcare. However, to fully realize these benefits, the tool’s deployment should be guided by a “<strong>Governance First</strong>” approach. By overlaying key regulatory and ethical considerations across each layer of AI risk, healthcare providers can ensure that AI tools like this are effective, safe, equitable, and trustworthy.</p><p>Here’s what that governance framework might look like:</p><h3>Integrated AI Governance Framework for Healthcare</h3><p>Given the spectrum of AI concerns and the potential that these layered risks can diminish healthcare goals, governance cannot be an afterthought — it must be integrated into the AI development process from the beginning. After all, the mantra of “<em>f</em><strong><em>irst, do no harm</em></strong>” is central to medical ethics and the mission of quality healthcare.</p><blockquote>By following this consolidated framework, healthcare organizations can better anticipate and mitigate AI-related risks, ensuring that AI systems enhance patient care while maintaining safety and effectiveness.</blockquote><p>Our proposed “<strong>Governance First</strong>” framework includes the following steps and considerations:</p><h4><strong>Establish Multidisciplinary Governance and Identify Project Risks</strong></h4><ul><li>Form a <a href="https://medium.com/@BrianMGreen/responsible-use-of-ai-in-healthcare-strategies-for-balancing-opportunities-and-risk-in-a-b29ab27259b4">diverse AI governance committee</a> that includes healthcare professionals, data scientists, ethicists, sociotechnical experts, patient advocates, and regulatory experts.</li><li>Identify and categorize potential risks across all relevant areas (technical, ethical, legal, socioeconomic, operational) and the 5 layers of AI risk. For example, consider how model collapse and discrimination might impact patient outcomes when developing an AI diagnostic tool.</li></ul><h4><strong>Conduct Readiness, Compliance, and Impact Assessments</strong></h4><ul><li>Evaluate how healthcare personally identifiable information (PII) and data handling regulations (e.g., HIPAA, GDPR) impact the project.</li><li>Perform comprehensive impact assessments (risks, benefits, ethical considerations) on patient care, clinical workflows, and potential biases.</li><li>Assess organizational AI readiness and identify potential priority-use cases and required resource investments.</li><li>Ensure the AI system’s documentation includes warnings, potential harms, and mitigation strategies for technical and sociotechnical limitations.</li></ul><h4><strong>Develop Robust Data Governance and Security Policies</strong></h4><ul><li>Implement strict data protection measures for patient information.</li><li>Develop transparent patient consent processes that clearly explain the AI’s role in diagnosis.</li><li>Establish clear protocols for data usage, storage, and sharing.</li><li>Create enterprise-wide data definitions and quality control processes.</li></ul><h4><strong>Align with Healthcare Objectives</strong></h4><ul><li>Ensure AI projects address genuine clinical needs and improve patient outcomes, optimizing patient touchpoints and engagement.</li><li>Validate the effectiveness of AI tools in various healthcare settings.</li><li>Adjust project scope based on regulatory requirements and risk assessments.</li><li>Ensure that AI tools align with clinical best practices and contribute to reducing health disparities.</li><li>Incorporate patient input on symptom reporting to refine the AI’s questioning algorithms.</li></ul><h4><strong>Implement Bias Mitigation and Ethical Guardrails</strong></h4><ul><li>Design processes to identify and address biases in training data and outputs.</li><li>Ensure diverse representation in the development and testing phases.</li><li>Conduct regular equity assessments in AI model design and validation.</li><li>Ensure cultural sensitivity and inclusivity in the AI’s language and approach.</li></ul><h4><strong>Ensure Transparency and Explainability</strong></h4><ul><li>Create user-friendly interfaces for healthcare providers to understand AI reasoning.</li><li>Design the user interface to facilitate HCPs’ clear communication of AI findings and recommendations to patients.</li><li>Develop clear documentation on AI diagnostic decision-making processes.</li><li>Create educational materials for patients about the AI tool’s use, risks, and benefits.</li><li>Establish clear, measurable outcomes that account for AI performance, risk mitigation, and sociotechnical considerations. For example, ensure diagnostic AI tools have measures for accuracy and equity in healthcare delivery.</li><li>Provide training to enhance AI and data literacy among stakeholders.</li></ul><h4><strong>Establish Continuous AI Monitoring and Feedback</strong></h4><ul><li>Implement continuous model performance monitoring, engage sociotechnical strategists to assess societal impacts, and establish feedback loops with clinical teams.</li><li>Regularly review and update AI models based on new medical knowledge and user feedback.</li><li>Create channels for healthcare providers and patients to provide input on AI system recommendations and outcomes.</li></ul><h3>Additional Regulatory and Ethical Considerations in the “Governance First” Approach</h3><p>To ensure that the CardiacAssistAI diagnostic tool and governance approach are effective and aligned with regulatory and ethical standards, it is crucial to overlay specific considerations across each layer of AI risk.</p><p>Below is a breakdown of the key regulatory and ethical considerations based on the use case of Dr. Smythe and Ms. Johnston’s interaction:</p><h3>Latent Risks</h3><h4><strong>Regulatory Compliance</strong></h4><ul><li><strong>Data Protection</strong>: The CardiacAssistAI tool must comply with HIPAA regulations, ensuring that patient data used during training and consultations is securely stored and processed. Data anonymization should be applied where possible to protect patient privacy.</li><li><strong>Non-Discrimination</strong>: The tool must be validated to ensure it does not introduce bias into diagnostic suggestions. This includes compliance with the U.S. Civil Rights Act to prevent discrimination based on race, gender, or socioeconomic status in diagnostic outcomes.</li></ul><h4><strong>Ethical Considerations</strong></h4><ul><li><strong>Fairness and Equity</strong>: The CardiacAssistAI tool should be rigorously tested across diverse patient demographics to ensure its diagnostic suggestions are equitable and do not disproportionately affect certain groups.</li><li><strong>Transparency</strong>: Dr. Smythe’s ability to explain the AI’s suggestions to Ms. Johnston reflects the ethical need for transparency. The tool’s decision-making process should be clear and understandable to both the provider and the patient, facilitating informed discussions.</li></ul><h3>Intrinsic Risks</h3><h4><strong>Regulatory Compliance</strong></h4><ul><li><strong>Medical Device Regulations</strong>: If the CardiacAssistAI tool is classified as a medical device, it must comply with FDA regulations for Software as a Medical Device (SaMD). This includes pre-market approval, clinical validation, and continuous post-market surveillance.</li><li><strong>Clinical Validation</strong>: The AI model must be clinically validated to ensure its accuracy and reliability in providing diagnostic suggestions. This could involve peer-reviewed studies or real-world evidence demonstrating its effectiveness in diverse clinical settings.</li></ul><h4><strong>Ethical Considerations</strong></h4><ul><li><strong>Patient Autonomy</strong>: In the example, Dr. Smythe uses the CardiacAssistAI tool to supplement his expertise, not replace it. This respect for patient autonomy ensures that AI supports clinical judgment rather than dictating it.</li><li><strong>Informed Consent</strong>: Before or at the beginning of her appointment, Ms. Johnston should be informed about the use of CardiacAssistAI in her diagnosis and give explicit consent, particularly if the AI’s suggestions significantly influence her treatment plan.</li></ul><h3>Accidental Risks</h3><h4><strong>Regulatory Compliance</strong></h4><ul><li><strong>Data Breach Notification</strong>: If the CardiacAssistAI tool or its training models inadvertently expose patient data, the healthcare system must comply with data breach notification laws and promptly inform affected patients and regulatory bodies.</li><li><strong>Liability and Accountability</strong>: Clear guidelines must be established to determine accountability if the CardiacAssistAI tool provides a faulty or harmful diagnosis. This includes ensuring that healthcare providers, AI developers, and the healthcare system understand their legal responsibilities along each workflow stage.</li></ul><h4><strong>Ethical Considerations</strong></h4><ul><li><strong>Privacy and Confidentiality</strong>: The CardiacAssistAI tool must have robust privacy protections to prevent accidental data exposure. This aligns with the ethical obligation to maintain patient confidentiality.</li><li><strong>Accountability</strong>: The healthcare system should establish clear lines of accountability for the CardiadAssistAI tool’s suggestions, ensuring that Dr. Smythe retains ultimate responsibility for the diagnostic decisions made during the consultation prior to any handoff to a specialist.</li></ul><h3>Malicious Risks</h3><h4><strong>Regulatory Compliance</strong></h4><ul><li><strong>Cybersecurity Standards</strong>: The CardiacAssistAI tool must adhere to stringent cybersecurity standards, such as those outlined by NIST, to protect against hacking or malicious data manipulation that could alter diagnostic outputs.</li><li><strong>Anti-Fraud Measures</strong>: The healthcare system must implement anti-fraud measures to detect and prevent any malicious use of the AI tool, such as tampering with diagnostic algorithms to produce incorrect or harmful suggestions.</li><li><strong>State/local and Emerging Standards</strong>: <a href="https://www.linkedin.com/pulse/lets-talk-politics-sb-1047-matt-konwiser-9zrff/">California’s SB 1047 requires audit</a>, reporting, documentation, and retention of metadata logs and other records of risk identification and mitigation efforts, among other areas of compliance. Other states have similar regulatory schemes or are in the process of developing them.</li></ul><h4><strong>Ethical Considerations</strong></h4><ul><li><strong>Harm Prevention</strong>: The CardiacAssistAI tool should be designed with safeguards against malicious use, prioritizing patient safety. This includes regular ethical red teaming and testing to identify vulnerabilities.</li><li><strong>Security Transparency</strong>: Patients like Ms. Johnston should be informed about the security measures that protect the CardiacAssistAI tool from tampering, reinforcing trust in the system’s integrity.</li></ul><h3>Personal Risks</h3><h4><strong>Regulatory Compliance</strong></h4><ul><li><strong>Patient Rights</strong>: The CardiacAssistAI tool must be designed to respect patient rights, ensuring its use aligns with existing patient rights regulations. This includes providing options for patients to opt out of AI-driven diagnoses if they prefer more traditional methods.</li><li><strong>Mental Health Considerations</strong>: The CardiacAssistAI tool’s interface and interaction style should comply with mental health regulations, avoiding designs that could increase patient anxiety or discomfort during consultations.</li></ul><h4><strong>Ethical Considerations</strong></h4><ul><li><strong>Human-Centered Design</strong>: The AI tool should enhance the patient-provider interaction, not replace it. Dr. Smythe’s use of the CardiacAssistAI tool to facilitate a transparent and informative discussion with Ms. Johnston exemplifies this ethical obligation.</li><li><strong>Ethical Use of AI</strong>: The healthcare system should ensure that AI is used ethically, enhancing care without creating dependencies or reducing the quality of human interaction in patient care.</li></ul><h3>Conclusion</h3><p>As generative AI continues to expand and evolve within healthcare, the risks and potential errors associated with its use must be carefully managed. By adopting a “<strong>Governance First</strong>” approach, healthcare organizations can navigate the complex spectrum of AI errors and ensure that AI systems are safe, reliable, and effective.</p><blockquote>Integrating governance into every stage of AI development is not just a best practice — it is mandatory for the future of healthcare.</blockquote><h4><strong>Call to Action</strong></h4><p>Healthcare leaders and AI developers must prioritize AI governance strategies and understand that this goes beyond customary <strong>data governance</strong> practices. With a “<strong>Governance First</strong>” framework for AI in healthcare, we can better anticipate and ameliorate the spectrum of AI concerns to build systems that truly enhance patient care while minimizing risks.</p><blockquote>Healthcare leaders should champion a patient-centric, governance-first approach and emphasize how this enhances AI’s effectiveness and acceptability in healthcare.</blockquote><p>As AI becomes more integrated into healthcare, providers, developers, and regulatory bodies must collaborate to embed robust governance strategies into every stage of AI development and deployment. This approach ensures that AI tools enhance, rather than compromise, patient care.</p><p>One proposal to help healthcare organizations and developers prioritize AI use cases would be for a rating or scoring system that provides a common rubric across various AI governance factors. Most enterprises find it challenging to develop and simplify such a rating system while making it interoperable and robust. One helpful resource in this area, the Coalition for Healthcare AI (CHAI), has working groups that have developed such recommendations. Their <a href="https://chai.org/assurance-standards-guide/">AI Governance Assurance checkpoints</a>, which integrate clinical risk level assessment and population impact evaluation in five stages of AI development, testing, and deployment, are a trailblazing effort.</p><p>Healthcare and business leaders, AI developers, data scientists, and sociotechnical experts must collaborate to prioritize governance, promote explainable AI, and mitigate the spectrum of AI concerns to build systems that enhance patient care while minimizing risks. By implementing an integrated AI governance framework, involving patients in the design process, and focusing on an ethical foundation, we can harness AI’s full potential in healthcare while safeguarding patient trust and safety and ensuring AI tools are equitable and trustworthy.</p><h4>About the Authors</h4><p><a href="https://medium.com/u/fcac4707ae16">Matt Konwiser</a> has been a professional in information technology, telecommunications, cybersecurity, and AI for over two decades. He is known for his thought leadership in IT innovation and business-technology alignment. He is IBM’s Northeast US CTO and Cross-Brand ATL Leader. M<a href="https://www.linkedin.com/in/mattkonwiser/">att on LinkedIn</a></p><p><a href="https://medium.com/u/82389cbc52ab">Brian M. Green</a> has worked in healthcare research, public health policy and programs, community engagement, health communications, and health tech commercial strategy for over two decades. He is the Chief AI Ethics Officer and Founder of <a href="https://health-vision.ai/">Health-Vision.AI</a>, LLC. Br<a href="https://www.linkedin.com/in/bgreen2/">ian on LinkedIn</a></p><h3>References</h3><ul><li><a href="https://www.forbes.com/sites/bernardmarr/2024/08/19/why-ai-models-are-collapsing-and-what-it-means-for-the-future-of-technology/">Why AI Models Are Collapsing And What It Means For The Future Of Technology</a></li><li><a href="https://medium.com/@BrianMGreen/no-one-wants-a-counterfeit-golden-ticket-a-lexicon-for-the-spectrum-of-generative-ai-errors-3dc84e379774">No One Wants a Counterfeit “Golden Ticket:” A Lexicon for the Spectrum of Generative AI Errors</a></li><li><a href="https://www.linkedin.com/pulse/lets-talk-politics-sb-1047-matt-konwiser-9zrff/">Let&#39;s Talk Politics - SB 1047</a></li><li><a href="https://www.linkedin.com/posts/mattkonwiser_generativeai-governance-ethicalai-activity-7223658533098000385-tYVH?/">How generative AI governance is explained | Matt Konwiser posted on the topic | LinkedIn</a></li><li><a href="https://medium.com/@BrianMGreen/responsible-use-of-ai-in-healthcare-strategies-for-balancing-opportunities-and-risk-in-a-b29ab27259b4">Responsible Use of AI in Healthcare — Strategies for Balancing Opportunities and Risk in a…</a></li><li><a href="https://www.computerworld.com/article/3487262/researchers-tackle-ai-fact-checking-failures-with-new-llm-training-technique.html">Researchers tackle AI fact-checking failures with new LLM training technique</a></li><li><a href="https://www.himss.org/resources/predictive-medicine-advancing-healthcare-through-better-data-governance">Predictive Medicine: Advancing Healthcare Through Better Data Governance</a></li><li><a href="https://chai.org/assurance-standards-guide/">Assurance Standards Guide - CHAI - Coalition for Health AI</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=12bb6bce2500" width="1" height="1" alt=""><hr><p><a href="https://medium.com/responsible-healthcare-ai/governance-first-navigating-the-spectrum-of-ai-concerns-and-risks-in-healthcare-12bb6bce2500">Governance First: Navigating the Spectrum of AI Concerns and Risks in Healthcare</a> was originally published in <a href="https://medium.com/responsible-healthcare-ai">Responsible Healthcare AI</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Responsible Health AI Welcomes You!]]></title>
            <link>https://medium.com/responsible-healthcare-ai/responsible-health-ai-welcomes-you-c0bde7b8bdc1?source=rss----fea69b1bd593---4</link>
            <guid isPermaLink="false">https://medium.com/p/c0bde7b8bdc1</guid>
            <dc:creator><![CDATA[Brian M. Green]]></dc:creator>
            <pubDate>Sat, 21 Sep 2024 20:56:47 GMT</pubDate>
            <atom:updated>2024-09-22T17:40:23.852Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/879/1*6fwYLS4MzKIruiG0HMATFw.png" /><figcaption>Image created by author in MS Designer using DALL-E 3</figcaption></figure><p>Dear Fellow Healthcare Innovators,</p><p>I’m thrilled to announce the launch of “<strong>Responsible Healthcare AI</strong>,” a newsletter dedicated to exploring the ethical implementation of AI in healthcare. As the AI landscape in healthcare continues to evolve, it is crucial to stay informed about the challenges and opportunities of generative AI, particularly for those concerned about patient centricity and ethics. In our inaugural issue, we dive into two critical topics:</p><ol><li><strong>Prioritizing Patient-Centricity in GenAI Healthcare Solutions</strong><br>We explore <a href="https://brian-m-green.com/blog/f/prioritizing-patient-centricity-in-genai-healthcare-solutions">how generative AI</a> can enhance patient care while maintaining a human-centered approach. From improving diagnostic accuracy to personalizing treatment plans, we discuss strategies to ensure AI serves as a tool to augment, not replace, the human touch in healthcare.</li><li><strong>Debunking the Top 5 Myths About Ethical AI Use in Healthcare</strong><br>We address <a href="https://brian-m-green.com/blog/f/top-5-myths-about-ethical-ai-use-in-healthcare">common misconceptions</a> surrounding AI in healthcare, such as the fear of AI replacing healthcare professionals or the belief that AI decisions are always unbiased. By confronting these myths head-on, we aim to foster a more nuanced understanding of AI’s role in healthcare.</li></ol><p>“<strong>Responsible Healthcare AI</strong>” is committed to providing you with insights, best practices, and thought-provoking discussions on how we can harness the power of AI while upholding the highest ethical standards in patient care.</p><p>Join us on this journey towards a future where AI and human expertise work in harmony to improve health outcomes for all. Stay tuned for our first full issue, coming soon to your inbox!</p><p>Brian M. Green<br>Editor, “Responsible Healthcare AI”</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c0bde7b8bdc1" width="1" height="1" alt=""><hr><p><a href="https://medium.com/responsible-healthcare-ai/responsible-health-ai-welcomes-you-c0bde7b8bdc1">Responsible Health AI Welcomes You!</a> was originally published in <a href="https://medium.com/responsible-healthcare-ai">Responsible Healthcare AI</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Demystifying the Black Box: Understanding Generative AI Algorithm Performance]]></title>
            <link>https://medium.com/responsible-healthcare-ai/demystifying-the-black-box-understanding-generative-ai-algorithm-performance-ce237e9f1119?source=rss----fea69b1bd593---4</link>
            <guid isPermaLink="false">https://medium.com/p/ce237e9f1119</guid>
            <category><![CDATA[ai-algorithms]]></category>
            <category><![CDATA[health-technology]]></category>
            <category><![CDATA[algorithms]]></category>
            <category><![CDATA[ai-healthcare-revolution]]></category>
            <category><![CDATA[generative-ai-tools]]></category>
            <dc:creator><![CDATA[Brian M. Green]]></dc:creator>
            <pubDate>Sat, 21 Sep 2024 19:04:36 GMT</pubDate>
            <atom:updated>2024-07-10T04:00:07.090Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="A Black stylized box, reminiscent of Pandora’s box, with blocks of data coming out of the opened lid of the box." src="https://cdn-images-1.medium.com/max/1024/1*L2xX_bH0j_ckv-j-VSWbPg.png" /><figcaption>Image generated by Author using MS CoPilot DALL-E 3</figcaption></figure><p>Generative AI (GenAI) and LLM algorithms are producing profound changes in various fields, from healthcare to retail. However, their complexity can make them seem like “black boxes,” obscuring how they function. Understanding key aspects of GenAI algorithms and their performance can empower users and stakeholders to make informed decisions about their adoption and use.</p><h3><strong>Why Evaluating AI Algorithm Performance Matters</strong></h3><p>The growing prevalence of AI applications necessitates a deeper understanding of how these algorithms perform. Not all AI algorithms are created equal. Their effectiveness can significantly vary based on how accurately they complete tasks and how widely they can be applied to different problems. Evaluating these aspects is crucial for developing reliable and effective AI solutions that meet real-world needs.</p><p>I have picked two key performance measures for this article that illustrate a simplified real-world example of AI algorithm performance measurement for two use cases that provide a comparative framework: a <strong>Retail Chatbot Agent</strong> and an <strong>AI-driven Therapeutic Target Discovery Engine</strong>. The performance measures we examine are:</p><ul><li><strong>Accuracy:</strong> How often the algorithm makes correct predictions or decisions.</li><li><strong>Coverage:</strong> The range of problems or scenarios the algorithm can handle.</li></ul><p>Some AI algorithms might prioritize <strong>coverage</strong> when applied to a “lower stakes” use case. However, in the critical domain of healthcare AI applications, ensuring high <strong>accuracy</strong> is paramount. Simply measuring raw accuracy might not tell the whole story, either.</p><p>For these examples, we consider the assessment of algorithm accuracy along a scale that incorporates utility and penalty values. Utility and penalty values can provide a more nuanced understanding of accuracy in the context of a specific AI algorithm evaluation example.</p><ul><li><strong>Utility:</strong> Represents the benefit or reward associated with a correct prediction. For instance, an accurate disease diagnosis in healthcare allows for timely intervention, potentially improving patient outcomes (high utility).</li><li><strong>Penalty:</strong> Represents the cost or negative impact of an incorrect prediction. For example, a misdiagnosis could lead to delayed treatment or unnecessary procedures, potentially harming the patient (high penalty).</li></ul><p>Combining Coverage, Accuracy, Utility, and Penalty metrics is crucial for assessing an algorithm’s suitability for different applications. Below, I provide examples of how these can be applied to AI use cases.</p><h3>Other Key Aspects of GenAI Algorithms</h3><p>Understanding GenAI algorithms goes beyond just accuracy and coverage. It is also important to dive deeper to know HOW AI models arrive at their decisions. In the context of healthcare, understanding the contents of the AI black box is absolutely essential. Lives and <a href="https://brian-m-green.com/blog/f/prioritizing-patient-centricity-in-genai-healthcare-solutions">patient well-being depend on</a> these models’ accuracy, explainability, and fairness. Additional concepts that help us with this understanding include:</p><h4><strong>Interpretability</strong> and Explainability</h4><p>We can analyze GenAI algorithms to understand their decision-making processes, allowing us to “look under the hood” as models are developed and tested pre-deployment. It is also important for post-deployment AI auditing and a system of continuous auditing for auditors to assess fairness, bias detection, and explainability of the AI systems.</p><p><strong>Interpretability</strong> empowers auditors to identify potential issues and ensure alignment with ethical and legal frameworks. <strong>Explainability</strong> is HOW these AI models reach decisions and achieve their desired outcomes in a way that can be simplified and presented to non-technical internal and external stakeholders.</p><h4><strong>Human-in-the-Loop (HITL)</strong></h4><p>Integrating human oversight into AI systems helps to ensure that algorithms are grounded in real-world needs and constraints. HITL helps mitigate risks, particularly in high-stakes environments like healthcare, by incorporating human judgment to manage exceptions and continuously improve AI performance. HITL should be <a href="https://medium.com/@BrianMGreen/responsible-use-of-ai-in-healthcare-strategies-for-balancing-opportunities-and-risk-in-a-b29ab27259b4">outlined in your AI governance framework</a>.</p><p>HITL also plays a vital role in AI audits. By involving human reviewers who assess the outputs of AI systems, auditors can ensure adherence to ethical guidelines. The results of the AI audit can be leveraged within the HITL framework to iterate improvements on the AI system. This iterative process can involve testing risk mitigation strategies and evaluating general performance metrics tied to the <a href="https://medium.com/@BrianMGreen/no-one-wants-a-counterfeit-golden-ticket-a-lexicon-for-the-spectrum-of-generative-ai-errors-3dc84e379774">AI system&#39;s real-world implementation</a>.</p><h4><strong>AI Algorithm Balancing Act — Accuracy, Coverage, and Optimization</strong></h4><p>Evaluating GenAI algorithms through a utility and penalty framework helps quantify their performance in real-world applications. This approach helps internal and external stakeholders understand the trade-offs between accuracy and coverage and how to balance them. Also, interpretability, explainability, and HITL mechanism play important roles as we optimize AI algorithms, as demonstrated in the following scenarios.</p><h3><strong>Use-case, Scenario 1: Retail Chatbot Agent</strong></h3><p>Consider a retail chatbot designed to assist customers with product recommendations. Here, the stakes for accuracy vs coverage for a wide range of customers are relatively low compared to other applications like healthcare.</p><p><strong>Utility and Penalty Values:</strong></p><ul><li><strong>Utility of Correct Response (<em>U</em>)</strong>: Increased customer satisfaction and sales, e.g., <strong><em>U</em></strong>=100 units.</li><li><strong>Penalty for Incorrect Response (<em>P</em>)</strong>: Potential customer dissatisfaction and lost sales, e.g.,<em> </em><strong><em>P</em></strong>=−50 units.</li></ul><p><strong>Algorithms:</strong></p><p><strong>Algorithm 1:</strong> 50% coverage, 90% accuracy</p><p><strong>Algorithm 2:</strong> 70% coverage, 80% accuracy</p><h4>Expected Utility Calculation</h4><p><strong>Algorithm 1: </strong>Expected Utility1 = 0.50 × (0.90 × 100 + 0.10 × −50) = 0.50 x (90−5) = 0.50 × 85 = <strong>42.5</strong></p><p><strong>Algorithm 2: </strong>Expected Utility2 = 0.70 × (0.80 × 100 + 0.20 × −50)\ = ​0.70 × (80–10) = 0.70<strong> </strong>× 70 = <strong>49</strong></p><p><strong>Final answer:</strong></p><ul><li><strong>Algorithm 1:</strong> Expected Utility = 42.5</li><li><strong>Algorithm 2:</strong> Expected Utility = 49</li></ul><h4><strong>Analysis and Balancing Act:</strong></h4><p><strong><em>Algorithm 2</em></strong>, with higher coverage, provides better overall utility despite lower accuracy. A higher coverage algorithm might be preferable, even with slightly lower accuracy, to ensure the chatbot can handle a more comprehensive range of customer queries and maintain a broad customer funnel. HITL can help fine-tune responses and manage exceptions, enhancing customer satisfaction and sales.</p><h3><strong>Use-case, Scenario 2: </strong>Therapeutic Target Discovery Engine</h3><p>Consider an AI algorithm designed to analyze clinical research multimodal data for potential drug targets and candidate therapeutic agents. Accuracy is essential in this context due to the high stakes involved in healthcare and the top priority that should be given to patient safety.</p><p><strong>Utility and Penalty Values:</strong></p><ul><li><strong>Utility of Correct Analysis (<em>U</em>)</strong>: Better research outcomes and the discovery of new therapeutic targets, e.g., <strong><em>U</em></strong>=2000 units.</li><li><strong>Penalty for Incorrect Analysis (<em>P</em>)</strong>: Wasted resources, potential patient harm, e.g., <strong><em>P</em></strong>=−10000 units.</li></ul><p><strong>Algorithms:</strong></p><p><strong>Algorithm 1: </strong>40% coverage, 98% accuracy</p><p><strong>Algorithm 2: </strong>60% coverage, 85% accuracy</p><h4>Expected Utility Calculation</h4><p><strong>Algorithm 1: </strong>Expected Utility1 = 0.40 × (0.98 × 2000 +0.02 ×−10000) =0.40 × (1960 − 200) = 0.40 × 1760 = <strong>704</strong></p><p><strong>Algorithm 2: </strong>Expected Utility2 = 0.60 × (0.85 × 2000 + 0.15 × −10000) = 0.60 × (1700 − 1500) = 0.60 × 200 = <strong>120</strong></p><p><strong>Final answer:</strong></p><ul><li><strong>Algorithm 1: </strong>Expected Utility = <strong>704</strong></li><li><strong>Algorithm 2: </strong>Expected Utility = <strong>120</strong></li></ul><h4><strong>Analysis and Balancing Act:</strong></h4><p><strong><em>Algorithm 1</em></strong>, with higher accuracy, is preferable despite lower coverage due to the high cost of errors. In this scenario, prioritizing higher accuracy is essential, even if it means sacrificing some coverage, to minimize the risk of errors that could harm patients. HITL can further ensure that the AI system’s recommendations are vetted by experts, improving reliability and trust in the system.</p><h3><strong>Applying the Framework: Evaluating Algorithm Performance</strong></h3><p>As shown in these contrasting use cases and comparing scenarios that each have two different AI algorithms, several crucial considerations come into play:</p><ul><li><strong>Coverage vs. Accuracy Trade-offs:</strong> Evaluating both aspects is essential to understanding the algorithm’s ability to handle a range of situations while maintaining its correct performance.</li><li><strong>Penalty Values and Ethical Implications:</strong> Assigning appropriate penalty values emphasizes different ethical considerations. For example, high penalties for potential patient harm in healthcare AI highlight the importance of accuracy in minimizing harm, risk, or bias.</li></ul><p>To balance successful AI performance, assessing the trade-offs between coverage, accuracy, and penalty values&#39; implications is important to understand what is happening inside the ‘black box.” We also want to optimize the algorithm performance as different considerations for the implementation, model evolution, and source data changes.</p><p>Understanding at least some of what is in the AI black box empowers stakeholders to make informed decisions about:</p><ul><li><strong>Algorithm Selection:</strong> Choosing the correct algorithm for the specific application and its risk tolerance.</li><li><strong>Optimization:</strong> Fine-tuning the algorithm to improve performance within acceptable resource constraints.</li><li><strong>Deployment:</strong> Deploying the algorithm responsibly, considering ethical implications and potential risks.</li></ul><p>While ensuring accuracy is paramount in healthcare AI, it’s not the only consideration. Complexity theory reminds us that AI models, especially those designed for processing complex medical data, often face a significant challenge: <strong>balancing accuracy and reliability with computational demands. </strong>Ensuring the algorithm’s accuracy and reliability while managing the <a href="https://www.linkedin.com/pulse/taming-cost-beast-cost-containment-strategies-genai-healthcare-green-q8qme/">high computational demands for processing complex</a> multimodal medical data matters for all of the above reasons.</p><h3>Conclusion: A Human-Centered Approach to AI Performance</h3><p>Understanding the balance between accuracy and coverage and incorporating HITL judgment into AI evaluation is crucial for ensuring AI solutions are:</p><ul><li><strong>Practical and Reliable:</strong> They can handle real-world tasks effectively and meet specific needs.</li><li><strong>Aligned with Needs:</strong> They are tailored to address the challenges and constraints of the application domain.</li><li><strong>Ethical:</strong> They prioritize responsible development and use, minimizing risks, and adhering to ethical principles.</li></ul><p>As we have discussed, HILT is essential for general applications, like retail chatbots, and specialized uses, like clinical research.</p><p>By setting clear performance expectations, considering the impact of errors through penalty values, and incorporating complexity theory to manage computational demands, we can develop AI systems that truly meet the demands of their intended use cases. Attention to these AI evaluation measures and incorporating a culture of continuous audits fosters transparency, trust, and, ultimately, the responsible and impactful use of AI technology across various fields.</p><p>For <a href="https://www.linkedin.com/pulse/navigating-vision-to-action-gap-organizational-success-brian-m-green-kz6fe/?trackingId=wBJXnEnkRa2IfCm5TTsF4g%3D%3D">consultants working in the AI space</a>, providing clear and engaging marketing or informational materials on how AI algorithms work, the data they use, and the measures in place to protect privacy can empower your clients to make more informed decisions about AI adoption.</p><h3>References and Other Resources</h3><h4>A Framework for Evaluating Machine Learning Models</h4><p>Scikit-learn: Machine Learning in Python, Pedregosa et al., JMLR 12, pp. 2825–2830, 2011.</p><p><a href="https://scikit-learn.org/stable/modules/model_evaluation.html">3.4. Metrics and scoring: quantifying the quality of predictions</a></p><h4>Defining and Measuring Algorithmic Fairness</h4><p><a href="https://learn.microsoft.com/en-us/azure/machine-learning/concept-fairness-ml?view=azureml-api-2">Machine learning fairness - Azure Machine Learning</a></p><h4>Patient-Centric AI Solutions</h4><p><a href="https://brian-m-green.com/blog/f/prioritizing-patient-centricity-in-genai-healthcare-solutions">Prioritizing Patient-Centricity in GenAI Healthcare Solutions</a></p><h4>AI Strategy and Readiness Assessment</h4><p><a href="https://www.linkedin.com/pulse/navigating-vision-to-action-gap-organizational-success-brian-m-green-kz6fe/?trackingId=wBJXnEnkRa2IfCm5TTsF4g%3D%3D">Navigating the Vision-to-Action Gap for Organizational Success in Health AI</a></p><h4>AI Governance and Responsible AI</h4><p><a href="https://medium.com/@BrianMGreen/responsible-use-of-ai-in-healthcare-strategies-for-balancing-opportunities-and-risk-in-a-b29ab27259b4">Responsible Use of AI in Healthcare — Strategies for Balancing Opportunities and Risk in a…</a></p><h4>AI Resource Planning and Compute Demands</h4><p><a href="https://www.linkedin.com/pulse/taming-cost-beast-cost-containment-strategies-genai-healthcare-green-q8qme/?trackingId=1NE393JyTaObVUEysZDf4w%3D%3D">Taming the Cost Beast: Cost-Containment Strategies for GenAI in Healthcare</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ce237e9f1119" width="1" height="1" alt=""><hr><p><a href="https://medium.com/responsible-healthcare-ai/demystifying-the-black-box-understanding-generative-ai-algorithm-performance-ce237e9f1119">Demystifying the Black Box: Understanding Generative AI Algorithm Performance</a> was originally published in <a href="https://medium.com/responsible-healthcare-ai">Responsible Healthcare AI</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[No One Wants a Counterfeit “Golden Ticket:” A Lexicon for the Spectrum of Generative AI Errors]]></title>
            <link>https://medium.com/responsible-healthcare-ai/no-one-wants-a-counterfeit-golden-ticket-a-lexicon-for-the-spectrum-of-generative-ai-errors-3dc84e379774?source=rss----fea69b1bd593---4</link>
            <guid isPermaLink="false">https://medium.com/p/3dc84e379774</guid>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[explainable-ai]]></category>
            <category><![CDATA[generative-ai-tools]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[llm]]></category>
            <dc:creator><![CDATA[Brian M. Green]]></dc:creator>
            <pubDate>Sat, 21 Sep 2024 19:04:22 GMT</pubDate>
            <atom:updated>2024-06-24T19:08:33.083Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="Man dressed like magician with a top hat hold a golden ticket" src="https://cdn-images-1.medium.com/max/1024/1*DxgNi5IxvGoGU3aqVArXKw.png" /><figcaption>Image created by author using DALL-E 3</figcaption></figure><h3>Why We Need to Talk About Generative AI Errors</h3><p>Generative AI (GenAI), particularly large language models (LLMs), is a transformative force in information access and task execution. These powerful tools promise to reshape our lives, enabling us to create compelling marketing content and simplify complex research.</p><p>However, like any tool, GenAI has limitations. One critical aspect that demands our attention is the issue of model errors. I propose a new way of conceptualizing terminology that recognizes a spectrum of GenAI Errors, a proposal that suggests we replace the ambiguous and generalized term “hallucinations.”</p><h3>A New Lexicon is Needed</h3><p>“Hallucinations” is a term borrowed from human psychology and <a href="https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10552880/">might not be the most accurate way</a> to describe LLM errors. Two significant limitations of using the term “hallucinations” are that it is an <strong>anthropomorphization</strong> and an <strong>oversimplification</strong> of the phenomenon. Hallucination implies a conscious experience of seeing things that aren’t there, which is not valid with LLMs. The term also fails to capture the underlying causes of LLM errors, like data bias or overfitting.</p><p>These errors can manifest in various ways, and accurately describing them is crucial for several reasons:</p><ul><li><strong>Intended Meaning and Building Trust</strong>: For GenAI to reach its full potential, users need to have faith in the information it provides. Fuzzy terms like “hallucinations” don’t provide a clear picture of the types of errors that might occur. I literally cringed the first time I heard a presenter talk about AI “hallucinations.” However, by accurately describing these errors, we can build a foundation of trust and foster a deeper understanding of the technology, a crucial step in its acceptance and integration.</li><li><strong>Understanding the Causes:</strong> By precisely describing different error categories (I propose using the terms “localized” factual errors and “metanarrative” inconsistencies that I will explain below), we can better understand the underlying causes within the training data or the model architecture itself.</li><li><strong>Enhancing Human Learning</strong>: GenAI is a tool designed to assist humans, not replace them. If users encounter incorrect information, they waste time “unlearning” these errors, hindering GenAI&#39;s efficiency as a learning and productivity tool.</li></ul><p>This article delves into the different types of errors GenAI models can generate, differentiating between specific, isolated factual mistakes (“localized”) and broader, interwoven narratives based on faulty connections or correlations among vast datasets (“metanarrative”) in LLMs.</p><h3><strong>Localized Errors</strong></h3><p>I use “localized errors” to describe specific and narrow factual errors. Imagine an LLM trained on a dataset with a typo claiming the population of France is 1 million (instead of 65 million). When prompted about France, the LLM might confidently spout this incorrect statistic.</p><p>These <strong>localized errors </strong>could arise from the following:</p><ul><li><strong>Limited or Biased Training Data</strong>: If the training data lacks variety or has a specific bias, the LLM might latch onto that and present it as a fact.</li><li><strong>Statistical Fluctuations</strong>: Random noise and variations in the data could lead the LLM to identify and present spurious correlations as accurate.</li><li><strong>Factual Errors:</strong> This general term encompasses all types of incorrect information generated by an LLM.</li><li><strong>Data Anomalies:</strong> This term emphasizes the role of errors or inconsistencies within the training data that are not thoroughly tested.</li></ul><h3><strong>Metanarrative Inconsistencies</strong></h3><p>I use “metanarrative inconsistencies” to describe a more complex and interwoven set of factual errors. Imagine an LLM or MLLM trained in a mix of historical documents, conspiracy theories, and science fiction. When asked about the moon landing, it might weave a story combining factual elements with invented details about a secret alien base.</p><p>These <strong>metanarrative inconsistencies</strong> arise from the following:</p><ul><li><strong>Overfitting</strong>: The LLM might be too good at finding patterns in the data, even if those patterns are invalid. This can lead to elaborate but ultimately false narratives.</li><li><strong>Inconsistent Training Data</strong>: If the training data contains contradictory information, the LLM might struggle to reconcile these differences. This might be more likely with MLLMs as a model struggles with reconciling inconsistent metaphors used in written vs images, etc. This can lead to internally inconsistent metanarratives.</li></ul><p><strong>Here are some additional points to consider:</strong></p><ul><li>The distinction might be blurry: Sometimes, a localized error could be a building block for a more significant metanarrative error.</li><li>Mitigating errors: Researchers are looking at ways to improve training data quality, develop techniques to detect and flag errors, and improve the reasoning abilities of LLMs.</li></ul><blockquote><strong>Training Data Matters!</strong> The quality and consistency of the training data significantly impact the likelihood of metanarrative errors. Well-curated, diverse training data can help mitigate these issues.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*6cONf5GWapHs41HclETY4Q.png" /><figcaption>Image created by author using DALL-E 3</figcaption></figure><h3>LLMs vs MLLMs and Error Types</h3><p>The likelihood of models producing localized errors versus metanarrative inconsistencies might differ depending on whether one uses text-based LLMs or multimodal models. Here are some of the differences based on these model types.</p><p><strong>LLMs:</strong></p><ul><li><strong>Prone to Localized Hallucinations:</strong> Their reliance on text data makes them more susceptible to inheriting factual errors or biases from the training data.</li><li><strong>Limited Potential for Metanarrative Inconsistency:</strong> They might struggle to create complex (and inaccurate) narratives because they cannot process and integrate information from different modalities (images, sounds, videos).</li></ul><p><strong>MLLMs:</strong></p><ul><li><strong>Risk of Both Types:</strong> MLLMs can inherit localized errors from text data but also generate metanarrative inconsistencies due to:</li><li><strong>Mixing Modalities:</strong> Misinterpreting relationships between text, images, and other data sources can lead to fantastical narratives.</li><li><strong>Overfitting Complexities:</strong> The increased complexity of multimodal data can make it harder for the model to distinguish actual patterns from coincidences, leading to elaborate but factually incorrect stories.</li></ul><p><strong>Why MLLMs are Prone to Metanarrative Errors:</strong></p><ul><li><strong>Multimodal Data Integration:</strong> MLLMs process information from various sources like text, images, and videos. If these data sources contain inconsistencies, the model might struggle to reconcile them and generate a unified narrative. This can lead to metanarrative errors where the LLM invents connections or explanations that don’t reflect reality.</li><li><strong>Limited Reasoning Abilities:</strong> Current MLLMs struggle with actual reasoning and understanding the nuances of language or imagery. This can lead them to create elaborate narratives based on superficial connections between data points.</li><li><strong>Metaphor Reconciliation:</strong> Reconciling complex metaphors across modalities (written vs. images) is challenging for MLLMs. They might struggle to understand the symbolic meaning of a written metaphor and generate a literal interpretation based on the corresponding image.</li></ul><blockquote><strong>Here’s how these terms might be used:</strong></blockquote><blockquote>“The LLM generated a localized error, which was a factually incorrect answer about historical events due to data-driven biases in its training corpus.”</blockquote><blockquote>“The multimodal LLM’s output contained problematic associations between images and text, leading to a metanarrative inconsistency.”</blockquote><p>The differences between text-based and multimodal LLMs highlight the importance of considering the training data and model architecture when evaluating the risk of the spectrum of error types. It’s important to note that this is a simplified analysis. The likelihood of localized error or metanarrative inconsistency depends on the specific training data, model architecture, and techniques.</p><p>Research on multimodal LLMs is ongoing, and mitigation strategies for hallucinations are being developed. As multimodal LLMs become more sophisticated, we’ll need to develop robust methods to ensure the accuracy and reliability of their outputs.</p><h3><strong>Techniques to Mitigate Errors along the Spectrum</strong></h3><p>While Retrieval-Augmented Generation (RAG) is a promising technique for improving factual accuracy in LLMs, it is not the only approach. Here is a breakdown of some other methods and how they can help minimize localized errors and metanarrative inconsistencies:</p><h4><strong>Techniques for Localized Errors</strong></h4><ul><li><strong>Data Augmentation</strong> involves artificially creating new training data points with slight variations. This helps LLMs generalize better and avoid overfitting specific patterns in the original data, which can lead to localized factual errors. An example of this could be an LLM trained in news articles. Data augmentation might involve creating new articles with slightly rephrased sentences or replacing synonyms to improve the model’s ability to handle unseen variations in language.</li><li><strong>Transfer Learning:</strong> Knowledge gained from a pre-trained LLM on a different task is leveraged for a specific new task. This can be helpful when limited data is available for the new task, reducing the risk of localized errors due to insufficient training data. An example is an LLM pre-trained in summarizing factual documents, which can be fine-tuned for summarizing scientific research papers. This pre-training provides a strong foundation for handling factual language, reducing the likelihood of localized errors in the scientific domain.</li><li><strong>Fact-Checking Integration:</strong> This involves integrating fact-checking APIs or databases within the LLM architecture as a “post-generation” check. The LLM generates the text first, and then the output is compared against external fact-checking resources (APIs or databases) to identify potential inconsistencies. For example, an LLM writing a news article can be connected to a fact-checking API that verifies the accuracy of dates, statistics, and named entities before publication.</li></ul><h4><strong>Techniques for Metanarrative Inconsistencies</strong></h4><ul><li><strong>Enhanced Contextual Understanding:</strong> This involves developing LLMs that can better understand the context and relationships between different pieces of information. This can help reduce metanarrative inconsistencies that arise from the LLM’s inability to reconcile disparate data points. For example, <strong>a</strong>n MLLM trained in historical text and fashion images might struggle to create a cohesive narrative about clothing trends. Improved contextual understanding would allow the MLLM to differentiate symbolic uses of clothing in text from literal representations in images.</li><li><strong>Multimodal Training with Consistency Checks:</strong> MLLMs designed to process multiple modalities (text, image, audio) can benefit from training emphasizing consistency across modalities. This reduces the risk of metanarrative inconsistencies arising from misinterpretations between different data types. An example of this is an LLM trained in news articles and videos, which can be specifically trained to identify and flag inconsistencies between the textual description of an event and the corresponding video footage.</li><li><strong>Explanation and Ablation Techniques</strong> allow researchers to understand how LLMs arrive at their outputs. By analyzing the reasoning process behind a metanarrative inconsistency, researchers can identify weaknesses in the model and develop targeted improvements. For example, researchers can pinpoint limitations in understanding historical context or artistic symbolism by analyzing the internal workings of an LLM that creates a nonsensical story based on a historical event and a painting.</li></ul><h3><strong>Choosing the Right Technique</strong></h3><p>The most effective technique depends on the type of error and the application of the LLM. In summary :</p><ul><li><strong>Localized Errors:</strong> RAG, Data augmentation, transfer learning, and fact-checking integration can be effective.</li><li><strong>Metanarrative Inconsistencies:</strong> Enhanced contextual understanding, multimodal training with consistency checks, and explanation/ablation techniques are more suitable.</li></ul><p>Remember, these techniques are constantly evolving, and researchers are exploring new ways to improve the accuracy and reliability of LLMs.</p><h3>Conclusion: Moving Beyond “Hallucinations”</h3><p>As I have outlined, the term “hallucination” is too simplistic and misleading for understanding generative AI errors. There is a spectrum of errors, from isolated factual or localized errors to elaborate, interwoven metanarrative inconsistencies. Correctly describing these distinctions is crucial for understanding the underlying model data limitations that may generate errors. It is also crucial for building trust in AI and ensuring it enhances, rather than hinders, human learning and productivity.</p><p>We can move towards a future where AI outputs are reliable and informative by utilizing a more precise lexicon and focusing on techniques like Retrieval-Augmented Generation (RAG) alongside other advancements that better ground and provide accurate and consistent quality outputs. This will require continued research on improving LLM reasoning abilities and understanding context.</p><p>Ultimately, fostering a collaborative relationship between people and AI, where we acknowledge its potential and limitations, will pave the way for <a href="https://medium.com/@BrianMGreen/responsible-use-of-ai-in-healthcare-strategies-for-balancing-opportunities-and-risk-in-a-b29ab27259b4">responsible and impactful future</a> applications of generative AI technologies.</p><h3><strong>Resources</strong></h3><p>Amodei, Dario, et al. “Concrete problems in AI safety.” arXiv preprint arXiv:1606.06565 (2016). <a href="https://arxiv.org/abs/1606.06565">https://arxiv.org/abs/1606.06565</a></p><p>Brundage, M., Avin, S., Clark, J., et al. (2018). The malicious use of artificial intelligence: Forecasting, prevention, and mitigation. <em>arXiv preprint </em><a href="https://arxiv.org/abs/1802.07228"><em>arXiv:1802.07228</em></a>.</p><p>Green, Brian M. (2024). Responsible Use of AI in Healthcare — Strategies for Balancing Opportunities and Risk in a Speculative Environment. <a href="https://medium.com/@BrianMGreen/responsible-use-of-ai-in-healthcare-strategies-for-balancing-opportunities-and-risk-in-a-b29ab27259b4">https://medium.com/@BrianMGreen/responsible-use-of-ai-in-healthcare-strategies-for-balancing-opportunities-and-risk-in-a-b29ab27259b4</a></p><p>Hatem R, Simmons B, Thornton JE. A Call to Address AI “Hallucinations” and How Healthcare Professionals Can Mitigate Their Risks. Cureus. 2023 Sep 5;15(9):e44720. <a href="https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10552880/pdf/cureus-0015-00000044720.pdf">doi: 10.7759/cureus.44720</a>. PMID: 37809168; PMCID: PMC10552880.</p><p>Lopatecki, Jason, Dhinakaran, Aparna, and Brown, Christopher. (2023). “A Novel Approach for Anomaly Detection Using Large Language Models.” <a href="https://arize.com/blog-course/anomaly-detection-using-large-language-models/">https://arize.com/blog-course/anomaly-detection-using-large-language-models/</a></p><p>Rocks, Jason W. and Mehta, Pankaj. (2022). “Memorizing without overfitting: Bias, variance, and interpolation in overparameterized models.” Physical Review Research, 4(1). <a href="https://journals.aps.org/prresearch/pdf/10.1103/PhysRevResearch.4.013201">doi 10.1103/PhysRevResearch.4.013201</a></p><p>Ye, Wenqian Ye, Zheng1, Guangtao, Cao, Xu, et al. (2024). “Spurious Correlations in Machine Learning: A Survey.” arXiv:2402.12715v1 [cs.LG] 20 Feb 2024. <a href="https://arxiv.org/html/2402.12715v1">https://arxiv.org/html/2402.12715v1</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=3dc84e379774" width="1" height="1" alt=""><hr><p><a href="https://medium.com/responsible-healthcare-ai/no-one-wants-a-counterfeit-golden-ticket-a-lexicon-for-the-spectrum-of-generative-ai-errors-3dc84e379774">No One Wants a Counterfeit “Golden Ticket:” A Lexicon for the Spectrum of Generative AI Errors</a> was originally published in <a href="https://medium.com/responsible-healthcare-ai">Responsible Healthcare AI</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>