<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Ankoor Talwar on Medium]]></title>
        <description><![CDATA[Stories by Ankoor Talwar on Medium]]></description>
        <link>https://medium.com/@ankoortalwar?source=rss-5605e615784b------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*fi9CqiXReE5Xgnx5MKQtmw.png</url>
            <title>Stories by Ankoor Talwar on Medium</title>
            <link>https://medium.com/@ankoortalwar?source=rss-5605e615784b------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Tue, 17 Nov 2026 08:33:54 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@ankoortalwar/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Limitations and Threats of AI; Lessons for Surgeons]]></title>
            <link>https://ankoortalwar.medium.com/limitations-threats-and-lessons-for-surgeons-0a1d1ec272e8?source=rss-5605e615784b------2</link>
            <guid isPermaLink="false">https://medium.com/p/0a1d1ec272e8</guid>
            <category><![CDATA[surgery]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[limitations]]></category>
            <category><![CDATA[regulation]]></category>
            <category><![CDATA[healthcare]]></category>
            <dc:creator><![CDATA[Ankoor Talwar]]></dc:creator>
            <pubDate>Mon, 25 Mar 2024 07:29:44 GMT</pubDate>
            <atom:updated>2024-03-25T07:34:18.960Z</atom:updated>
            <content:encoded><![CDATA[<h4>AI in Surgery: Part IV</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/0*OrUaF8kNitsAdzSz.jpg" /></figure><p><strong>LIMITATIONS AND THREATS</strong></p><p>Our enthusiasm for foundation models must be tempered by limitations for their adoption. These limitations span from hallucination and generation of false information, potential for bias, challenges with integration into our healthcare systems, processing the sheer size and heterogeneity of our data lakes, energy consumption, ethical concerns, and regulation concerns.</p><p>Researchers have found that as LLMs grow larger, their likelihood and extent of reinforcing bias and spreading misinformation also grows.62 These models are often trained from publicly available sources from the internet, which do not proportionally represent the breadth of human language and experience. “Hallucination”, the generation of false information, is a well-described phenomenon of foundation models. The misinformation is well-constructed and portrayed with high confidence. For example, Galactica, an early scientific LLM from Meta, created a scientific report on the health benefits of eating crushed glass, and ChatGPT generated a text on the benefits of adding crushed porcelain to an infant’s breastmilk.63</p><p>But the hallucination can be more subtle and beguiling, making it difficult for humans to discern without having prior knowledge. For example Beutel et al., asked ChatGPT to summarize some medical studies and found it hallucinated facts like the study period, participants, and conclusions, which they only knew having been familiar with the literature in question.64 Image-to-text models can also hallucinate inaccurate captions. While this is usually easy to discern, in the surgical setting this may lead to misdiagnosis and inappropriately guided treatment.</p><p>Hallucination will never disappear from foundation models, so it is an open question how much hallucination is acceptable for healthcare, especially given the ethical and legal concerns. Med-PaLM 2, was found to provide inaccurate/irrelevant information nearly twice as often as physicians when answering long-form consumer medical questions, underscoring how much improvement is needed.35 Fortunately several groups are working on ways to define metrics of hallucination, identify areas of development, and find solutions.65</p><p>Models can also generate false information by propagating misinformation from its training data, generally from the internet. Developers of these models have agreed there is risk for bias in terms of underrepresentation, overrepresentation, and misrepresentation.27 Indeed, information on the internet is mainly contributed by young males from developed countries; overrepresents Western history, politics, and values; and includes hate speech targeting marginalized groups. There is evidence of LLMs portraying each of these biases.66,67</p><p>Similar biases also affect image-based models. CLIP, an image classification model developed by OpenAI, once captioned a picture of NASA astronaut Eileen Collins in her space suit as “housewife in an orange jumpsuit”.68 Thus, as these models grow larger using internet data, they can increase power imbalances between groups, reinforce dominant viewpoints from the internet, and potentially exacerbate inequalities. Nevertheless, the path forward is to identify biases and build in counterbalances to produce better models. RAG is one potential method to enhance healthcare specific models. Certainly, this is an emerging focus of research in the field.</p><p>Healthcare providers are broadly in agreement about the necessity of foundation models in harnessing the vast healthcare data lake to create generalist medical intelligence.5 However, a major challenge for healthcare entrepreneurs has always been integration of innovation in the industry. Our healthcare system is highly fragmented and involves many players, it is difficult to find ways to produce meaningful progress. An example is the EHR; the EHR marketplace is becoming increasingly concentrated among a top few companies, but interoperability and integration across these platforms is an important concern for digital innovators.69 In addition to the hospital and clinic settings, one must also consider databases associated with pharmacies, home health agencies, dialysis centers, and other ancillary care services. It is promising that previously discussed industry leaders in clinical documentation are also invested in bringing generative AI to their services.45,47</p><p>This challenge is further amplified by the size and heterogeneity of our data lakes. For example, in the NLP space, clinical documents from different sources may use medical abbreviations and terminology differently. Fortunately, researchers are working on platforms which can account for this heterogeneity.70 Future LLMs could explore this framework for more powerful trans-disciplinary language extraction. Undoubtedly finding key data among multi-modal, high-dimensional healthcare datasets and extracting their value will continue to be a challenge for researchers and clinicians, but it is one we are well-suited to overcome.</p><p>Energy consumption in data science is a notable threat. Foundation models are particularly energy-dependent in development, training, and usage.71 GPT-3 was trained on 10,000 NVIDIA V100 GPUs and required about 190,000 kWh and producing 85,000 kg CO2 equivalents, the same as 18 gasoline powered cars in one year.72 As healthcare providers, we must acknowledge the well-established concerns of energy consumption leading to environmental detriment and poor population health. The next generation of foundation model developers must find ways to curtail energy usage or incorporate cleaner energy sources. Fortunately, this is a top priority for the AI community and it appears will improve with technological progress.72</p><p>There are overarching ethical considerations when discussing foundation models and artificial intelligence in the future of healthcare. It relates to all stakeholders of our healthcare system: patients, providers, payers, the government. For example, using a model to diagnose an illness or predict an outcome has greater ramifications than designing a model to communicate information.34 Previous discussions on bias and disinformation also remain relevant. There are also questions on how to ethically use AI models on genetic data, and how to use genetic information to improve patient care. Finally, there is an ongoing debate about the risks of artificial general intelligence (AGI). AGI refers to AI systems that can perform a wide range of tasks, and which can adapt to new tasks and environments. There are currently no known examples of true AGI, as there are many technical and philosophical challenges that need to be overcome. Questions of machines achieving consciousness, for example, are within the realm of AGI. This only reinforces the importance of involving all healthcare stakeholders in the future development of AI.</p><p>As foundation models have proliferated and brought to light the disruptive potential for artificial intelligence, coupled with its notable limitations, there has been renewed interest in regulating this technology. Indeed, both developers and consumers stand to benefit from regulations in this space, especially as it relates to healthcare and surgery. To date the most prominent regulations for AI have occurred in Europe. In March 2023, Italian regulators temporarily banned ChatGPT, under the premise that it violated the European General Data Protection Regulation (GDPR). Specifically, there were concerns over OpenAI’s access and processing of people’s private data, access to minors, and misinformation.73 That ban has since been lifted, as OpenAI has worked to address these concerns. The European Artificial Intelligence Act (AIA), the Data Governance Act (DGA), the Open Data Directive (ODD) are a few other regulations.74 While the DGA and ODD work together to protect public health data for development of beneficial technology, the AIA is fundamentally flawed in its regulative capacity. The AIA outlines guidelines for AI development in the healthcare space which is human-centered, respects privacy, and encourages innovation.74,75 It also prohibits certain systems which are deemed too high-risk. However, the law is meant for only those systems which are directly used in healthcare. This means this regulation may not apply to foundation models, which are intermediate models generalized for many purposes. There is similar uncertainty for healthcare-specific foundation models, such as ClinicalGPT or Med-PaLM 2, as they currently have no specific clinical applications but may be used to build even further downstream healthcare applications. The regulatory standards around generative AI are even murkier in the U.S., as there is no comprehensive federal AI legislation.</p><p>It remains to be seen whether developers of the foundation models or developers of downstream applications have greater responsibility in protecting against bias and unacceptable risk. It has been suggested in <em>Gonzalez v. Google</em> that technology which create content are not protected from liabilities in Section 230 of the Communications Decency Act.76 This could make parent entities like OpenAI accountable for downstream models. Certainly, AI developers, healthcare entrepreneurs, and the public sector must better define the scope and practice of foundation models in the next generation of healthcare.</p><p>Our enthusiasm for foundation models must be tempered by a commitment to scientific rigor. While it may be easier for surgeons in small teams to prototype powerful AI models, such models must be validated in larger populations and assessed in clinical implementation. This will require buy-in from larger groups of stakeholders (i.e. patients, providers, institutions). A potential guide for development of AI-based surgical decision support tools may be the Algorithm-Based Clinical Decision Support (ABCDS) Model of Oversight.77 This framework consists of four stages for AI development: model development, silent evaluation/feasibility, prospective effectiveness evaluation, and general deployment. The framework crucially emphasizes several checkpoints for an ABCDS Oversight Committee to monitor the tool in production. This type of AI governance ecosystem promotes ethical model development, impact, quality control, accountability, and collaboration. Other ethical guidelines share these concepts and should also be considered.78,79</p><p><strong>LESSONS FOR SURGEONS</strong></p><p>The immense potential of generative AI and foundation models within surgery must be realized to continue pushing the boundaries of optimal surgical care. Because of the vast promise and notable limitations of generative AI, it is vital for surgeons to be involved its proliferation. Empowered by foundation models, even small teams with limited quantities of healthcare data can employ “few-shot” learning to prototype intricate AI applications for subsequent validation and development. But the information within the data lake is too complex for software engineers to tackle alone. Inappropriate data, metrics, and research questions will create confounding interpretations. It is conceivable, for example, that an AI model might determine a certain surgical dressing plays an outsized role in risk of postoperative complications, just because of the nature of the data available. Thus, the clinical context remains essential. Surgeons can direct areas of focus and provide that necessary clinical context to ensure success.</p><p>Now more than ever, it is crucial for surgeons to have a basic proficiency in AI methods and current developments. This way they can better ideate and communicate with software engineers and data scientists to form productive collaborations. They can also better communicate with institutional leadership to form a healthy AI development ecosystem that promotes innovation and patient-centered care. As such, the authors propose improved education for surgeons in AI. Incorporating an evidence-based mindset within surgical resident education is not a new idea, and appreciating the nuances of AI is the natural progression for training future surgeon-scientists.80 This education should merge teaching of principles of AI with topics like the ethical deployment of AI in healthcare.</p><p><strong>CONCLUSION</strong></p><p>Surgeons must embrace the power of modern computing to harness the potential of big data in healthcare. Healthcare data is vast and heterogenous, and can be conceptualized as a data lake. Artificial intelligence will be crucial in harnessing the structured, semi-structured, and unstructured data in each patient’s data lake. In the past, AI development in healthcare has been limited by availability of data and computing resources. Foundation models have since democratized AI methods for surgeons and researchers. Foundation models have also unlocked a new era of generative AI, an emerging force which can revolutionize surgical care and patient outcomes by creating new multimedia information. However, enthusiasm for this technology must be tempered by its notable limitations including hallucination, misinformation, bias, regulation, environmental concerns, and ethical concerns. It is crucial for surgeons to understand the state of AI, the promise of foundation models, and their risks to empower the next generation of patient-centered surgical care.</p><p><em>References</em></p><p>62. Zhang D, Maslej N, Brynjolfsson E, et al. The AI Index 2022 Annual Report. Published online May 2, 2022. doi:10.48550/arXiv.2205.03468</p><p>63. Birhane A. ChatGPT, Galactica, and the Progress Trap. <em>Wired</em>. Accessed May 8, 2023. <a href="https://www.wired.com/story/large-language-models-critique/">https://www.wired.com/story/large-language-models-critique/</a></p><p>64. Beutel G, Geerits E, Kielstein JT. Artificial hallucination: GPT on LSD? <em>Crit Care</em>. 2023;27:148. doi:10.1186/s13054–023–04425–6</p><p>65. Ji Z, Lee N, Frieske R, et al. Survey of Hallucination in Natural Language Generation. <em>ACM Comput Surv</em>. 2023;55(12):1–38. doi:10.1145/3571730</p><p>66. Zhang H, Lu AX, Abdalla M, McDermott M, Ghassemi M. Hurtful Words: Quantifying Biases in Clinical Contextual Word Embeddings. Published online March 11, 2020. doi:10.48550/arXiv.2003.11515</p><p>67. Guo W, Caliskan A. Detecting Emergent Intersectional Biases: Contextualized Word Embeddings Contain a Distribution of Human-like Biases. In: <em>Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society</em>. AIES ’21. Association for Computing Machinery; 2021:122–133. doi:10.1145/3461702.3462536</p><p>68. Birhane A, Prabhu VU, Kahembwe E. Multimodal datasets: misogyny, pornography, and malignant stereotypes. Published online October 5, 2021. doi:10.48550/arXiv.2110.01963</p><p>69. Sorace J, Wong HH, DeLeire T, et al. Quantifying the competitiveness of the electronic health record market and its implications for interoperability. <em>International Journal of Medical Informatics</em>. 2020;136:104037. doi:10.1016/j.ijmedinf.2019.104037</p><p>70. Roh J, Park S, Kim BK, Oh SH, Lee SY. Unsupervised multi-sense language models for natural language processing tasks. <em>Neural Netw</em>. 2021;142:397–409. doi:10.1016/j.neunet.2021.05.023</p><p>71. Desislavov R, Martínez-Plumed F, Hernández-Orallo J. Compute and Energy Consumption Trends in Deep Learning Inference. Published online September 12, 2021. doi:10.48550/arXiv.2109.05472</p><p>72. Patterson D, Gonzalez J, Hölzle U, et al. The Carbon Footprint of Machine Learning Training Will Plateau, Then Shrink.</p><p>73. Intelligenza artificiale: il Garante blocca ChatGPT. Raccolta illecita di dati personali. Assenza di sistemi per la verifica dell’età dei minori. Accessed May 10, 2023. <a href="https://www.garanteprivacy.it/home/docweb/-/docweb-display/docweb/9870847">https://www.garanteprivacy.it:443/home/docweb/-/docweb-display/docweb/9870847</a></p><p>74. Wójcik MA. Foundation Models in Healthcare: Opportunities, Biases and Regulatory Prospects in Europe. In: Kő A, Francesconi E, Kotsis G, Tjoa AM, Khalil I, eds. <em>Electronic Government and the Information Systems Perspective</em>. Lecture Notes in Computer Science. Springer International Publishing; 2022:32–46. doi:10.1007/978–3–031–12673–4_3</p><p>75. van Oirschot J, Ooms G. <em>Interpreting the EU Artificial Intelligence Act for the Health Sector</em>. Health Action International; 2022. <a href="https://haiweb.org/wp-content/uploads/2022/02/Interpreting-the-EU-Artificial-Intelligence-Act-for-the-Health-Sector.pdf">https://haiweb.org/wp-content/uploads/2022/02/Interpreting-the-EU-Artificial-Intelligence-Act-for-the-Health-Sector.pdf</a></p><p>76. Haupt CE, Marks M. AI-Generated Medical Advice — GPT and Beyond. <em>JAMA</em>. 2023;329(16):1349–1350. doi:10.1001/jama.2023.5321</p><p>77. Algorithm-Based Clinical Decision Support (ABCDS) Oversight — Duke AI Health. Accessed May 10, 2023. <a href="https://aihealth.duke.edu/algorithm-based-clinical-decision-support-abcds/">https://aihealth.duke.edu/algorithm-based-clinical-decision-support-abcds/</a></p><p>78. Ethics and governance of artificial intelligence for health. Accessed October 1, 2023. <a href="https://www.who.int/publications-detail-redirect/9789240029200">https://www.who.int/publications-detail-redirect/9789240029200</a></p><p>79. Solanki P, Grundy J, Hussain W. Operationalising ethics in artificial intelligence for healthcare: a framework for AI developers. <em>AI Ethics</em>. 2023;3(1):223–240. doi:10.1007/s43681–022–00195-z</p><p>80. Ward TM, Mascagni P, Madani A, Padoy N, Perretta S, Hashimoto DA. Surgical data science and artificial intelligence for surgical education. <em>J Surg Oncol</em>. 2021;124(2):221–230. doi:10.1002/jso.26496</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=0a1d1ec272e8" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Generative AI for Healthcare]]></title>
            <link>https://ankoortalwar.medium.com/generative-ai-for-healthcare-a5f469b80a32?source=rss-5605e615784b------2</link>
            <guid isPermaLink="false">https://medium.com/p/a5f469b80a32</guid>
            <category><![CDATA[nlp]]></category>
            <category><![CDATA[surgery]]></category>
            <category><![CDATA[generative-ai]]></category>
            <category><![CDATA[large-language-models]]></category>
            <dc:creator><![CDATA[Ankoor Talwar]]></dc:creator>
            <pubDate>Mon, 25 Mar 2024 07:23:45 GMT</pubDate>
            <atom:updated>2024-03-25T07:30:31.386Z</atom:updated>
            <content:encoded><![CDATA[<h4>AI in Surgery: Part III</h4><p>Foundation models, like ML as a whole, can also be separated into discriminative and generative models. The first generation of foundation models were discriminative, performing solely classification tasks (e.g. BERT for NLP). However, the newer foundation models have disrupted the software industry because they are generative, able to create user-friendly information, while still performing discriminative tasks with high accuracy.</p><p><em>Large Language Models</em></p><p>Certainly, the most well-developed generative foundation models are large language models (LLMs), such as Generative Pre-training Transformer (GPT).14 GPT is the underlying technology behind the consumer-facing platform, ChatGPT™ from OpenAI©. Alternate LLMs include Pathways Language Model (PaLM) from Google©, Large Language Model Meta AI (LLaMA) from Meta©, BigScience Large Open-science Open-access Multilingual Language Model (BLOOM) lead by HuggingFace©.15,27–29 All these models have varying levels of accuracy and efficiency with natural language processing that relate to how they are built.</p><p>It is critical to make a distinction here, as the set of LLMs includes both non-generative (e.g. BERT) and generative models. However, the largest, most powerful, and most impactful LLMs are generative, which will encompass the bulk of our forthcoming discussion.</p><p>As previously mentioned, techniques in ML and NLP are combined to create an LLM. In general, there are 3 overarching steps for training an LLM: 1) pre-training a transformer network, a type of artificial neural network, through supervised learning; 2) running the transformer network<a href="#_ftn1">[1]</a> on certain inputs to generate outputs, gathering human feedback on these input-outputs, and using the human preference data to train a reward model; and 3) fine-tuning the transformer network using the reward model through unsupervised reinforcement learning.14 Steps 2 and 3 have together been termed “reinforcement learning with human feedback”. The characteristics of the pre-trained model, the reward model, and the methods of reinforcement learning all determine the performance of the final LLM. As these models proliferate, researchers must find the optimal way to fine-tune an LLM for healthcare purposes.</p><p>It is worth knowing how LLMs handle text-based queries via tokenization. Tokenization is the process of splitting text (sequences of characters) into short “meaningful” subsequences, (Figure 5). Every LLM has an in-built mechanism (i.e. tokenizer) which tokenizes the input text. The tokenizer is pre-trained to identify common words and patterns of text. The LLM then processes each token one by one, statistically modeling relationships between tokens, to generate a response.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*AC2V7k_V23h-tiJQRBMX1A.png" /><figcaption>Figure 5. An example of how a LLM might tokenize an input string of text.</figcaption></figure><p>GPT was developed as a “pre-training” model, which means that it was trained on a large dataset without any specific application in mind. The previous iteration, GPT-3, includes 175 billion parameters in its neural network.14 PaLM has several iterations, with its largest model having 540 billion parameters.27 Technical details for GPT-4 have not been released, but its pre-trained model is rumored to include over 1 trillion parameters.31 These models are trained to learn the patterns and structure of language in a general sense so that they can then be further trained for specific tasks. Indeed, a pre-trained GPT-4 is already a powerful tool, as it has achieved passing scores on reproductions of the USMLE Steps 1, 2CK, and 3 exams.32 Developers can thereby use LLMs, like GPT, as a strong “foundation” to develop more specialized NLP models.</p><p>Because GPT is a generative model, it can perform tasks discussed earlier such as text summarization, language translation, and question answering. Further, is can also perform discriminative tasks because their holistic modeling of their immense body of pre-trained data; any further specialized training allows them to do subtasks like information extraction with high fidelity.33</p><p>Fine-tuning of GPT or other generative LLMs (e.g. PaLM, LLaMA, BLOOM) can create specialized models for medical and surgical applications. Work from Singhal et al. at Google, established Med-PaLM, a fine-tuned version of Flan-PaLM (itself a fine-tuned version of PaLM), and found it performed well on MultiMedQA, a large medical question answering dataset.34 Interestingly, pre-trained GPT-4 subsequently outperformed Med-PaLM on the same dataset, underscoring its specific potential in this space.32 Most recently, the Google team released Med-PaLM 2 which was competitive with and even outperformed GPT-4 on some domains of the same dataset.35 While these results are interesting for clinicians to consider and potentially build off of, it underscores that development of healthcare-facing models has largely been confined to AI research instead of for clinical translation.</p><p>Researchers have gradually been exploring how LLMs can be further refined for the healthcare setting. Indeed the last few months have seen a proliferation of healthcare-oriented startups employing generative LLMs. Doximity© has harnessed GPT-4 to create Docs GPT, a HIPAA-compliant online interface for providers to create outlines of clinical correspondence, patient education materials, and more.36 Kahun© has used GPT-4 to summarize the most up-to-date literature to support clinical decision-making in primary and urgent care settings.37 Livewello© allows patients to upload raw genetic data from third party sources and answers questions about genetic risks.38</p><p>While these models are already being applied for medical research and patient health information, more clinically-oriented applications of generative LLMs could even further streamline surgical care. Figure 6 depicts these applications, outlining both discriminative and generative tasks. The discriminative tasks include classification (e.g. medical coding, document labelling) and information extraction. The generative tasks include clinical text summarization, generation, translation, and medical question answering.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*qqg0WS74Km6nhYzuEfFSVA.png" /><figcaption>Figure 6. The multifaceted applications of NLP in surgery include. Pink tasks are discriminative, blue tasks are generative, purple tasks can be either discriminative or generative.</figcaption></figure><p>It is commonplace for patient EHRs to now contain hundreds of clinical documents. Manually searching for key information can be quite tedious, underscoring the necessity of NLP in this realm. However, the language in clinical documentation has unique vocabulary and syntax compared to other types of text. Further, clinical jargon can even be confusing to surgeons across disciplines. For example, “RA” can mean room air, right arm, right atrium, or rheumatoid arthritis.39 Is no surprise, therefore, that information extraction and summarization is the most prominent application of NLP in the clinical world.</p><p>Information extraction is considered a discriminative task. For example, identifying the type of suture used to close the abdominal fascia after a laparotomy, can be considered discriminative. This type of information extraction is called “named entity recognition” — deriving the names of patients, medical procedures, medications, etc. directly written in a document.40 The previously discussed ClinicalBERT, and more downstream models, can perform this task.</p><p>Summarization, on the other hand, is a generative task, and requires a more holistic understanding of what a document says to outline its meaning. For clinical narrative summarization, researchers have had promising results in with the GPT architecture.41 Agrawal et al. recently demonstrated that pre-trained (i.e. “zero-shot”, without further training) and “few-shot” trained versions of GPT were able to perform the discriminative task of information extraction from clinical notes with high accuracy, as well.42 Peng, et al., trained GPT-3 on the GatorTron repository of clinical documentation and other data sources to create GatorTronGPT.43 This model was able to perform discriminative information extraction with greater accuracy than models like BioGPT, PubMedBERT, and GPT-2. Further, in their study, GatorTronGPT actually generated clinical documentation which passed the Turing test after evaluation by multiple physicians, reinforcing the potential of this technology in clinical documentation.</p><p>Generative foundation models outside of GPT also have potential in the healthcare space. A group from China recently developed ClinicalGPT, a model created by fine-tuning BLOOM on clinical documentation, including various English and Chinese language medical datasets.44 They found it performed better than ChatGLM (a Chinese analog of ChatGPT), LLAMA, and pre-trained BLOOM on medical question answering tasks.</p><p>Indeed, future development of clinically-oriented LLMs should use generative foundation models to solve both the discriminative and generative tasks of NLP. One can imagine a future where the search bar of an EHR is integrated with an LLM to allow rapid and accurate retrieval of clinical information from the depths of a patient’s record. Indeed, industry leader Epic has already announced a forthcoming integration of GPT-4 into its platform.45 This would allow surgeons to retrieve very specific information immediately to aid in patient care. It would also allow surgeons to extract information for powerful analytics and predictive modeling in surgical care.</p><p>LLMs have the potential to radically alter the surgical workflow by reducing administrative burden associated with medical billing and clinical documentation. There are already groups working on using generative NLP and speech-to-text transcription systems to help surgical providers dictate and auto-populate necessary clinical documents.46 Several healthcare startups, such as Ambience©, are attempting to generate clinical documentation real time during clinical encounters.24 Industry leader Nuance© also announced a new service which will incorporate GPT-4.47 These systems may also be trained to learn the ICD and CPT code system to automatically assign codes for specific patient-provider interactions.48</p><p>Generative LLMs should also assist in translation of clinical materials. Automatically translating clinical documents for patients whose primary language is not English would be an immense benefit to the healthcare systems. This is particularly true in surgical care, where a highly specialized surgeon may need to communicate surgical results to a patient or their primary care provider who is more comfortable with another language. The scope of translation also encompasses populations within a language but in which individuals have varying levels of health literacy. It has been suggested that translating clinical documents uniquely for geriatric patients may improve the patient-provider interaction.49 More generally, translating medical terms like “PCP” and “MI” into terms that are more familiar to the patient population will be useful, for example in post-operative instructions. Generative LLMs can provide the basis for such translation.</p><p>Access to 24/7 care on-demand remains an ambitious goal for the healthcare community. Preoperative and postoperative interactions are essential for optimal surgical care. “Question answering” is a discipline in NLP involving systems that automatically answer questions posed by humans, using a relevant context and in a natural language. As previously mentioned, both GPT-4 and Med-PaLM 2 have shown promise in the medical question answering space.32,34</p><p>Systems which answer patient questions preoperatively and postoperatively will be especially beneficial to surgeons. A surgical question answering platform, powered by a LLM, may aid in postoperative monitoring and improve to access the healthcare system. An example generated from ChatGPT demonstrates this potential (see Figure, Supplemental Digital Content 1, which answers “My surgical dressing is soaked in blood. What do I do?”). While the response shows deference to healthcare providers, it provides important guidance to elevate and apply pressure to the surgical site, and directions on when to seek immediate medical assistance. If a model like this is integrated with a patient’s EHR, it may allow for more specific information based on the patient’s history, the surgery performed, and the surgeon’s postoperative instructions. Wang et al., have experimented with retrieval-augmented generation (RAG) to incorporate this specificity for medical information.50 RAG allows outside knowledge sources to augment the input prompt to an LLM to generate a more accurate response. Their preliminary work shows that RAG-assisted ChatGPT is able to provide a more focused medical response to a sample patient question. Similar techniques may also tailor the responses to the patient’s language or literacy level.</p><p>Interpreting patient data for diagnosis would be a more powerful example of question answering. This would be especially useful application in resource-limited settings around the world. Shea, et al. attempted to use pre-trained GPT-4 to diagnose patients from written clinical histories.51 According to the authors, this proof-of-concept exploration was both “challenging and promising”. While these prototypes of medical question answering undoubtedly need refinement for clinical translation, input from surgeons can help guide synergistic development of this technology.</p><p><em>Foundation Models and Imaging</em></p><p>The promise of generative AI through foundation models is also being applied to image interpretation and generation. Latent diffusion models (LDMs) are foundation models which can generate images from text inputs.52 These models are trained to map the latent space of an image generator network to text descriptions. The models then iterate to develop highly detailed images in this space which correspond to the descriptions. Some well-known LDMs are DALL-E (Deep Algorithms for Lifelong Learning — Evolution), developed by OpenAI and Stable Diffusion.52,53 These foundation models have been trained on large datasets of natural images and their captions.</p><p>Chambon, et al. has explored how Stable Diffusion can be fine-tuned on radiology using the MIMIC-CXR database of chest X-rays.54 They subsequently created a specialized model, RoentGen, which can generate normal and pathologic chest X-rays.55 Several examples of RoentGen’s outputs are adapted in Figure 7. Undoubtedly, they improve upon the pre-trained Stable Diffusion. Pinaya, et al. used a similar paradigm to create their own LDMs which can generate normal T1-weighted brain MRIs.56 Their models can alter output images to align with variables like age, sex, and certain structural volumes. These generative technologies can be applied to surgical education. Artificial medical images can test trainees in an unbiased fashion, without knowing any other patient details.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*zO15UOnz5-z4CB2rF8ueIQ.png" /><figcaption>Figure 7. Adapted with permission from Chambon, et al. (2022).55 Depicted are normal and pathologic examples of output synethtic chest X-rays from the RoentGen specialized latent diffusion model.</figcaption></figure><p>The newest foundation models, such as GPT-4, are multi-modal, allowing for both vision and text input and generating text outputs.57 Such models are coming to surgery. Yuan, et al. published their development of Florence, a vision-to-text foundation model which can interpret a vast array of images, fine and coarse, static and dynamic, two-dimensional and multi-dimensional.58 This model was the first general-purpose computer vision system which can automatically identify structures within images and develop captions. Other models have since been developed, such as Segment Anything Model (SAM) by Meta.59 Shi, et al. applied a pre-trained SAM to “segment” — highlight regions of interest — various types of medical imaging (radiology, retinal and endoscopic optical coherence tomography, etc.). As expected, while the zero-shot SAM was moderately accurate at these tasks, a few-shot trained SAM performed even better.60 Recent work from Wu, et al. created RadFM, the first generative radiology-specific foundation model for both 2D and 3D images.61 RadFM established new state-of-the-art benchmarks in various generative vision tasks, such as modality recognition, visual question answering, report generation, and diagnosis.</p><p>Certainly, generative deep learning is not new in the medical imaging space. There are already deep learning models which can segment medical images, improve image resolution, and remove artifact. The advantage of these generative foundation models is that they enable transfer learning to create even more powerful models, allow for generation between media, and can be readily adapted by end users for more specific tasks. Fine-tuning of Florence, SAM, RadFM or similar models might enable surgeons to automatically interpret medical images in varying levels of detail. Further, this technology might enable surgeons to automate the identification of key anatomic patterns in their patients. All these advancements would improve surgeons’ ability to risk-assess, diagnose, and treat patients. The field of surgery is only beginning to realize the immense potential of foundation models.</p><p><a href="#_ftnref1">[1]</a> Transformer models are unique because they allow for attention-based modeling, which enables models to decide what inputs are most important. A broader discussion of transformer networks can be found in a landmark paper by Vaswani et al.30</p><p><em>References</em></p><p>27. Chowdhery A, Narang S, Devlin J, et al. PaLM: Scaling Language Modeling with Pathways. Published online October 5, 2022. doi:10.48550/arXiv.2204.02311</p><p>28. Touvron H, Lavril T, Izacard G, et al. LLaMA: Open and Efficient Foundation Language Models. Published online February 27, 2023. doi:10.48550/arXiv.2302.13971</p><p>29. Workshop B, Scao TL, Fan A, et al. BLOOM: A 176B-Parameter Open-Access Multilingual Language Model. Published online March 13, 2023. doi:10.48550/arXiv.2211.05100</p><p>30. Vaswani A, Shazeer N, Parmar N, et al. Attention Is All You Need. Published online December 5, 2017. doi:10.48550/arXiv.1706.03762</p><p>31. Schreiner M. GPT-4 architecture, datasets, costs and more leaked. THE DECODER. Published July 11, 2023. Accessed September 30, 2023. <a href="https://the-decoder.com/gpt-4-architecture-datasets-costs-and-more-leaked/">https://the-decoder.com/gpt-4-architecture-datasets-costs-and-more-leaked/</a></p><p>32. Nori H, King N, McKinney SM, Carignan D, Horvitz E. Capabilities of GPT-4 on Medical Challenge Problems. Published online March 20, 2023. doi:10.48550/arXiv.2303.13375</p><p>33. Hu Y, Ameer I, Zuo X, et al. Zero-shot Clinical Entity Recognition using ChatGPT. Published online March 28, 2023. doi:10.48550/arXiv.2303.16416</p><p>34. Singhal K, Azizi S, Tu T, et al. Large Language Models Encode Clinical Knowledge. Published online December 26, 2022. doi:10.48550/arXiv.2212.13138</p><p>35. Singhal K, Tu T, Gottweis J, et al. Towards Expert-Level Medical Question Answering with Large Language Models. Published online May 16, 2023. doi:10.48550/arXiv.2305.09617</p><p>36. Docs GPT. Accessed September 18, 2023. <a href="https://www.doximity.com/docs-gpt-info">https://www.doximity.com/docs-gpt-info</a></p><p>37. Kahun — Solutions. Accessed September 18, 2023. <a href="https://www.kahun.com/solutions">https://www.kahun.com/solutions</a></p><p>38. Livewello | Genome Data Analysis. Accessed September 18, 2023. <a href="https://livewello.com/">https://livewello.com/</a></p><p>39. Shilo L, Shilo G. Analysis of abbreviations used by residents in admission notes and discharge summaries. <em>QJM</em>. 2018;111(3):179–183. doi:10.1093/qjmed/hcx241</p><p>40. Huang K, Altosaar J, Ranganath R. ClinicalBERT: Modeling Clinical Notes and Predicting Hospital Readmission. Published online November 28, 2020. doi:10.48550/arXiv.1904.05342</p><p>41. Chintagunta B, Katariya N, Amatriain X, Kannan A. Medically Aware GPT-3 as a Data Generator for Medical Dialogue Summarization. Published online September 9, 2021. doi:10.48550/arXiv.2110.07356</p><p>42. Agrawal M, Hegselmann S, Lang H, Kim Y, Sontag D. Large Language Models are Few-Shot Clinical Information Extractors. Published online November 30, 2022. doi:10.48550/arXiv.2205.12689</p><p>43. Peng C, Yang X, Chen A, et al. A Study of Generative Large Language Model for Medical Research and Healthcare. Published online May 22, 2023. doi:10.48550/arXiv.2305.13523</p><p>44. Wang G, Yang G, Du Z, Fan L, Li X. ClinicalGPT: Large Language Models Finetuned with Diverse Medical Data and Comprehensive Evaluation. arXiv.org. Published June 16, 2023. Accessed October 1, 2023. <a href="https://arxiv.org/abs/2306.09968v1">https://arxiv.org/abs/2306.09968v1</a></p><p>45. Epic and Microsoft Bring GPT-4 to EHRs | Epic. Accessed September 30, 2023. <a href="https://www.epic.com/epic/post/epic-and-microsoft-bring-gpt-4-to-ehrs">https://www.epic.com/epic/post/epic-and-microsoft-bring-gpt-4-to-ehrs</a></p><p>46. Kaufman DR, Sheehan B, Stetson P, et al. Natural language processing-enabled and conventional data capture methods for input to electronic health records: A comparative usability study. <em>JMIR Medical Informatics</em>. 2016;4(4). doi:10.2196/medinform.5544</p><p>47. Nuance and Microsoft Announce the First Fully AI-Automated Clinical Documentation Application for Healthcare — Mar 20, 2023. Accessed September 30, 2023. <a href="https://news.nuance.com/2023-03-20-Nuance-and-Microsoft-Announce-the-First-Fully-AI-Automated-Clinical-Documentation-Application-for-Healthcare">https://news.nuance.com/2023-03-20-Nuance-and-Microsoft-Announce-the-First-Fully-AI-Automated-Clinical-Documentation-Application-for-Healthcare</a></p><p>48. Dong H, Falis M, Whiteley W, et al. Automated clinical coding: what, why, and where we are? <em>NPJ Digit Med</em>. 2022;5(1):159. doi:10.1038/s41746–022–00705–7</p><p>49. Abdi S, Witte L de, Hawley M. Exploring the Potential of Emerging Technologies to Meet the Care and Support Needs of Older People: A Delphi Survey. <em>Geriatrics</em>. 2021;6(1):19. doi:10.3390/geriatrics6010019</p><p>50. Wang C, Ong J, Wang C, Ong H, Cheng R, Ong D. Potential for GPT Technology to Optimize Future Clinical Decision-Making Using Retrieval-Augmented Generation. <em>Ann Biomed Eng</em>. Published online August 2, 2023. doi:10.1007/s10439–023–03327–6</p><p>51. Shea YF, Lee CMY, Ip WCT, Luk DWA, Wong SSW. Use of GPT-4 to Analyze Medical Records of Patients With Extensive Investigations and Delayed Diagnosis. <em>JAMA Network Open</em>. 2023;6(8):e2325000. doi:10.1001/jamanetworkopen.2023.25000</p><p>52. Ramesh A, Pavlov M, Goh G, et al. Zero-Shot Text-to-Image Generation. Published online February 26, 2021. doi:10.48550/arXiv.2102.12092</p><p>53. Rombach R, Blattmann A, Lorenz D, Esser P, Ommer B. High-Resolution Image Synthesis with Latent Diffusion Models. In: <em>2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)</em>. IEEE; 2022:10674–10685. doi:10.1109/CVPR52688.2022.01042</p><p>54. Chambon P, Bluethgen C, Langlotz CP, Chaudhari A. Adapting Pretrained Vision-Language Foundational Models to Medical Imaging Domains. Published online October 8, 2022. doi:10.48550/arXiv.2210.04133</p><p>55. Chambon P, Bluethgen C, Delbrouck JB, et al. RoentGen: Vision-Language Foundation Model for Chest X-ray Generation. Published online November 23, 2022. doi:10.48550/arXiv.2211.12737</p><p>56. Pinaya WHL, Tudosiu PD, Dafflon J, et al. Brain Imaging Generation with Latent Diffusion Models. Published online September 15, 2022. doi:10.48550/arXiv.2209.07162</p><p>57. OpenAI. GPT-4 Technical Report. Published online March 27, 2023. doi:10.48550/arXiv.2303.08774</p><p>58. Yuan L, Chen D, Chen YL, et al. Florence: A New Foundation Model for Computer Vision. Published online November 22, 2021. doi:10.48550/arXiv.2111.11432</p><p>59. Kirillov A, Mintun E, Ravi N, et al. Segment Anything. arXiv.org. Published April 5, 2023. Accessed October 1, 2023. <a href="https://arxiv.org/abs/2304.02643v1">https://arxiv.org/abs/2304.02643v1</a></p><p>60. Poudel K, Dhakal M, Bhandari P, Adhikari R, Thapaliya S, Khanal B. Exploring Transfer Learning in Medical Image Segmentation using Vision-Language Models. arXiv.org. Published August 15, 2023. Accessed October 1, 2023. <a href="https://arxiv.org/abs/2308.07706v2">https://arxiv.org/abs/2308.07706v2</a></p><p>61. Wu C, Zhang X, Zhang Y, Wang Y, Xie W. Towards Generalist Foundation Model for Radiology. arXiv.org. Published August 4, 2023. Accessed October 1, 2023. <a href="https://arxiv.org/abs/2308.02463v3">https://arxiv.org/abs/2308.02463v3</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a5f469b80a32" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[What is Artificial Intelligence? A Primer for Physicians]]></title>
            <link>https://ankoortalwar.medium.com/what-is-artificial-intelligence-a-primer-for-physicians-0ffc65df4144?source=rss-5605e615784b------2</link>
            <guid isPermaLink="false">https://medium.com/p/0ffc65df4144</guid>
            <category><![CDATA[healthcare]]></category>
            <category><![CDATA[computer-vision]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[natural-language-processi]]></category>
            <category><![CDATA[foundation-models]]></category>
            <dc:creator><![CDATA[Ankoor Talwar]]></dc:creator>
            <pubDate>Mon, 25 Mar 2024 07:14:33 GMT</pubDate>
            <atom:updated>2024-03-25T07:30:44.126Z</atom:updated>
            <content:encoded><![CDATA[<h4>AI in Surgery: Part II</h4><p>Intelligence is defined as “the ability to acquire and apply knowledge and skills”.6 Artificial intelligence is a field of computer science that focuses on the development of systems that can perform tasks that normally require human intelligence, such as perception, learning, problem-solving, and decision-making. Modern AI models are trained to perform a particular cognitive task very well. In doing so, AI has the potential to improve efficiency, reduce errors, and increase productivity.7</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*tF3c8REPbCprf01xw-l6Hg.png" /><figcaption>Figure 2. The machine-human interface of narrow AI methods. These methods mimic narrow forms of human intelligence with greater accuracy, speed, and scale, allowing us to use them for complex data.</figcaption></figure><p>AI methods, in fields like machine learning (ML), natural language processing (NLP), computer vision (CV), and audio processing (AP), have been developed to mimic components of human intelligence (Figure 2). The advantage of these AI methods over human intelligence is that they can perform their tasks with greater accuracy, speed, and scale. ML is a field of AI that focuses on creating models which learn from data. NLP is a field of artificial intelligence that focuses on the interaction between computers and humans through natural languages, such as English, Spanish, or Chinese. NLP tasks include text classification, information extraction, translation, summarization, and sentiment analysis. CV is a field of AI which aims to enable computers to understand visual information, including both images and video. CV tasks include image classification, object detection, and facial recognitition. AP is a field of AI focused on computers processing, interpreting, and manipulating audio signals. Some AP tasks include speech recognition, speech development, and music composition.</p><p>It is important to understand that these fields of AI can intersect to produce powerful models. In the way a child must learn to understand language, one might use machine learning to “train” a natural language processing model to behave as if it understands language.</p><p>The most widely employed narrow AI method in surgery is ML.7 Machine learning algorithms, which train computers on how to learn from data, can be described in several ways. Supervised learning describes when machines learn from annotated datasets to make predictions on new datasets. Of note, these annotated datasets are not necessarily manually labelled. Within surgery, supervised ML has been used to predict adverse events following surgery and measure risk of incisional hernia after abdominal surgery, among several applications.8,9 ML also includes unsupervised learning, whereby machines can identify groups and patterns within unannotated data. Other disciplines in ML include reinforcement learning and deep learning. Reinforcement learning is analogous to operant conditioning, where positive and negative rewards offer feedback for a model to continually learn a task.7 Deep learning describes systems which use multi-layered neural networks to understand complex data. The “deepness” refers to the fact that there are several layers in the neural network between the input and output. This allows neural networks to process even more complex data.</p><p>While deep learning is increasingly utilized in healthcare, it is limited because it can lack “explainability”, the ability for us to understand the decisions an AI model makes.10 The inner layers of a deep learning model can be a “black box” to the teams of developers and surgeons who design them — there may be input variables (i.e. “features”) that a deep learning model relies on, but which do not fit a clinical context, thus biasing the model. One way this could happen is if a deep learning model overfits its training data, capturing noise and random fluctuances in the training data instead of the underlying patterns, which prevent it from generalizing to outside testing data. When an unexplainable model analyzes testing data and creates a false output, it is difficult to troubleshoot the error in its architecture. As such, there is a healthy tension in healthcare informatics between using powerful deep learning or other ML methods which are more readily explainable. Importantly, none of these descriptions of ML are mutually exclusive. There are algorithms which can incorporate supervised learning with reinforcement learning and deep learning, as we will soon explore.</p><p>When viewed in the context of the data lake, it makes sense how ML algorithms can use surface-level, structured data. As we explore deeper levels of the data lake, we encounter unstructured data that is difficult for machines to process. One cannot simply insert an image, for example, into a spreadsheet or calculator. We must find ways to interpret these alternative sources of data.</p><p>One approach is by systematically defining surface-level, structured features from more complex data. For example, our group has attempted to extract hundreds of numeric features from radiology to predict incisional hernia after laparotomy.11,12 While this endeavor has shown potential for future clinical translation, it is very cost-intensive and computing-inefficient. This is where deep learning techniques have taken the spotlight, as the inner layers of a deep neural network can potentially process these complex data. The challenge is actually creating the deep neural network in the first place, which require more data and computing power than traditional ML.10</p><p>Traditionally, researchers created ML models for specific tasks from the ground up. This required large training datasets for each model. Over recent years, there has been a paradigm shift in how to build ML models. This shift suggests that one should start with general-purpose systems, termed “foundation models”, and further train them for specific tasks.13 As Bommasani et al. discuss, this evolution allows healthcare teams to “rapidly prototype and build highly dynamic and generative AI-infused applications”.13 In this way, foundation models have the potential to democratize powerful AI techniques for various settings, including surgery.</p><p><strong>FOUNDATION MODELS</strong></p><p>A foundation model is an ML model that serves as the starting point for further development or improvement. They are deep neural network architectures pre-trained on extremely large datasets. The foundation model can perform generalized tasks, such as language or image processing, with varying degrees of performance in its original (“pre-trained”) form. However, they can be further trained on more specific datasets to improve performance on downstream applications.13 For example, a foundation model for image classification might be a convolutional neural network (CNN) trained on a large dataset of labeled images. This model could then be used as a basis for more complex models that perform better for specific types of image classification. In this way, the foundation model can be thought of as a general-purpose technology. Unlike the past generation of ML, in which models needed to be built for each application, foundation models can be rapidly repurposed for many applications. Further training of a foundation model can be regarded as “transfer learning”, as one essentially transfers the network of a base model and builds layers on top of them to create downstream models.13 “Few-shot learning” is a special case of transfer learning, whereby the model is further trained on only a few cases to achieve high accuracy for a given task.14 This is compared to traditional ML models which require training on many cases to achieve accuracy on a specific task. Figure 3 demonstrates the differences between traditional supervised ML and few-shot learning using foundation models. “Fine-tuning” is a related concept whereby a foundation model’s existing layers and features are adjusted before further training for downstream models.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*tKlwrSKAIuffw4f4WN1N6Q.png" /><figcaption>Figure 3. Differences between traditional supervised machine learning, and “few-shot” learning using foundation models. The latter allows development of AI applications in smaller teams, with fewer resources, and with less training data. In this way, foundation models will democratize AI for healthcare applications.</figcaption></figure><p>AI adoption in healthcare and surgery has been limited thus far, as compared to other industries, because health data is sensitive and, therefore, restricted in availability. Foundation models align with the current healthcare milieu because they facilitate development of AI models using small datasets (few-shot learning). They allow smaller groups of healthcare innovators to employ powerful AI techniques.</p><p>The first foundation models were large language models (LLMs) for NLP tasks. While there were some several early pioneers, the first major foundation model was Bidirectional Encoder Representations from Transformers (BERT), developed by Google© in 2018.15 This model was pre-trained on the entire English Wikipedia (2.5 billion words at the time) and BookCorpus (800 million words) datasets. Its base model included 12 layers in its neural network and 110 parameters. At the time, BERT achieved state-of-the-art benchmarks on many NLP tasks. In fact, in 2019, Google search engine began using BERT in their English search queries.</p><p>While the pre-trained LLMs are somewhat effective in extracting certain information, an LLM may need to be fine-tuned on a relevant dataset to achieve better performance. Researchers have gradually been exploring how LLMs can be further refined for the healthcare setting. For example, BioBERT was a model developed from the BERT architecture.16 It was trained on a dataset of the broad biological literature, and has been effective at tasks such as extracting information from scientific papers and predicting the functions of genes.16</p><p>Because of the sensitive nature of protected health information, development of downstream models trained on clinical documentation has been gradual. Alsentzer et al., were able to train BERT on clinical documentation from the MIMIC-III v1.4 database, a free critical care database that includes clinical notes, to create ClinicalBERT.17 Several groups have used this publicly available tool to extract indicators of sleep apnea from scanned sleep study reports, detect critical findings on radiology reports, and identify sentences describing procedures from surgical reports.18–21 These investigations found ClinicalBERT to be a better model of extracting free text information than traditional NLP algorithms.</p><p>Yang et al., compared the performance of ClinicalBERT and other models based on LLMs with their own NLP model, GatorTron, which was created using traditional machine learning without transfer learning from foundation models.22 Unlike these other models, GatorTron was created de novo from billions of words from de-identified clinical notes, scientific literature, and Wikipedia. While it demonstrated marginally greater precision and recall ability for some language tasks, it is worth noting how time and resource-intensive it was to create GatorTron in the first place, as it required hundreds of dedicated computers and a supercomputing infrastructure provided by NVIDIA.22 Such resources would be a major limitation for most surgeons and research teams. This is compared to ClinicalBERT which was created as a foundation model and is available for public use and further training. It is plausible fine-tuning BERT, ClinicalBERT, or another foundation model on the training data for GatorTron might create an even superior model — something which we will soon discuss.</p><p>While BERT and similar models were key drivers for software engineers to democratize ML for industry, they are limited because they can only classify text and cannot generate information. They are good, for example, at identifying words or phrases, but are unable to holistically understand texts and summarize them. Our current moment in the AI revolution emerged from models which could actually generate new multimedia (e.g. text, image, video, audio). These models have sparked widespread interest and adoption of consumer-friendly AI applications.</p><p><em>Discriminative vs. Generative Models</em></p><p>Before we continue our discussion of foundation models, it is important to understand a key transition in the AI landscape over recent years, from “discriminative” to “generative” models.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*cvc8pbuEj0JzshlBEHBqlg.png" /><figcaption>Figure 4. Discriminative AI can create boundaries to classify new data points. Generative AI can create new data points which fit a certain class.</figcaption></figure><p>All AI models can be categorized as either “discriminative” or “generative” (Figure 4). Discriminative algorithms focus on learning the decision boundaries between different classes of data. These algorithms use existing data to create models which can then predict or classify new data. The simplest discriminative ML algorithm is a logistic regression. There are more advanced methods for non-linearly separable data, such as support vector machines, random forests, and neural networks. To date, most ML applications in surgery have been for discriminative tasks, such as putting labels or diagnoses on images and encounters. For example, the MySurgeryRisk calculator can classify surgical patient risk on a number of postoperative complications with high discriminatory ability.8</p><p>On the other hand, generative algorithms, also called “generative AI”, aim to learn the underlying probability distribution of data. In doing so, they are able to create new data instances. For example, while a discriminative model would apply information about the differences between cats and dogs, a generative model more holistically models what a cat is and what a dog is to then solve downstream tasks. In healthcare, a discriminative model might be able to process a radiograph and identify whether there is a mass or not, whereas a generative model is needed to create a comprehensive written report of the radiographic findings.</p><p>Any AI applications which produce new multimedia information will use generative AI, such as summarizing clinical encounters, answering patient questions, translating medical information between languages and literacy levels, and creating technical specifications for custom implants.23 Examples of generative AI algorithms include Generative Adversarial Networks (GAN), autoregressive, and stable diffusion models, all of which are types of deep neural networks. Recent healthcare startups have used generative AI to complete clinical documentation for clinical encounters in real time, create synthetic medical imaging, and intra-operatively view tissue perfusion in real time.24–26</p><p>Increasingly, generative AI models can also perform discriminative tasks. In the past, these discriminative abilities were weaker than those of discriminative algorithms. However, with the advent of large foundation models, generative models have become increasingly reliable for discriminative tasks, underscoring their importance in the next generation of healthcare-facing AI applications.</p><p>In summary, discriminative AI is about identifying patterns in data to make predictions or decisions, while generative AI is about learning the underlying structure of data which allows it to create new information. This makes them ideal for harnessing the complex healthcare data lake, and to power AI applications for surgeons and patients alike.</p><p><em>References</em></p><p>6. Oxford English Dictionary. intelligence, n. In: Oxford University Press; 2023. <a href="https://doi.org/10.1093/OED/2404969105">https://doi.org/10.1093/OED/2404969105</a></p><p>7. Hashimoto DA, Rosman G, Rus D, Meireles OR. Artificial Intelligence in Surgery: Promises and Perils. <em>Annals of Surgery</em>. 2018;268(1). doi:10.1097/SLA.0000000000002693</p><p>8. Bihorac A, Ozrazgat-Baslanti T, Ebadi A, et al. MySurgeryRisk: Development and Validation of a Machine-Learning Risk Algorithm for Major Complications and Death after Surgery. <em>Ann Surg</em>. 2019;269(4):652–662. doi:10.1097/SLA.0000000000002706</p><p>9. Basta MN, Kozak GM, Broach RB, et al. Can We Predict Incisional Hernia?: Development of a Surgery-specific Decision-Support Interface. <em>Annals of Surgery</em>. 2019;270(3). doi:10.1097/SLA.0000000000003472</p><p>10. Ras G, Xie N, van Gerven M, Doran D. Explainable Deep Learning: A Field Guide for the Uninitiated. Published online September 13, 2021. doi:10.48550/arXiv.2004.14545</p><p>11. McAuliffe PB, Desai AA, Talwar AA, et al. Preoperative Computed Tomography Morphological Features Indicative of Incisional Hernia Formation after Abdominal Surgery. <em>Annals of Surgery</em>.:10.1097/SLA.0000000000005583. doi:10.1097/SLA.0000000000005583</p><p>12. Talwar AA, Desai AA, McAuliffe PB, et al. Optimal Image-Based Biomarkers For Prediction Of Incisional Hernia Formation. <em>Hernia: The Journal of Hernias and Abdominal Wall Surgery</em>. Published online in press.</p><p>13. Bommasani R, Hudson DA, Adeli E, et al. On the Opportunities and Risks of Foundation Models. Published online July 12, 2022. doi:10.48550/arXiv.2108.07258</p><p>14. Brown TB, Mann B, Ryder N, et al. Language Models are Few-Shot Learners. Published online July 22, 2020. doi:10.48550/arXiv.2005.14165</p><p>15. Devlin J, Chang MW, Lee K, Toutanova K. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. Published online May 24, 2019. Accessed February 7, 2023. <a href="http://arxiv.org/abs/1810.04805">http://arxiv.org/abs/1810.04805</a></p><p>16. Lee J, Yoon W, Kim S, et al. BioBERT: a pre-trained biomedical language representation model for biomedical text mining. <em>Bioinformatics</em>. 2020;36(4):1234–1240. doi:10.1093/bioinformatics/btz682</p><p>17. Alsentzer E, Murphy J, Boag W, et al. Publicly Available Clinical BERT Embeddings. In: <em>Proceedings of the 2nd Clinical Natural Language Processing Workshop</em>. Association for Computational Linguistics; 2019:72–78. doi:10.18653/v1/W19–1909</p><p>18. Bombieri M, Rospocher M, Dall’Alba D, Fiorini P. Automatic detection of procedural knowledge in robotic-assisted surgical texts. <em>Int J Comput Assist Radiol Surg</em>. 2021;16(8):1287–1295. doi:10.1007/s11548–021–02370–9</p><p>19. Banerjee I, Davis MA, Vey BL, et al. Natural Language Processing Model for Identifying Critical Findings-A Multi-Institutional Study. <em>J Digit Imaging</em>. Published online November 7, 2022. doi:10.1007/s10278–022–00712-w</p><p>20. Hsu E, Malagaris I, Kuo YF, Sultana R, Roberts K. Deep learning-based NLP data pipeline for EHR-scanned document information extraction. <em>JAMIA Open</em>. 2022;5(2):ooac045. doi:10.1093/jamiaopen/ooac045</p><p>21. Kumar A, Goodrum H, Kim A, Stender C, Roberts K, Bernstam EV. Closing the loop: automatically identifying abnormal imaging results in scanned documents. <em>J Am Med Inform Assoc</em>. 2022;29(5):831–840. doi:10.1093/jamia/ocac007</p><p>22. Yang X, Chen A, PourNejatian N, et al. GatorTron: A Large Clinical Language Model to Unlock Patient Information from Unstructured Electronic Health Records. Published online December 16, 2022. doi:10.48550/arXiv.2203.03540</p><p>23. Chau RCW, Chong M, Thu KM, et al. Artificial intelligence-designed single molar dental prostheses: A protocol of prospective experimental study. <em>PLoS One</em>. 2022;17(6):e0268535. doi:10.1371/journal.pone.0268535</p><p>24. Ambience | Your documentation on autopilot. Accessed April 12, 2023. <a href="https://www.ambiencehealthcare.com/">https://www.ambiencehealthcare.com/</a></p><p>25. Activ Surgical | Intraoperative Surgical Intelligence | Med Tech. Activ Surgical. Accessed April 12, 2023. <a href="https://www.activsurgical.com/">https://www.activsurgical.com/</a></p><p>26. Segmed. Accessed April 12, 2023. <a href="https://www.segmed.ai/">https://www.segmed.ai/</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=0ffc65df4144" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The Healthcare Data Lake]]></title>
            <link>https://ankoortalwar.medium.com/the-healthcare-data-lake-e7938401a7b5?source=rss-5605e615784b------2</link>
            <guid isPermaLink="false">https://medium.com/p/e7938401a7b5</guid>
            <category><![CDATA[healthcare]]></category>
            <category><![CDATA[data]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[surgery]]></category>
            <category><![CDATA[ai]]></category>
            <dc:creator><![CDATA[Ankoor Talwar]]></dc:creator>
            <pubDate>Mon, 25 Mar 2024 07:06:20 GMT</pubDate>
            <atom:updated>2024-05-07T15:06:02.328Z</atom:updated>
            <content:encoded><![CDATA[<h4>AI in Surgery: Part I</h4><p>High value healthcare, including surgical care, is increasingly evidence-informed. This has allowed surgery to become more patient-centered and improve outcomes. The past two decades have seen a proliferation of big data guiding progress in numerous industries.[1] While the size and complexity of healthcare data at our disposal grows, our discipline is still at the precipice of understanding how to exploit its potential. The power of modern computing provides us with the capacity to harness this data and guide the next generation of surgical care.</p><p>The purpose of this commentary is to understand how evolving methods in artificial intelligence (AI) can harness the immense healthcare data lake to improve surgical care. Throughout the parts, I will provide an overview of foundation models which have proliferated to the forefront of AI development. Further, I highlight the frameshift from discriminative to generative AI algorithms which have shaken up the industry. <strong>Foundation models have democratized AI for even small teams of surgeons and scientists.</strong> They have <strong>powered a new era of generative AI</strong> applications which can enhance surgical productivity and outcomes.</p><p><em>The Healthcare Data Lake</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*mzpxCF1XfY45Fj6NDDv4zA.jpeg" /><figcaption>Figure 1. The Healthcare Data Lake. In addition to surface-level, code-based data, electronic health records now incorporate multitudes of alternative sources of data. Modern computing can harness the wealth of information in the healthcare data lake to enable human-designed, machine-powered decision making.</figcaption></figure><p>The availability and complexity of healthcare data has grown immensely. A single patient can generate up to 80 megabytes of electronic health record (EHR) data every year.[2] The breadth of EHR information can be conceptualized as a <em>“data lake”</em> (Figure 1), an amalgamation of data that is heterogenous and from multiple sources.[3] The surface level of the lake is comprised of traditional, coded metrics such as patient demographics (e.g. age, sex, BMI) as well as diagnostic and procedural codes (e.g. ICD and CPT codes). These data are highly “structured” or stored in ways that are easily interpretable by computers and humans, alike. The beginning of the 21st century was hallmarked by a thorough understanding of how these clinicodemographic factors relate to patient care, such as the ACS NSQIP surgical risk calculator.[4] As we explore the deeper regions of the data lake, we uncover the vast amount of “semi-structured” or “unstructured” patient data. This includes radiology and clinical imaging (including operative video), digital pathology, clinical documentation (e.g. office notes, operative notes, therapy evaluations, medication administration records), nutritional data, wearable sensor data (e.g. continuous glucose monitors, sleep monitors), patient reported outcome measures, environmental data (e.g. air quality index, UV index), population/epidemiological data (e.g. demographics, contact tracing), and genetics. These data cannot be fully utilized with conventional analytic tools. One cannot simply insert an image, for example, into a spreadsheet or calculator. We must find ways to interpret these alternative sources of data.</p><p>Each patient’s EHR is its own data lake, a large repository of all clinical information in its raw, native form. <strong>Modern computing methods, such as generative AI, can dip into the data lake and select information as necessary for computational tasks</strong>. In the next parts we will review the development of foundation models and generative AI in healthcare. Ultimately, harnessing the data lake has the potential to transform surgical care by enabling human-designed, machine-powered decision-making tools.[5]</p><p><em>References</em></p><p>1. Hilbert M, López P. The world’s technological capacity to store, communicate, and compute information. <em>Science</em>. 2011;332(6025):60–65. doi:10.1126/science.1200970</p><p>2. Huesch MD, Mosher TJ. Using It or Losing It? The Case for Data Scientists Inside Health Care. <em>NEJM Catalyst</em>. Published online May 4, 2017. Accessed February 6, 2023. <a href="https://catalyst.nejm.org/doi/full/10.1056/CAT.17.0493">https://catalyst.nejm.org/doi/full/10.1056/CAT.17.0493</a></p><p>3. Stein B, Morrison A. <em>The Enterprise Data Lake: Better Integration and Deeper Analytics</em>. PricewaterhouseCooper; 2014.</p><p>4. Bilimoria KY, Liu Y, Paruch JL, et al. Development and evaluation of the universal ACS NSQIP surgical risk calculator: a decision aid and informed consent tool for patients and surgeons. <em>J Am Coll Surg</em>. 2013;217(5):833–842.e1–3. doi:10.1016/j.jamcollsurg.2013.07.385</p><p>5. Moor M, Banerjee O, Abad ZSH, et al. Foundation models for generalist medical artificial intelligence. <em>Nature</em>. 2023;616(7956):259–265. doi:10.1038/s41586–023–05881–4</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e7938401a7b5" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>