𝐑𝐀𝐆 𝐟𝐨𝐫 𝐃𝐢𝐚𝐠𝐧𝐨𝐬𝐭𝐢𝐜 𝐑𝐞𝐚𝐬𝐨𝐧𝐢𝐧𝐠 𝐟𝐨𝐫 𝐂𝐥𝐢𝐧𝐢𝐜𝐚𝐥 𝐍𝐨𝐭𝐞𝐬 (𝐃𝐢𝐑𝐞𝐂𝐓)

Empowering clinicians with grounded, real-time insights.

🧠 Project Overview and Motivation

In the high-stakes environment of healthcare, quick access to reliable and up-to-date clinical knowledge is vital. While Large Language Models (LLMs) like GPT or LLaMA are powerful, their knowledge is static and confined to their training data. This limitation can result in outdated information or hallucinated content — especially dangerous in clinical settings.

Our project, DiReCT (Diagnostic Reasoning for Clinical Notes), aims to solve this using Retrieval-Augmented Generation (RAG). DiReCT brings together cutting-edge components like LangChain, FAISS, and Bio_ClinicalBERT to enable contextually aware, evidence-grounded diagnostic support for clinicians.

Get Waghib Ahmad’s stories in your inbox

Join Medium for free to get updates from this writer.

Remember me for faster sign in

Hosted on Hugging Face Spaces with a Streamlit frontend and powered by Groq’s ultra-fast LLaMA 3.3 70B model, DiReCT is a fully functional clinical assistant AI prototype for the future of decision support.+-

⚙️ Methodology

1. Data Collection and Preprocessing

We used both diagnosis flowcharts and patient case notes from the publicly available MIMIC-IV-Ext dataset. These documents were processed and loaded using Python scripts that:

• Parse JSON files for structured flowcharts
• Extract relevant fields from patient cases
• Format all information into a consistent schema

def load_clinical_data():
    # Load diagnosis flowcharts and patient case notes
    # Returns a list of Document objects

2. Embeddings and Vector Store

To semantically represent medical documents, we used Bio_ClinicalBERT via HuggingFace:

embeddings = HuggingFaceEmbeddings(
    model_name="emilyalsentzer/Bio_ClinicalBERT",
    model_kwargs={"device": "cuda"}
)

Documents were chunked and embedded into a FAISS vector store:

def build_vectorstore():
    # Chunk documents, embed, and return FAISS vectorstore

3. RAG Pipeline

The RAG system works by retrieving relevant clinical information and passing it along with the user query to a powerful LLM.

def run_rag_chat(query, vectorstore):
    # Uses LangChain to retrieve and generate response

🔄 How DiReCT Works

1. Query Input — Clinician enters a diagnostic question.
2. Retrieval — LangChain + FAISS search MIMIC-IV clinical notes using Bio_ClinicalBERT embeddings.
3. Prompt Building — Top results are merged with the query.
4. Generation — Prompt is sent to Groq-hosted LLaMA 3.3 70B.
5. Response — Result shown via a Streamlit app on Hugging Face Spaces.

4. Evaluation

To measure reliability and accuracy, we added basic evaluation metrics:

• Hit Rate: Measures overlap between expected and retrieved documents
• Faithfulness: Cosine similarity between context and generated answer

def evaluate_rag_response(response, embeddings):
    # Computes hit rate and faithfulness scores

🔍 Key Challenges

• Data Parsing: Clinical notes are often unstructured. Structuring patient data and flowcharts in a consistent way required significant cleaning.
• Prompt Engineering: Getting the LLaMA model to produce medically sound, structured responses required prompt tuning.
• Evaluation: Designing automatic evaluation for RAG systems — especially in clinical NLP — is non-trivial and required customized metrics.

📈 Findings and Lessons Learned

• Bio_ClinicalBERT significantly improves retrieval relevance compared to generic embeddings.
• FAISS + LangChain provided a scalable and modular foundation for RAG pipelines.
• Hosting on Hugging Face Spaces with Groq makes it feasible to deliver near real-time results with powerful LLMs.
• Clinical RAG systems benefit greatly from hybrid datasets (structured + unstructured) to provide contextual reasoning.

🚀 Try It Out

• Live Demo on Hugging Face Spaces: DiReCT Demo
• Full Source Code: GitHub Repository

We hope DiReCT inspires future applications in clinical NLP, decision support systems, and real-world AI for medicine.

#RAG #LLM #LangChain #Groq #BioClinicalBERT #MIMICIV #ClinicalAI #Streamlit #AIinMedicine #OpenSource #MedicalNLP #DiReCT