<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Alexandra Lorenzo on Medium]]></title>
        <description><![CDATA[Stories by Alexandra Lorenzo on Medium]]></description>
        <link>https://medium.com/@alexandra.lorenzodebrionne?source=rss-cca46a503e30------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*gKvznXE4JF6HuDwC</url>
            <title>Stories by Alexandra Lorenzo on Medium</title>
            <link>https://medium.com/@alexandra.lorenzodebrionne?source=rss-cca46a503e30------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Thu, 19 Nov 2026 09:49:39 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@alexandra.lorenzodebrionne/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Knowledge Graphs Meet LLMs: Comparative Analysis of GraphRAG, LLM, and RAG in
Biomedical…]]></title>
            <link>https://medium.com/capgemini-invent-lab/knowledge-graphs-meet-llms-comparative-analysis-of-graphrag-llm-and-rag-in-biomedical-fea243aa1555?source=rss-cca46a503e30------2</link>
            <guid isPermaLink="false">https://medium.com/p/fea243aa1555</guid>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[graphrag]]></category>
            <category><![CDATA[biomedical]]></category>
            <dc:creator><![CDATA[Alexandra Lorenzo]]></dc:creator>
            <pubDate>Thu, 12 Dec 2024 10:04:51 GMT</pubDate>
            <atom:updated>2024-12-12T10:04:51.734Z</atom:updated>
            <content:encoded><![CDATA[<h3>Knowledge Graphs Meet LLMs: Comparative Analysis of GraphRAG, LLM, and RAG in<br>Biomedical Applications (Part 2/2)</h3><p><em>L. Lacombe, A. Lorenzo de Brionne, A. Mejri, D. Muro, D. Hervault</em></p><p>The biomedical domain presents unique challenges for AI models due to its complexity, requiring precise understanding of intricate relationships between diseases, drugs, and genes. In this comparative study, we evaluate the performance of LLM, RAG, and GraphRAG on STaRK-Prime, a specialized biomedical dataset. By analyzing their ability to answer complex queries, we uncover the strengths and limitations of each approach.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*ORnJAmue6go3lpuD" /><figcaption>Photo by <a href="https://unsplash.com/@tayfundikmen?utm_source=medium&amp;utm_medium=referral">Tayfun Dikmen</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><h3>1. Practical Application in the Biomedical Sector</h3><p>To effectively compare Large Language Models (LLMs), Retrieval-Augmented Generation (RAG), and Graph RAG, we need :</p><ul><li>A model trained on the same dataset as those used for RAG and GraphRAG</li><li>PDF or text files containing all nodes and relations required for the RAG model.</li><li>A Knowledge Graph specifically designed for the GraphRAG model</li><li>An evaluation dataset</li></ul><p><em>It sounds challenging, doesn’t it?</em></p><h4>1.1. Stark Prime: A Semi-Structured Knowledge Graph</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*JLSCi0KdSDP6q3XXII8rww.png" /></figure><p>We based our analysis on a biomedical dataset known as <a href="https://stark.stanford.edu/dataset_prime.html">STaRK-Prime</a> <a href="https://arxiv.org/abs/2404.13207">[1]</a>. The primary goal was to analyze drug interactions and predict adverse effects by leveraging a vast corpus of biomedical literature and an existing knowledge graph. STaRK-Prime is designed to address complex biomedical questions involving diseases, drugs, genes/proteins, and other relevant entities.</p><p>The Semi-Structured Knowledge Graph was constructed on Orphanet, NCBI, UCSC, … with a total number of 129.375 nodes and 8 millions links.</p><h4>1.2. Stark Prime: Nodes description</h4><p>STaRK-Prime includes nodes with its descriptions, including 129.375 text documents, resulting in a dataset of 1.4Gb. As defined in the Figure 5 these datasets will be used to train the RAG model.</p><figure><img alt="An example of a Node ID with its description" src="https://cdn-images-1.medium.com/max/1024/1*3wD8BY-l-9eWlT261mk-qA.png" /></figure><h4>2.3. Stark Prime: Human-Generated Q&amp;A Datasets</h4><p>With 94 Questions generated and annotated by datasets we can compare multiple models. As example:</p><figure><img alt="Example of 3 human-generated questions on the STaRK-Prime Datasets" src="https://cdn-images-1.medium.com/max/1024/1*wawq7VZQKPonZM8C_w7ZNg.png" /></figure><h3>2. Models Implementation</h3><p>This section defines how each model was implemented, it provides an overview of the progression from Large Language Models (LLMs) to Retrieval-Augmented Generation (RAG) models and finally to GraphRAG, highlighting the evolution and advancements in these techniques.</p><h4>2.1. LLM</h4><p>Generates text based on vast amounts of pre-trained data. While powerful, it does not inherently include a retrieval mechanism, and its outputs may lack precision in specific, data-intensive contexts like biomedicine.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/454/1*XY3hvmlQRS26Q76YPQIQQg.png" /></figure><p>We further optimize the prompt to improve performance and subsequently export the results for evaluation. The foundation of our model is GPT-4o.</p><h4>2.2 RAG</h4><p>The Retrieval-Augmented Generation (RAG) approach enhances generative AI is based on node descriptions, providing the foundational content for retrieval. Each node description is transformed into a high-dimensional vector using OpenAI’s text-embedding-ada model.</p><p>This embedding process converts text into numerical representations. The Maximal Marginal Relevance (MMR) technique is employed to retrieve documents. MMR balances relevances (retrieving the most similar documents based on embeddings) with diversity (avoiding redundant similar results). After retrieving the top relevant document, they are passed to an LLM based on GPT-4o.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*I_khAbkCSE5raPlfpe52WA.png" /></figure><h4>2.3 Our GraphRAG</h4><p>In this architecture, (already defined in the Part 1), the process starts with keyword extraction using an LLM, followed by a vector search on the knowledge graph to identify relevant nodes and relationships. These retrieved relationships are then used to enrich the LLM’s context, enabling it to generate a more informed response.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-O7rzJ3x7Uul0NtniEnSEg.png" /></figure><h3>3. Results</h3><h4>3.1 Evaluation Methodology</h4><p>We evaluate the LLM, RAG, and GraphRAG models using human-annotated datasets. In this evaluation phase, the responses generated by the models were compared against a set of annotated queries to assess their accuracy. Each prediction was first checked for alignment with nodes in the Knowledge Graph. Following this, an exact match comparison was performed between the predictions and the Knowledge Graph nodes, and the results were then compared with the human-annotated responses to calculate metrics.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/340/1*3t7xqFg0Pt2xriy_tMjMMQ.png" /></figure><iframe src="https://cdn.embedly.com/widgets/media.html?url=https%3A%2F%2Fmath.embed.fun%2F7finAdjqtx3jVGuT5Ts1Di&amp;type=text%2Fhtml&amp;schema=embed&amp;display_name=Embed+Fun&amp;src=https%3A%2F%2Fmath.embed.fun%2Fembed%2F7finAdjqtx3jVGuT5Ts1Di" width="200" height="96" frameborder="0" scrolling="no"><a href="https://medium.com/media/2b3515a6bdee82d72bfb01f981d9d8ef/href">https://medium.com/media/2b3515a6bdee82d72bfb01f981d9d8ef/href</a></iframe><iframe src="https://cdn.embedly.com/widgets/media.html?url=https%3A%2F%2Fmath.embed.fun%2Fd8YJuV5ifDHdTug4xsGGZh&amp;type=text%2Fhtml&amp;schema=embed&amp;display_name=Embed+Fun&amp;src=https%3A%2F%2Fmath.embed.fun%2Fembed%2Fd8YJuV5ifDHdTug4xsGGZh" width="200" height="96" frameborder="0" scrolling="no"><a href="https://medium.com/media/97f7fb8ee5d7388b2eb09068453cd914/href">https://medium.com/media/97f7fb8ee5d7388b2eb09068453cd914/href</a></iframe><ul><li><strong>True Positives (TP)</strong>: The number of instances where the model’s predicted answer exactly matches the ground truth.</li><li><strong>False Positives (FP)</strong>: The number of instances where the model’s predicted answer does not match the ground truth</li><li><strong>False Negative (FN)</strong>: The number of instances where the model’s answer does not match the predictions.</li></ul><h4>3.2 Results Analysis</h4><p>The results of our comparative analysis are summarized as follows:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*IQvyrD0LBlQkWTBwBpoz-A.png" /></figure><p>As shown, GraphRAG outperforms RAG and even LLM in terms of recall, indicating it misses fewer elements. While the results are relatively low, this is often due to the difficulty of estimation, especially in the biomedical field, which is characterised by high human variability. Similar results have been found in the paper <a href="https://arxiv.org/abs/2404.13207">[1]</a>, even though the metrics may differ. Another finding is that the in-context answer retrieval, whether for document retrieval or subgraph retrieval, is significantly higher for GraphRAG (70%) compared to RAG (43%).</p><p>This means there is significant potential to further improve the results of GraphRAG.</p><h3>Conclusion and Next steps</h3><p>In conclusion, GraphRAG outperformed both LLM and RAG, showcasing the power of Knowledge Graphs to generate more contextually accurate and relevant answers, particularly in tasks that involve understanding complex biomedical relationships.</p><p>While LLM and RAG also provided useful predictions, they lacked the same level of contextual depth and precision, especially when dealing with intricate relational tasks. LLM, which primarily relies on large volumes of unstructured text, often struggled to capture the nuanced relationships<br>required for accurate predictions in these domains. RAG, although it incorporates some structured information, did not leverage the interconnected nature of the data as effectively as GraphRAG.</p><p>One drawback of GraphRAG, however, is the complexity involved in creating a Knowledge Graph. Recently, Microsoft offers an end-to-end process <a href="https://www.microsoft.com/en-us/research/blog/graphrag-unlocking-llm-discovery-on-narrative-private-data/">[2]</a> for generating a Graph and utilizing GraphRAG. They have garnered over 18,000 stars on GitHub, highlighting the growing importance of enhancing generative AI models with greater explainability. Overall, while we’ve explored a range of GraphRAG patterns, this is just the beginning of an exciting journey. The current patterns represent the early stages of GraphRAG’s evolution, and as we move forward, new patterns will emerge, offering fresh insights and possibilities.<br>Finding the ideal GraphRAG pattern for a given application is not straightforward — it requires experimenting with different retrieval patterns and thoroughly evaluating the results.</p><h3>References</h3><p>[1] Shirley Wu, Shiyu Zhao, Michihiro Yasunaga, Kexin Huang, Kaidi Cao, Qian Huang, Vassilis N.Ioannidis, Karthik Subbian, James Zou, and Jure Leskovec. Stark: Benchmarking llm retrieval on textual and relational knowledge bases, 2024. <a href="https://arxiv.org/abs/2404.13207">https://arxiv.org/abs/2404.13207</a>.</p><p>[2] Microsoft Research. Graphrag: Unlocking llm discovery on narrative private data, 2024. <a href="https://www.microsoft.com/en-us/research/blog/graphrag-unlocking-llm-discovery-on-narrative-private-data/">https://www.microsoft.com/en-us/research/blog/graphrag-unlocking-llm-discovery-on-narrative-private-data/</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=fea243aa1555" width="1" height="1" alt=""><hr><p><a href="https://medium.com/capgemini-invent-lab/knowledge-graphs-meet-llms-comparative-analysis-of-graphrag-llm-and-rag-in-biomedical-fea243aa1555">Knowledge Graphs Meet LLMs: Comparative Analysis of GraphRAG, LLM, and RAG in
Biomedical…</a> was originally published in <a href="https://medium.com/capgemini-invent-lab">Capgemini Invent Lab</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Knowledge Graphs Meet LLMs: Introducing the Power of GraphRAG (Part 1/2)]]></title>
            <link>https://medium.com/capgemini-invent-lab/knowledge-graphs-meet-llms-introducing-the-power-of-graphrag-part-1-2-c039561e60f4?source=rss-cca46a503e30------2</link>
            <guid isPermaLink="false">https://medium.com/p/c039561e60f4</guid>
            <category><![CDATA[large-language-models]]></category>
            <category><![CDATA[graphrag]]></category>
            <category><![CDATA[knowledge-graph]]></category>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[retrieval-augmented-gen]]></category>
            <dc:creator><![CDATA[Alexandra Lorenzo]]></dc:creator>
            <pubDate>Wed, 04 Dec 2024 10:14:55 GMT</pubDate>
            <atom:updated>2024-12-04T10:35:41.643Z</atom:updated>
            <content:encoded><![CDATA[<h4>A. Lorenzo de Brionne, L. Lacombe, A. Mejri, D. Muro, D. Hervault</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*FO7cFZDsZkxT4OVy" /><figcaption>Photo by <a href="https://unsplash.com/@firmbee?utm_source=medium&amp;utm_medium=referral">Firmbee.com</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><h3>1. The Emergence of GraphRAG</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/227/1*k4teSAZa2W9_QIYe5fieFQ.png" /></figure><p>Large Language Models (LLMs) excel at generating human-like text but can produce hallucinations. Retrieval-Augmented Generation (RAG) mitigates this, by retrieving relevant information from external sources to support the LLM with more contextual information. While RAG significantly reduces hallucinations, it introduces challenges such as retrieving irrelevant, redundant, or misleading information and struggles with complex queries requiring a synthesis of information across an entire corpus.</p><p>For example, consider the biomedical question: <em>“What are the connections between protein X and its role in disease Y, considering known drug interactions and pathways?”</em> Answering this, requires integrating insights from molecular biology, disease pathology, and pharmacology. Traditional RAG may retrieve disjointed or redundant pieces of information from various documents without effectively integrating them, leading to incomplete or potentially misleading answers.</p><p>To address these limitations, we propose GraphRAG, a novel framework with growing interest, that integrates structured databases, such as knowledge graphs (KGs), into retrieval-augmented generation <a href="http://dx.doi.org/10.1109/TKDE.2024.3352100">[4]</a> <a href="https://www.microsoft.com/en-us/research/blog/graphrag-unlocking-llm-discovery-on-narrative-private-data/">[5]</a>. By leveraging the interconnected nature of knowledge graphs, GraphRAG enhances contextual relevance, reduces noise, and delivers more accurate responses for complex, domain specific queries. In this first part, we present the GraphRAG and the main difference between GraphRAG, traditional RAG, and standalone LLMs.</p><h4>1.1. What are Knowledge Graphs?</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/340/1*neKHg09pnCpb98uYe-X-9Q.png" /></figure><p>Knowledge Graph represents a network of entities — such as objects, events, or concepts — and shows how these entities are related to one another. These graphs are stored in graph databases and visualized as a structure of interconnected nodes (entities) and edges (relationships). Notable examples include <strong>DBPedia</strong> and <strong>Wikidata</strong>, which compile and link extensive information into an accessible graph format.</p><p>Ontology is also a term you might come across. Ontologies form the backbone of the formal semantics in a Knowledge Graph, acting as the data schema that defines the concepts, categories, and relationships within the graph. In short, ontologies define the structure and rules, while Knowledge Graphs apply those rules to real data to enable practical usage.</p><h4>1.2. What is a GraphRAG?</h4><p>GraphRAG (Graph-based Retrieval-Augmented Generation) is an advanced version of the RAG framework that integrates Knowledge Graphs (KGs) to enhance information retrieval and generation.</p><p>Overall, GraphRAG offers several key advantages:</p><ul><li><strong>Enhanced Explainability</strong>: GraphRAG’s reliance on Knowledge Graphs allows it to provide more transparent results by tracing how information was retrieved and how relationships were used, offering clearer, more understandable reasoning.</li><li><strong>Improved Understanding of Relationships and Context</strong>: By leveraging the structured relationships in Knowledge Graphs, GraphRAG better understands the context and connections between entities, leading to more accurate and nuanced responses, particularly useful in complex domains like biomedicine.</li><li><strong>Advanced Search Scalability</strong>: GraphRAG uses Knowledge Graphs to efficiently navigate large and complex datasets, enabling faster and more precise searches by focusing on relevant connections between data points.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/454/1*twAn9UPOVVhBwHEhrqbVzg.png" /></figure><h3>2. GraphRAG Implementation</h3><h4>2.1. How to include Knowledge Graph according to your needs</h4><p>Knowledge Graphs (KGs) can enhance Large Language Models (LLMs) at multiple stages: training, inference, and post-inference<a href="https://arxiv.org/abs/2308.09729">[7]</a>.</p><figure><img alt="LLM augmentation with Knowledge Graph" src="https://cdn-images-1.medium.com/max/1024/1*ThS30ImYCULIuag-WxwOWQ.png" /></figure><ul><li><strong>Training</strong>: Models like <strong>KEPLER</strong> <a href="https://arxiv.org/abs/1911.06136">[6]</a> integrate KGs into the training objective by modifying the loss function through joint optimization. This approach combines language modeling and knowledge embedding losses, allowing the model to learn representations informed by both textual data and structured knowledge.</li><li><strong>Inference</strong>: Dynamic knowledge fusion techniques, such as those used in EMAT <a href="https://arxiv.org/abs/2210.16773">[9]</a> encode external knowledge into a key-value memory system and employ fast search methods to quickly retrieve relevant information during inference. Another straightforward methodology is KG prompting, which converts structured KGs into text sequences fed as context into LLMs, enhancing their reasoning capabilities. For example, MindMap <a href="https://arxiv.org/abs/2308.09729">[7]</a> transforms graph structures into mind maps, enabling LLMs to reason by consolidating facts from KGs and their implicit knowledge — similar to our methodology.</li><li><strong>Post-Inference</strong>: KGs can be used to analyze and interpret the results produced by LLMs. For instance, QA-GNN <a href="https://arxiv.org/abs/2104.06378">[10]</a> integrates knowledge graphs to validate and explain the answers generated by the model. QA-GNN constructs a graph neural network over KG concepts related to the question, enhancing interpretability and reasoning at each step.</li></ul><p>To explore further how Knowledge Graphs enhance LLMs, you may refer to the following papers from Pan et al. <a href="http://dx.doi.org/10.1109/TKDE.2024.3352100">[4]</a>. Our methodology is based on KG prompting and utilizes a streamlined version of the MindMap <a href="https://arxiv.org/abs/2308.09729">[7]</a> approach to identify entities and establish connections between them. We retrieve paths and links among entities and present this information in a simplified format for easier comprehension.</p><p>While we do not apply pruning, we do perform reordering. Unlike MindMap, which generates extensive connections through paths and links using LLMs, we avoid incorporating this strategy to keep the LLM out of the context creation process.</p><h4>2.2. Our GraphRAG</h4><p>Let’s deep dive in each step of the GraphRAG implementation. In this architecture, the process starts with keyword extraction using an LLM, followed by a vector search on the knowledge graph to identify relevant nodes and relationships. These retrieved relationships are then used to enrich the LLM’s context, enabling it to generate a more informed response.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-O7rzJ3x7Uul0NtniEnSEg.png" /></figure><h4>2.2.1. Keyword Extraction</h4><p>The process begins with keyword extraction, where a large language model (LLM) analyzes the user query (e.g., <em>Which drug targets PDE4B ?</em>) to identify key entities or keywords essential for linking to relevant nodes within the Knowledge Graph. To enhance entity extraction, the query is also embedded as a vector, enabling a vector search to identify nodes in the Knowledge Graph that are most similar to the query. To ensure comprehensive coverage of relevant entities, a Top-N retrieval approach is employed, capturing a set of the most relevant entities from the Knowledge Graph to minimize the risk of missing critical information.</p><h4>2.2.2 Paths Extraction</h4><p>Once the keywords and entities are identified, Paths Extraction finds the connections between these entities in the Knowledge Graph. The system is using Neo4J to search for entities either:</p><ul><li>The Shortest Paths between matched entities (for cases with sufficient keywords and entities).</li><li>The Hop-1, nearest neighbours, of those entities when fewer keywords or entities are identified.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*WJXX0DNAwLYfdNN4ObcQgw.png" /></figure><p>These paths capture the relationships and context that are important to answering the user’s query, connecting entities with relevant information across the knowledge graph. These strategies allow for capturing both direct and contextual relationships between entities, enhancing the knowledge graph’s capacity to answer the user’s query effectively.</p><h4>2.2.3 Reordering Paths</h4><p>Path re-ordering serves a critical function by restructuring paths in cases where context length must be limited due to LLM constraints. By prioritizing the most relevant paths first, this step partially addresses the limited context length as well as the “lost-in-the-middle”<a href="https://arxiv.org/abs/2307.03172">[3]</a> issue, ensuring that key information is positioned at the beginning and thus maximizing the LLM’s response quality. Paths are ordered strictly in descending order of relevance scores (a combination of similarity to the query and inverse TF-IDF and based on relations preferences), but this ordering alone does not fully address the “lost-in-the-middle” problem as outlined in <a href="https://arxiv.org/abs/2404.16811">[1]</a>.</p><h4>2.2.4 Graph-To-Text</h4><p>In this step, structured relationships and entities from the graph are converted into text sequences. The Graph-To-Text component transforms the graph’s structured data into human-readable text that can be processed by the LLM. All relationships are formatted as triplets and regrouped when necessary by node names. This conversion allows the LLM to incorporate the explicit knowledge contained in the graph as part of its reasoning process.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ZjaT8n-uzTayNN2zZzL2qw.png" /></figure><h4>2.2.5 Prompt Engineering to LLM Generation</h4><p>Finally, the enriched context, together with the original query, is fed into the LLM for answer generation. Prompt Engineering is critical here, as the structure of the prompt can significantly impact the coherence and accuracy of the response <a href="https://arxiv.org/abs/2310.14735">[2]</a>.</p><pre>prompt_template = &quot;&quot;&quot;<br>You are a virtual assistant designed to assist biomedical professionals by providing precise and relevant answers to their inquiries.<br>Base your answers solely on the relationships and entities provided in the context enclosed by ####. Do not use outside knowledge or assumptions.<br><br>### Guidelines:<br>1. Identify Key Entities and Relationships:<br>- Find entities mentioned in the question within the context.<br>2. Trace Relevant Paths:<br>- Follow logical paths connecting these entities through the given relationships.<br>3. Extract Relevant Information:<br>- Extract entities and relationships directly relevant to the question.<br>.....<br>### Example 1:<br>**Question 1:**<br>&quot;I am studying types of anemia. What are some specific types of anemia?&quot;<br>**Context:**<br>anemia -&gt; parent-child -&gt; (iron-deficiency anemia, hemolytic anemia, sickle-cell anemia, aplastic anemia)<br>breast cancer -&gt; associated_with -&gt; (DKK1, SLC6A3, ESR2)<br>**Answer:**<br>[&#39;iron-deficiency anemia&#39;, &#39;hemolytic anemia&#39;, &#39;sickle-cell anemia&#39;, &#39;aplastic anemia&#39;]<br>**Question:**<br>{question}<br>**Context:**<br>####<br>{context}<br>####<br>&quot;&quot;&quot;</pre><p>By carefully designing a prompt that includes structured knowledge from the Knowledge Graph and directing the LLM to focus on specific relationships, we enable the LLM to generate a more informed and precise answer.</p><h3>Conclusion</h3><p>In this exploration of GraphRAG, we’ve highlighted how integrating Knowledge Graphs with LLMs can address the challenges of complex query resolution, offering enhanced precision, context-awareness, and explainability. By leveraging structured relationships, GraphRAG transforms the way information is retrieved and synthesized. Stay tuned for Part 2, where we will dive into a comparative analysis of LLM, RAG, and GraphRAG, focusing on their applications in the biomedical domain.</p><h3>References</h3><p>[1] Shengnan An, Zexiong Ma, Zeqi Lin, Nanning Zheng, and Jian-Guang Lou. Make your llm fully utilize the context, 2024. <a href="https://arxiv.org/abs/2404.16811">https://arxiv.org/abs/2404.16811</a>.</p><p>[2] Banghao Chen, Zhaofeng Zhang, Nicolas Langrené, and Shengxin Zhu. Unleashing the potential of prompt engineering in large language models: a comprehensive review, 2024. <a href="https://arxiv.org/abs/2310.14735">https://arxiv.org/abs/2310.14735</a>.</p><p>[3] Nelson F. Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and Percy Liang. Lost in the middle: How language models use long contexts, 2023. <a href="https://arxiv.org/abs/2307.03172">https://arxiv.org/abs/2307.03172</a>.</p><p>[4] Shirui Pan, Linhao Luo, Yufei Wang, Chen Chen, Jiapu Wang, and Xindong Wu. Unifying large language models and knowledge graphs: A roadmap. IEEE Transactions on Knowledge and Data Engineering, 36(7):3580–3599, July 2024. ISSN 2326–3865. doi: 10.1109/tkde.2024.<br>3352100. <a href="http://dx.doi.org/10.1109/TKDE.2024.3352100">http://dx.doi.org/10.1109/TKDE.2024.3352100</a>.</p><p>[5] Microsoft Research. Graphrag: Unlocking llm discovery on narrative private data, 2024. <a href="https://www.microsoft.com/en-us/research/blog/graphrag-unlocking-llm-discovery-on-narrative-private-data/">https://www.microsoft.com/en-us/research/blog/graphrag-unlocking-llm-discovery-on-narrative-private-data/</a>.</p><p>[6] Xiaozhi Wang, Tianyu Gao, Zhaocheng Zhu, Zhengyan Zhang, Zhiyuan Liu, Juanzi Li, and Jian Tang. Kepler: A unified model for knowledge embedding and pre-trained language representation, 2020. <a href="https://arxiv.org/abs/1911.06136">https://arxiv.org/abs/1911.06136</a>.</p><p>[7] Yilin Wen, Zifeng Wang, and Jimeng Sun. Mindmap: Knowledge graph prompting sparks graph of thoughts in large language models, 2024. <a href="https://arxiv.org/abs/2308.09729">https://arxiv.org/abs/2308.09729</a>.</p><p>[8] Shirley Wu, Shiyu Zhao, Michihiro Yasunaga, Kexin Huang, Kaidi Cao, Qian Huang, Vassilis N.Ioannidis, Karthik Subbian, James Zou, and Jure Leskovec. Stark: Benchmarking llm retrieval on textual and relational knowledge bases, 2024. <a href="https://arxiv.org/abs/2404.13207">https://arxiv.org/abs/2404.13207</a>.</p><p>[9] Yuxiang Wu, Yu Zhao, Baotian Hu, Pasquale Minervini, Pontus Stenetorp, and Sebastian Riedel. An efficient memory-augmented transformer for knowledge-intensive nlp tasks, 2022. <a href="https://arxiv.org/abs/2210.16773">https://arxiv.org/abs/2210.16773</a>.</p><p>[10] Michihiro Yasunaga, Hongyu Ren, Antoine Bosselut, Percy Liang, and Jure Leskovec. Qa-gnn: Reasoning with language models and knowledge graphs for question answering, 2022. <a href="https://arxiv.org/abs/2104.06378">https://arxiv.org/abs/2104.06378</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c039561e60f4" width="1" height="1" alt=""><hr><p><a href="https://medium.com/capgemini-invent-lab/knowledge-graphs-meet-llms-introducing-the-power-of-graphrag-part-1-2-c039561e60f4">Knowledge Graphs Meet LLMs: Introducing the Power of GraphRAG (Part 1/2)</a> was originally published in <a href="https://medium.com/capgemini-invent-lab">Capgemini Invent Lab</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Face Mask Detector using Deep Learning (YOLOv3)]]></title>
            <link>https://medium.com/face-mask-detector-using-deep-learning-yolov3/face-mask-detector-using-deep-learning-yolov3-209b57f77e92?source=rss-cca46a503e30------2</link>
            <guid isPermaLink="false">https://medium.com/p/209b57f77e92</guid>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[object-detection]]></category>
            <category><![CDATA[yolov3]]></category>
            <dc:creator><![CDATA[Alexandra Lorenzo]]></dc:creator>
            <pubDate>Sat, 16 May 2020 09:11:03 GMT</pubDate>
            <atom:updated>2020-05-16T09:38:16.376Z</atom:updated>
            <content:encoded><![CDATA[<h4>Authors: <a href="https://medium.com/u/a1a19d7671e2">Louise Rodriguez</a> &amp; <a href="https://medium.com/u/2311e6882c8e">Alexandra Lorenzo</a></h4><p>Numerous papers [1] and codes have been made to detect people wearing masks. Our goal is to create a <strong>transparent</strong> model from Open Source Data and Open Source packages.</p><p>For this project we decided to use Pytorch for its simplicity, if you want to use Tensorflow API, I suggest you downgrade to the 1.14 version. Please find <a href="https://github.com/TannerGilbert/Tutorials/blob/master/Tensorflow%20Object%20Detection/object_detection_with_own_model.ipynb">here</a> a tutorial in Tensorflow to implement your own object detection model.</p><h3>Choose your object detection model</h3><p>The first thing to do is to choose your object detection algorithm between Faster RCNN, SSD, FPN, YOLO and more.</p><p>For this project, we want something fast since we will implement the model on a small machine such as Raspberry Pi, NVIDIA or Jetson Nano. So, the most appropriate algorithms are single shot methods such as YOLO or SSD.</p><p>The fundamental difference between RCNN models and single shot methods is speed ! YOLO or SSD eliminate the bounding box proposals steps by using a single deep neural network predicting both the boundary boxes and the classes at the same time.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/425/1*HouoFjWSIFo3gIcOnvUzKg.png" /><figcaption><strong>Figure 1:</strong> image extracted from the paper “Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks”</figcaption></figure><p>We decided to use YOLO v3, but you can try other single shot models.</p><h4>YOLO (You Only Look Once)</h4><p>Yolov3 is an object detection network part of yolo family (Yolov1, Yolov2). It can detect various things of different sizes, runs quite fast and make real-time inference possible on various devices. It takes images as input, pass it through neural network and get a vector of bounding boxes and class predictions as output. Yolo V3 uses DarkNet-53 to make features detection followed by convolutional layers. Darknet-53 is a 53 layers Convolutional Neural network trained on ImageNet mainly compose of 3 × 3 and 1× 1 filters with skip connections like the residual network in ResNet.</p><p>Yolov3 predicts bounding boxes using dimension clusters as anchor boxes. Anchor boxes predefined different shapes and are calculated on coco dataset using k-means clustering. Yolov3 uses in total 9 anchor boxes (3 anchors boxes at 3 different scales). For example, if I want to detect humans, we usually search for vertical rectangular boxes. While if we look for cars, it would be probably a horizontal rectangular boxe.</p><blockquote>Sum-up parenthesis:</blockquote><blockquote><strong>Anchor Boxes:</strong> predefined landmark rectangles for bounding boxes to pick and use offsets to give location for a detected object</blockquote><blockquote><strong>Bounding Boxes:</strong> predicted rectangle for a detected object relative to an anchor box</blockquote><p>The input image is divided into a S*S grid of cells. Each cell predicts B bounding boxes and C class probabilities thanks to anchor boxes that are fed to the network before training and prediction.</p><p><strong>Bounding boxes: </strong>Each bounding boxes predictions has 5 components</p><ul><li>4 coordinates ( x, y, that represent the center of the box relative to the grid cell location and w, h that represent the width and the height of the box relative to the cell location). These coordinates have to be normalized based on image size.</li><li>and 1 confidence score of whether there is an object or not. It is defined as the probability that there is an object multiplied by the IoU, in other words it is the “<em>the intersection over union (IOU) between the predicted box and the ground truth.” </em>that tells us if we have a good or bad outcome.</li></ul><p>Thus, S*S cells predict B predictions that results in S*S*B*5 outputs related to bounding box predictions.</p><p><strong>Class Probability: </strong>Each cell is responsible for predicting class probability that an object belongs to a specific class. That makes S*S*C total probabilities.</p><p>Finally we get a S*S*(B*5+C) tensor as output.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/340/1*zgfN-r-XPC7q9nvT-5R9QQ.png" /><figcaption><strong>Figure 2</strong>: <a href="https://blog.paperspace.com/how-to-implement-a-yolo-object-detector-in-pytorch/">https://blog.paperspace.com/how-to-implement-a-yolo-object-detector-in-pytorch/</a></figcaption></figure><p>After that, a selection could be made. In fact we can get rid of low confidence value predictions, by applying a simple thresholding of IOU &gt; 0.5 (by convention the bounding box is correct if the IoU is greater than 0.5).</p><p>Also we apply a non-maximum suppression that will take the bounding box with the highest probability and take all other closest bounding boxes with the highest IoU and suppress it.</p><p>If you have multiple boxes for one object check your non maximum function !</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*gKJLZuZO4RnqpF9DLXg-Ug.png" /><figcaption><strong>Figure 3</strong>: YOLO Architecture</figcaption></figure><h4>Our face mask detection dataset</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/640/1*BKtHLrN01u4mQQGDixJAJA.gif" /><figcaption><strong>Figure 4: </strong>CVAT tool to annotate persons “without mask” and “with mask”. We scrapped Google Images to create this dataset to build a face mask detector using the Deep Learning Framework Pytorch. Total <strong>507</strong> images.</figcaption></figure><p>In all Machine Learning project you need a dataset and sometimes it could be quite difficult to find the correct one.</p><p>For that, we created our own dataset by gathering images from the google image and also from the <a href="https://www.kaggle.com/ciplab/real-and-fake-face-detectio">Kaggle Dataset</a> “Real and fake face detection”. Few people (such as <a href="https://www.pyimagesearch.com/2020/05/04/covid-19-face-mask-detector-with-opencv-keras-tensorflow-and-deep-learning/"><strong>Adrian Rosebrock</strong></a>)<strong> </strong>also generate their own dataset by adding automatically masks on people with openCV algorithms which was quite interesting and could be faster than what we did.</p><p>as we said, we scrapped the Google images using the <a href="https://pypi.org/project/selenium/">selenium</a> package and ChromeDriver extension. We search various images including people alone or not and from different origins so as to reduce our biais.</p><p>Then we decided to label about 1000 images with CVAT tool (you can also use other tools such as LabelImg or else). Why CVAT? You can export easily to Yolo format. Furthermore you can quickly get to grips with the tool.</p><p>The final dataset is available on <a href="https://www.kaggle.com/alexandralorenzo/maskdetection">Kaggle</a>.</p><h3>Train our model</h3><p>We implemented YOLO v3 from <a href="https://github.com/ultralytics/yolov3"><em>ultralytics</em></a><em>, </em>you will need to rearrange the dataset for this package.</p><p>You will end up with the following folders and files:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/283/1*rrwuFxbnCzhTrXeA57vWdA.png" /><figcaption><strong>Figure 5</strong>: data architecture</figcaption></figure><ul><li>Images file : containing two/three folders one for the train set (you can find the .jpg files for the training step) the other for the validation set (you can find the .jpg files for the validation step)</li><li>Labels file : containing two/three folders one for the train set (you can find the .txt files for the training step containing the bounding boxes coordinates) the other for the validation set (you can find the .txt files for the test step). For recall, each lines containing in each txt files represent one bounding box coordinate (in this example we’ve got 4 bounding boxes). These numbers represents in the following orders, the class (0 or 1), x_center, y_center, the width and the height. Warning: this data have to be normalised, that is to say they have to be between 0 and 1.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/485/1*w7b4lb-DAudI7F2dhfYqvw.png" /><figcaption><strong>Figure 6</strong>: Labels examples</figcaption></figure><ul><li>mask.name file : in this file you can find your chosen classes (Mask, Without_mask)</li><li>train.txt : contains the paths of your images in Images file</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/164/1*XxB3WtsuaESU68fUzRaZ3Q.png" /><figcaption><strong>Figure 7: </strong>train.txt file</figcaption></figure><ul><li>mask.data : contains the number of classes you choose (here 2), the path of your train.txt file, your object.name path and a parameter backup set to backup/</li></ul><blockquote>classes = 2 <br>train = ../mask/train.txt<br>valid = ../mask/test.txt<br>names = data/mask.names<br>backup=backup/</blockquote><p>Also, you need to configure your Darknet yolo.cfg that contains all your Darknet CNN parameters like the number of batch, the number of convolutional kernels there are in a layer, your learning rate etc. You have to change in every [yolo] sections the number of classe (here we have to classes so we set it to 2). Also in every [convolutional] section right above each [yolo] section set the filter parameter equal to (classes + 5)*3. in our case we have (2 + 5) * 3 = 21 classes.</p><p>To train your model, you will need GPUs, I recommend using the Google Cloud Platform which <a href="https://cloud.google.com/free">new GCP users have 300$ credits</a> which is large enough for our study. If not, you can use <a href="https://www.paperspace.com/pricing">paperspace</a>.</p><h3>Results</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/510/1*SjyJmUYk_oyU-VDDcbY-vg.png" /><figcaption><strong>Figure 8</strong>: Example of output</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*fSioHmImLpiHiEMJNbxmmA.png" /><figcaption><strong>Figure 9</strong>: Model Performance analysis</figcaption></figure><h3>Conclusion &amp; Next Steps</h3><p>This project was only creating in the purpose of learning, you can find the <a href="https://github.com/alexlorenzo/masked-faces-detection">Github</a> and <a href="https://www.kaggle.com/alexandralorenzo/yolov3-startkit">Kaggle</a> implementation.</p><p>It could be interesting to:</p><ol><li>Try an other object detector such as <strong>SSD (Single Shot Detector) </strong>by <a href="https://d2l.ai/chapter_references/zreferences.html#liu-anguelov-erhan-ea-2016">[Liu et al., 2016]</a> or <strong>FPN</strong> (<strong>Feature Pyramid Network</strong>) by Facebook AI Research.</li><li>Play with hyperparameters</li></ol><p>The model will be included in an embedded devices, in the next article I will explain how to add the YOLO v3 model in a RaspberryPi 4. Stay tuned !</p><p>We would like to thank all my Quantmetry colleagues, without whom it wouldn’t have been possible.</p><p>Hope you enjoy this article ! Don’t hesitate to ask questions, comments or even disagreed.</p><p>Authors: Louise Rodriguez &amp; Alexandra Lorenzo</p><h3>Ressources</h3><p>[1] Z.Wang, G.Wang, B.Huang, Z.Xiong, Q.Hong, H.Wu, P. Yi, K. Jiang, N.Wang, Y.Pei, H.Chen, Y.Miao, Z.Huang, and J.Liang. 2020.<a href="https://arxiv.org/pdf/2003.09093.pdf"> Masked Face Recognition Dataset and Application</a></p><p>[2] J.Huang, V.Rathod, C.Sun, M.Zhu, A.Korattikara, A.Fathi, I.Fischer Z.Wojna, Y.Song, S.Guadarrama, K.Murphy. 2017. <a href="https://arxiv.org/pdf/1611.10012.pdf">Speed/accuracy trade-offs for modern convolutional object detectors</a></p><p>[3] J.Redmon, S.Divvala, R.Girshick, A.Farhadi. <a href="https://pjreddie.com/darknet/yolo/">You Only Look Once: Unified, Real-Time Object Detection</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=209b57f77e92" width="1" height="1" alt=""><hr><p><a href="https://medium.com/face-mask-detector-using-deep-learning-yolov3/face-mask-detector-using-deep-learning-yolov3-209b57f77e92">Face Mask Detector using Deep Learning (YOLOv3)</a> was originally published in <a href="https://medium.com/face-mask-detector-using-deep-learning-yolov3">Face Mask Detector using Deep Learning (YOLOv3)</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>