<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Shayak Sen on Medium]]></title>
        <description><![CDATA[Stories by Shayak Sen on Medium]]></description>
        <link>https://medium.com/@shayak.sen?source=rss-11f696611617------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*WqOzpczCJbQsTHKk</url>
            <title>Stories by Shayak Sen on Medium</title>
            <link>https://medium.com/@shayak.sen?source=rss-11f696611617------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sun, 22 Nov 2026 11:53:49 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@shayak.sen/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Build and Evaluate LLM Apps with LlamaIndex and TruLens]]></title>
            <link>https://medium.com/trulens/build-and-evaluate-llm-apps-with-llamaindex-and-trulens-fd6bb4d86aca?source=rss-11f696611617------2</link>
            <guid isPermaLink="false">https://medium.com/p/fd6bb4d86aca</guid>
            <category><![CDATA[llamaindex]]></category>
            <category><![CDATA[large-language-models]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[nlp]]></category>
            <category><![CDATA[llm]]></category>
            <dc:creator><![CDATA[Shayak Sen]]></dc:creator>
            <pubDate>Fri, 23 Jun 2023 21:55:00 GMT</pubDate>
            <atom:updated>2023-06-23T21:55:00.932Z</atom:updated>
            <content:encoded><![CDATA[<p>Build and Evaluate LLM Apps with LlamaIndex and TruLens</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*sH6VWjRhBGgk2PwB" /></figure><p><strong>Authors:</strong> Anupam Datta, Shayak Sen, Jerry Liu, Simon Suo</p><p>LlamaIndex is a popular open source framework for building LLM apps. TruLens is an open source library for evaluating, tracking, and iterating on LLM apps to improve their quality. The LlamaIndex and TruLens teams are actively collaborating to enable LLM app developers to rapidly build, evaluate, and iterate on their apps.</p><p>In the latest release of TruLens, we introduce tracing for LlamaIndex based LLM applications that allow you to evaluate and track your experiments with just a few lines of code. This lets you automatically evaluate a number of different components of the application stack including:</p><ul><li>App inputs and outputs</li><li>LLM calls</li><li>Retrieved context chunks from an index</li><li>Latency</li><li>Cost and Token Counts</li></ul><p>Check out this <a href="https://github.com/truera/trulens/blob/releases/rc-trulens-eval-0.3.0/trulens_eval/examples/vector-dbs/llama_index/quickstart.ipynb">notebook</a> to get started and read along to get a step by step view.</p><h3>How do I actually use this?</h3><h3>Build A LlamaIndex App</h3><p>LlamaIndex lets you connect your data to LLMs and rapidly build applications for a number of different use cases.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/2a3dcf435b9ea4311c9403ace03f33e2/href">https://medium.com/media/2a3dcf435b9ea4311c9403ace03f33e2/href</a></iframe><p>Once you build your app, you can easily query your data:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/02e9771d5073e11f094cbff452b07929/href">https://medium.com/media/02e9771d5073e11f094cbff452b07929/href</a></iframe><p>And you get an appropriate response.</p><ul><li>Growing up, the author wrote short stories, programmed on an IBM 1401, and nagged his father to buy him a TRS-80 microcomputer. He wrote simple games, a program to predict how high his model rockets would fly, and a word processor. He also studied philosophy in college, but switched to AI after becoming bored with it. He then took art classes at Harvard and applied to art schools, eventually attending RISD.</li></ul><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/5630f09b692201eaa961bbe576fc61f7/href">https://medium.com/media/5630f09b692201eaa961bbe576fc61f7/href</a></iframe><h3>Wrap A LlamaIndex App with TruLens</h3><p>With TruLens, you can wrap LlamaIndex query engines with a TruLlama wrapper. This wrapper preserves all LlamaIndex behavior, but traces all of the intermediate steps so that they can be individually evaluated.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/c1cc5765bc606a44bed8f9c2b194ba1f/href">https://medium.com/media/c1cc5765bc606a44bed8f9c2b194ba1f/href</a></iframe><p>The wrapped app can now be queried in the exact same way:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/1c32ef7f1d1c6f2459797415b0cdfdd6/href">https://medium.com/media/1c32ef7f1d1c6f2459797415b0cdfdd6/href</a></iframe><p>Except, now the details of the query are logged by TruLens.</p><h3>Add Feedback Functions</h3><p>Now to evaluate the behavior of your models, we can add feedback functions to your wrapped application. Note that as a developer you only need to <strong>add a few lines of code</strong> to start using feedback functions in your apps. You can also easily add functions tailored to the needs of your application.</p><p>Our goal with feedback functions is to programmatically check the app for quality metrics.</p><ul><li>The first feedback function checks for language match between the prompt and the response. It’s a useful check since a natural user expectation is that the response is in the same language as the prompt. It is implemented with a call to a HuggingFace API that programmatically checks for language match.</li><li>The next feedback function checks how relevant the answer is to the question by using an Open AI LLM that is prompted to produce a relevance score.</li><li>Finally, the third feedback function checks how relevant individual chunks retrieved from the vector database are to the question, again using an OpenAI LLM in a similar manner. This is useful because the retrieval step from a vector database may produce chunks that are not relevant to the question and the quality of the final response would be better if these chunks are filtered out before producing the final response.</li></ul><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/3ceca03b27fd3b14e4b112b07e7b2c03/href">https://medium.com/media/3ceca03b27fd3b14e4b112b07e7b2c03/href</a></iframe><h3>Explore In Dashboard</h3><p>Every query that is tracked can now be viewed in the TruLens dashboard. After running the feedback functions on a set of records (interactions), you can</p><p>see the aggregate results of the evaluation on a leaderboard;</p><p>then drill down into an app version and examine how it is performing on individual records .</p><p>These steps can help you understand the quality of an app version and its failure modes.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*I2ns-CdbV2FURhn3" /></figure><p>In this example, the model is doing fairly well on the relevance and language match feedback evaluations, but seems to be doing poorly on qs_relevance. This can be an indicator that the retrieved chunks are often irrelevant. This can be a significant source of “hallucinations” in retrieval-augmented generative AI apps.</p><p>We can now drill down and identify specific instances where this may be an issue:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*j6sqZ14sDBwzz2G7" /></figure><p>Let’s look at a good example first. “What did the author do growing up?”</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*ICKl7Q7L4QQywEmD" /></figure><p>In this example, we retrieved two chunks from the index both of which were fairly relevant to the the question and as a result the LLM summarizes it into a relevant and factually correct answer.</p><p>On the other hand, let’s look at an example where this didn’t go so well: “Where was the author born?”. In this example, the app confidently provides an incorrect answer.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*P3vXMlBvrEbvsbDK" /></figure><p>In this example, the two pieces of context retrieved had moderate relevance to the question. Further, neither context contained the answer. Even though our relevance feedback function (which doesn’t check for factual correctness) didn’t detect an issue, because the underlying chunks were not very relevant, this was a strong indicator that something was off. Indeed, this is an example of the model hallucinating on a question that is fairly easy to fact check.</p><h3>Iterate on your App</h3><p>Once you find issues like this with your app, it can be helpful to iterate on your prompts, models and chunking approaches to optimize your app. As you do this, you can track the performance of each version of your model with TruLens. Here is an example of a dashboard with multiple iterations testing against each other.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*bbjqUoo09KrTi7iU" /></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=fd6bb4d86aca" width="1" height="1" alt=""><hr><p><a href="https://medium.com/trulens/build-and-evaluate-llm-apps-with-llamaindex-and-trulens-fd6bb4d86aca">Build and Evaluate LLM Apps with LlamaIndex and TruLens</a> was originally published in <a href="https://medium.com/trulens">TruLens</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Evaluate and Track your LLM Experiments: Introducing TruLens]]></title>
            <link>https://medium.com/trulens/evaluate-and-track-your-llm-experiments-introducing-trulens-86028fe9b59a?source=rss-11f696611617------2</link>
            <guid isPermaLink="false">https://medium.com/p/86028fe9b59a</guid>
            <category><![CDATA[mlops]]></category>
            <category><![CDATA[large-language-models]]></category>
            <category><![CDATA[generative-ai]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[developer-tools]]></category>
            <dc:creator><![CDATA[Shayak Sen]]></dc:creator>
            <pubDate>Wed, 24 May 2023 13:02:43 GMT</pubDate>
            <atom:updated>2023-05-24T13:02:43.451Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="TruLens for LLMs, Evaluate and track your LLM experiments" src="https://cdn-images-1.medium.com/max/820/0*xdRZEiZGixtk5rWi.jpg" /><figcaption>Image by author</figcaption></figure><p>Today, we are excited to announce <a href="https://www.trulens.org/">TruLens for LLM Applications</a> — the first open source software to evaluate and track LLM experiments. You can download it now from PyPi — <a href="https://pypi.org/project/trulens-eval/">TruLens for LLMs</a> — and integrate it into your app dev process with just a few lines of code. You can leverage TruLens to rapidly iterate on your LLM apps, guard against risks from LLMs, and make tradeoffs among performance, cost, and other factors.</p><h3>Developer activity with LLMs has hit an inflection point</h3><p>The release of ChatGPT by OpenAI six months ago and its rapid acquisition of users (100 million in just 2 months!) has made obvious the tremendous opportunity and surrounding risks of Large Language Models (LLMs) to companies, governments, and the public at large. In addition to OpenAI’s GPT series, a strong set of LLMs are also available for use from other organizations, including Anthropic’s Claude, co:here, Google’s PaLM 2, Meta’s LLaMA, and HuggingFace/Big Science’s BLOOM.</p><p>The rise of LLMs has also catalyzed tremendous <strong>developer activity,</strong> with tens of thousands of developers building LLM applications, including question-answering, chatbots, document summarization, search, generative writing tasks such as marketing copy, and more. This activity has been facilitated by:</p><p><strong>(1) <em>easy-to-use APIs</em></strong> (e.g. from <a href="https://platform.openai.com/docs/introduction/key-concepts">OpenAI</a>) via which LLMs can be called without the need for hard-to-build MLOPs pipelines to collect and clean data, train, and deploy models; and</p><p><strong>(2) <em>developer frameworks</em>, </strong>such as <a href="https://python.langchain.com/en/latest/index.html">LangChain</a> and <a href="https://github.com/jerryjliu/llama_index">LlamaIndex</a> for augmenting LLMs with additional resources (e.g. external data stored in vector databases like <a href="https://www.pinecone.io/">Pinecone</a> and <a href="https://www.trychroma.com/">Chroma</a>).</p><h3>Experimentation is key to building powerful LLM apps — but it’s too hard to do today</h3><p>The workflow for building LLM applications involves significant experimentation. Consider a developer building a Question Answering application. After developing the first version of the app, she may test it by supplying a series of questions and manually reviewing the answers. If a question is incorrectly answered, she may make adjustments — changing prompts or hyperparameters of the retrieval system or fine-tuning LLMs — and then re-testing the revised app. This iterative, highly manual process continues until she gets the app to a satisfactory state.</p><p>This process is time consuming. Until now, there were no useful tools to measure app performance and quality metrics. It was also difficult to track how metrics are improving (or not) after each iteration.</p><h3>TruLens helps you refine LLM experiments quickly with programmatic feedback</h3><p>TruLens fills this gap in the LLMOps tech stack by supporting LLM developers with tooling to evaluate and track LLM experiments. TruLens leverages a new framework that we’re calling <a href="https://truera.com/whats-missing-to-evaluate-foundation-models-at-scale/"><em>feedback functions</em></a> to programmatically evaluate the quality of inputs, outputs, and intermediate results from LLM applications, thus scaling up the human review steps. While TruLens comes with an out-of-the-box set of feedback functions, we invite developers to add functions tailored to their application needs to the library. Out-of-the-box feedback functions today include:</p><ul><li>Truthfulness</li><li>Question answering relevance</li><li>Harmful or toxic language</li><li>User sentiment</li><li>Language mismatch</li><li>Response verbosity</li></ul><figure><img alt="How to track and evaluate LLM applications with TruLens" src="https://cdn-images-1.medium.com/max/1024/1*GXYbGwjceVM_OtISOXVAUQ.png" /><figcaption>Image by author</figcaption></figure><p><em>Figure 1: Workflow for faster LLM app development using TruLens</em></p><h3>How to accelerate and improve LLM app development with TruLens</h3><p>Developers can use TruLens as they are building their LLM applications in Python by following these steps, illustrated with a running example of a Question-Answering (QA) application. You can also go directly to the <a href="https://trulens.org/trulens_eval/quickstart/">TruLens for LLMs Quick Start page</a> to get started with using TruLens right away.</p><p><strong>1. Build your LLM app</strong></p><p>When you are using TruLens, you would build the first version of your LLM app following your standard workflow.</p><p>We built a Question Answering app named “TruBot” following the widely used paradigm of retrieval-augmented LLMs. This approach grounds the app’s responses in a source of truth or knowledge base — TruEra’s website in this case. It involved chaining together the OpenAI LLM with the Pinecone vector database in the LangChain framework (see Appendix A for more details). The code for the key steps for implementing this app are shown in Figure 2.</p><figure><img alt="Retrieval Augmented Question Answering with Langchain and Pinecone" src="https://cdn-images-1.medium.com/max/521/1*QrNyJzj3Ms55WAgGDgcVJw.png" /><figcaption>Image by author</figcaption></figure><p><em>Figure 2: Example of a chained Question Answering LLM app, TruBot</em></p><p><strong>2. Connect your LLM app to TruLens and log inputs and responses</strong></p><p>The next step is to instrument the app with TruLens to log inputs and responses from the chain. Note that this step is very easy — it involves wrapping the previously created chain, running it, and logging the interaction with <strong>just 3 lines of code</strong>.</p><figure><img alt="Calling TruLens with Langchain" src="https://cdn-images-1.medium.com/max/635/1*qT6RIApX0a9Pxh8enh3HXg.png" /><figcaption>Image by author</figcaption></figure><p><em>Figure 3: How to connect a Question Answering LLM app to TruLens</em></p><p><strong>3. Use feedback functions to evaluate and log the quality of LLM app results</strong></p><p>The third step is to run feedback functions on the prompt and responses from the app and to log the evaluation results. Note that as a developer you only need to <strong>add a few lines of code</strong> to start using feedback functions in your apps (see Figure 4(a)). You can also easily add functions tailored to the needs of your application.</p><p>Our goal with feedback functions is to programmatically check the app for quality metrics.</p><ul><li>The first feedback function checks for language match between the prompt and the response. It’s a useful check since a natural user expectation is that the response is in the same language as the prompt. It is implemented with a call to a HuggingFace API that programmatically checks for language match.</li><li>The next feedback function checks how relevant the answer is to the question by using an Open AI LLM that is prompted to produce a relevance score.</li><li>Finally, the third feedback function checks how relevant individual chunks retrieved from the vector database are to the question, again using an OpenAI LLM in a similar manner. This is useful because the retrieval step from a vector database may produce chunks that are not relevant to the question and the quality of the final response would be better if these chunks are filtered out before producing the final response.</li></ul><figure><img alt="Adding Trulens feedback functions" src="https://cdn-images-1.medium.com/max/555/1*Dz1EsUJVAIwA41EOGUxgLg.png" /></figure><p><em>Figure 4(a): Example of feedback functions to check for language match &amp; relevance</em></p><ul><li>You can also configure TruChain to use automatic feedback computation and logging as shown in Figure 4(b). This combines steps 2 and 3 into <strong>one line of code</strong>.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/645/1*9z6LV0NBbFEOR5xzQYrD0Q.png" /></figure><p><em>Figure 4(b): Configuring TruChain to use automatic feedback computation and logging of a feedback function for LLMs</em></p><p>Note that feedback functions are a general abstraction. They can be implemented in a number of ways. One practical consideration in using feedback functions is cost, in particular, when they are implemented by calling LLMs over APIs. We note that there are a number of free implementations that are available from OpenAI and Huggingface that provide a useful starting point. Over time, we also expect that the library of feedback functions will incorporate other low cost mechanisms, including but not limited to previous generation of BERT-style foundation models, as well as with simpler rule based systems. We refer the interested reader to our <a href="https://truera.com/whats-missing-to-evaluate-foundation-models-at-scale/">article on feedback functions</a> for more detail.</p><p><strong>4. Explore in dashboard</strong></p><p>After running the feedback functions on a set of records (interactions), you can</p><ul><li>see the aggregate results of the evaluation on a leaderboard (see Figure 5);</li><li>then drill down into an app version (or chain) and examine how it is performing on individual records (see Figure 6).</li></ul><p>These steps can help you understand the quality of an app version and its failure modes.</p><p>For example, in Figure 5, we see that this app version (Chain 0) is not doing well on the language match score from our feedback function. Drilling down in Figure 6, we discover that when questions are asked in German it responds in English instead of German — that’s the failure mode.</p><p>The app is doing well on overall relevance of responses to questions. However, it is performing poorly on the <em>qs_relevance</em> feedback function. Recall that this function checks how relevant individual chunks retrieved from the vector database are to the question. See Appendix B for an explanation of this failure mode.</p><figure><img alt="Chain leaderboard" src="https://cdn-images-1.medium.com/max/1024/1*vDgVEkn-8TIgZt4tHUjM4Q.png" /><figcaption>Image by author</figcaption></figure><p><em>Figure 5: Example of a feedback function chain leaderboard for evaluating LLM apps</em></p><figure><img alt="Evaluations" src="https://cdn-images-1.medium.com/max/1024/1*96paek5L7VpqEuqK8OxfLQ.png" /><figcaption>Image by author</figcaption></figure><p><em>Figure 6: The Evaluations table provides a drill down of the evaluation at the record level. Note that on the language match test, this chain does not perform well when asked questions in German whereas it correctly responds in Spanish when asked a question in Spanish.</em></p><p><strong>5. Iterate to get to the best chain</strong></p><p>Armed with an understanding of the failure modes of the first chain, you then proceed to iterate on your app.</p><p>To address the language match failure mode, you adjust your prompt as shown in Figure 7. Running the evaluation again on the new version of the app (Chain 1), you see an improvement on the leaderboard — the feedback function score for language_match went from 0.44 to 0.93 (see Figure 8).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*GfQgopw1Xwtu-JHx5cIUJw.png" /><figcaption>Image by author</figcaption></figure><p><em>Figure 7: Adjusted prompt to address language match failure mode that is surfaced by a feedback function. Note the explicit instruction to answer in the same language as the question.</em></p><p>To address the irrelevant chunks failure mode, you filter the contexts using the <em>qs_relevance</em> feedback function as shown in Figure 8. Running the evaluation again on the new version of the app (Chain 3), you see an improvement on the leaderboard — the feedback function score for <em>qs_relevance</em> increases significantly (see Figure 8). Chain 4 incorporates both the language prompt adjustment and the context filtering step.</p><figure><img alt="Chain leader board with experiments" src="https://cdn-images-1.medium.com/max/1024/1*dBGbOGr6QBTxLmdjpaRsWA.png" /><figcaption>Image by author</figcaption></figure><p><em>Figure 8: The Leaderboard shows the quality of chains across records as scored by various feedback functions. Chain 4 incorporates both the language prompt adjustment and the context filtering step.</em></p><p>While in our running example, we start with one app version, you could also experiment with various choices in building your app, e.g. LLMs to call out to, hyperparameters to play with, such as the chunk size to break your documents down to in order to compute embeddings to store in a vector database, and more. You would then run all these app versions through Steps 1–5 to evaluate and track your experiments to select the best version.</p><h3>Try TruLens today</h3><p>TruLens is free, available today, and ready to help you evaluate and track your LLM experiments. TruLens increases developer productivity while offering greater visibility and confidence in the quality of LLM applications. We hope that you give it a try, give it a Github star, and let us know how it’s working out for you!</p><p>Give it a spin: <a href="https://pypi.org/project/trulens-eval/">Get TruLens</a></p><p>Give it a star: <a href="https://github.com/truera/trulens/tree/main/trulens_eval">TruLens on Github</a></p><p>Learn more: <a href="https://www.trulens.org/">TruLens site</a></p><p>Join the community: <a href="https://communityinviter.com/apps/aiqualityforum/josh">AI Quality Forum Slack Community</a></p><p><em>This blog was co-written with Anupam Datta, Piotr Mardziel, and Yuvnesh Modi.</em></p><h3>Appendix A: Building a retrieval-augmented LLM app</h3><p>We built a simple Question Answering app named TruBot in two steps following the widely used paradigm of retrieval-augmented LLMs. This approach grounds the apps responses in a source of truth — a knowledge base.</p><p><strong>Step 1:</strong> Create embeddings of chunked up documents from the TruEra website (the knowledge base) with an OpenAI LLM and store in Pinecone, a vector database.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Dj0tzRQQR-9dOb3Ywf2bqQ.png" /><figcaption>Image by author</figcaption></figure><p><em>Figure 10: Create embeddings of chunked up documents from the TruEra website (the knowledge base) with an OpenAI LLM and store in Pinecone, a vector database.</em></p><p><strong>Step 2:</strong> Create an app by chaining the OpenAI LLM with Pinecone vector database and use it to answer questions as shown in Figure 3.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*AEo5Cbu4VwRYMuBPcOMHLg.png" /><figcaption>Image by author</figcaption></figure><p><em>Figure 11: Steps for answering questions in an LLM app using the chained Question Answering application.</em></p><h3>Appendix B: Checking for relevance of chunks</h3><p>The TruBot app was tested with a feedback function that checks how relevant individual chunks retrieved from the vector database are to the question.</p><p>The results of this test when asked the question “Who is Shayak?” are shown in Figure 10. The vector database returned 4 context chunks — the first two were individually relevant to the question (they were about Shayak) and the last two were not relevant to the question (they were about a different TruEra employee — Shameek). The final response was affected by the irrelevant chunks — the last sentence in the response includes information about Shameek in answering the question about Shayak.</p><p>Notice that the feedback function correctly gave high relevance scores to the first two chunks and low relevance scores to the last two chunks. This information can be used to only send the first two chunks for summarization, thus improving the quality of the final response.</p><figure><img alt="Checking for relevance of text" src="https://cdn-images-1.medium.com/max/1024/1*zprza4gptTp4jSb3E66Tsg.png" /><figcaption>Image by author</figcaption></figure><p><em>Figure 12: Feedback function scores of qs_relevance, i.e. how relevant individual chunks are to the question.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=86028fe9b59a" width="1" height="1" alt=""><hr><p><a href="https://medium.com/trulens/evaluate-and-track-your-llm-experiments-introducing-trulens-86028fe9b59a">Evaluate and Track your LLM Experiments: Introducing TruLens</a> was originally published in <a href="https://medium.com/trulens">TruLens</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>