<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Sheldon L. on Medium]]></title>
        <description><![CDATA[Stories by Sheldon L. on Medium]]></description>
        <link>https://medium.com/@sheldon88?source=rss-fa9f017abe9d------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*GJyoa7Nyf31saLR0FP_-RQ.jpeg</url>
            <title>Stories by Sheldon L. on Medium</title>
            <link>https://medium.com/@sheldon88?source=rss-fa9f017abe9d------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sun, 22 Nov 2026 01:49:17 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@sheldon88/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Evaluation on LLMs]]></title>
            <description><![CDATA[<div class="medium-feed-item"><p class="medium-feed-image"><a href="https://medium.com/@sheldon88/evaluation-on-llms-4914215459d7?source=rss-fa9f017abe9d------2"><img src="https://cdn-images-1.medium.com/max/600/1*E48XSforJKkuF_hwKhTrDQ.png" width="600"></a></p><p class="medium-feed-snippet">The advent of large language models (LLMs) such as ChatGPT and others, has brought forth a new era of possibilities and challenges. Coming&#x2026;</p><p class="medium-feed-link"><a href="https://medium.com/@sheldon88/evaluation-on-llms-4914215459d7?source=rss-fa9f017abe9d------2">Continue reading on Medium »</a></p></div>]]></description>
            <link>https://medium.com/@sheldon88/evaluation-on-llms-4914215459d7?source=rss-fa9f017abe9d------2</link>
            <guid isPermaLink="false">https://medium.com/p/4914215459d7</guid>
            <category><![CDATA[large-language-models]]></category>
            <category><![CDATA[chatgpt]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[genrative-ai]]></category>
            <category><![CDATA[model-evaluation]]></category>
            <dc:creator><![CDATA[Sheldon L.]]></dc:creator>
            <pubDate>Sun, 04 Jun 2023 03:30:28 GMT</pubDate>
            <atom:updated>2023-06-04T03:30:28.183Z</atom:updated>
        </item>
        <item>
            <title><![CDATA[Are LLMs Changing the Way We Search?]]></title>
            <link>https://medium.com/@sheldon88/are-llms-changing-the-way-we-search-72c9e720c166?source=rss-fa9f017abe9d------2</link>
            <guid isPermaLink="false">https://medium.com/p/72c9e720c166</guid>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[search]]></category>
            <category><![CDATA[data-scien]]></category>
            <category><![CDATA[chatgpt]]></category>
            <dc:creator><![CDATA[Sheldon L.]]></dc:creator>
            <pubDate>Wed, 26 Apr 2023 13:14:21 GMT</pubDate>
            <atom:updated>2023-04-26T13:14:21.557Z</atom:updated>
            <content:encoded><![CDATA[<p>For a long period of time (more than two decades since Google was founded in 1998 or even earlier if we count Yahoo who was the first company to index the web), we have been used to the way how we look up things on the internet — have a simple search bar where you type in the key words, then look up the pop-up results to find the answer to your questions. This was once considered the most efficient way to look up things.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/822/1*LkfDcmwktLU-6iRDsNHZjQ.png" /></figure><p>However, without further ado, ChatGPT which is the disruptor in town, or more broadly speaking, LLMs (Large Language Models) are changing the way we search.</p><h4>How the index search works?</h4><p>To elaborate the point I just made, let’s first understand how the index search works.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/646/1*-1Z9omkUtnAEgY6_cg8M3g.png" /><figcaption>Index Search Diagram (Simplified)</figcaption></figure><p>As illustrated on the diagram above, there are three key steps in the index search. More details can be found <a href="https://developers.google.com/search/docs/fundamentals/how-search-works#:~:text=Introducing%20the%20three%20stages%20of%20Google%20Search,-Google%20Search%20works&amp;text=Indexing%3A%20Google%20analyzes%20the%20text,relevant%20to%20the%20user&#39;s%20query.">here</a>.</p><p><strong>Crawling</strong> — search engine has crawlers constantly look for new and updated web pages including text, image and video content and pass to index engine.</p><p><strong>Indexing</strong> — An index engine processes the content passed by the crawler and convert it into a vector space then save to index database.</p><p><strong>Ranking</strong> — When the user searches for an item, search engine looks for the most relevant content (based on text similarity, geolocation, history etc.) and passes to the ranking engine to provide ranked search results.</p><p>Let’s use this example to illustrate. Say I would like to know how to make an rest API call to OpenAI model endpoint in python.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/887/1*etAGP0t5wMF2kRnfWXPARw.png" /></figure><h4>How are LLMs changing the game?</h4><p>Just a few months ago, most people would have found the above results satisfactory. One may just click on the links and figure out how to use it. But not any longer since LLMs took over the control in the search game…</p><p>This is what I got from ChatGPT backed Bing Chat.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*n21WEfSIAoAppn6VXNzDPw.png" /></figure><p>LLMs have read the ranked search content for me, and provided me with the most relevant information based on its own understanding on my query and the ranked content. More importantly, it provides sources for the answer, from which I can do further reading if needed. Depends on how busy I get on that day, I can choose to just use the code provided by Bing (ChatGPT) or do a further reading.</p><p><em>How was this done by Bing Chat? Do I need to feed the whole OpenAI API reference document to ChatGPT in a single prompt? I thought there is limit of tokens on a single prompt?</em></p><p>Yes, there is a limit on number of tokens. ChatGPT-3.5 (Turbo) has limit of 4096 tokens (approx. 3072 words) whereas GPT-4 has limit of 8192 tokens (approx. 6144 words). But this is not how it’s done.</p><p>Many people call this a ChatBot (including Microsoft themselves), but to me, this is still a search which is enhanced by LLMs. Let me show you the index search diagram again, this time we added the LLM component.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/618/1*5anXl3r6iEyRwuMt87TrNA.png" /><figcaption>LLM-enhanced search</figcaption></figure><p>The final response passed to the user is a combination of GPT model response and the links of the source saved in the index database.</p><h4>Does this work for the internal knowledge base search for a company?</h4><p>Yes, absolutely. The only difference is that we don’t need crawlers any more as in this case, we would just need to ingest all the internal knowledge base articles/documents to a database. Microsoft has an example of using their Cognitive Search and Azure OpenAI service for this use case. Further details can be found in their <a href="https://github.com/Azure-Samples/azure-search-openai-demo">official Github repository</a>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ICG5iJ630W8UwjNdWMFJ_w.png" /><figcaption>Source: <a href="https://github.com/Azure-Samples/azure-search-openai-demo">https://github.com/Azure-Samples/azure-search-openai-demo</a></figcaption></figure><h4>What’s next?</h4><p>I will be writing a series of blogs on the topics to cover:</p><ol><li>The internal knowledge base ChatGPT implementation using Azure managed services.</li><li>Open-source framework implementation using a Python Package called Langchain.</li></ol><p>Thanks for reading and please follow me to stay tuned for the new articles!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=72c9e720c166" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>