<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Arun Saxena on Medium]]></title>
        <description><![CDATA[Stories by Arun Saxena on Medium]]></description>
        <link>https://medium.com/@asaxena_43463?source=rss-99f899e3817a------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*-sHVHiZB-T-7iMaQHhwNJg.jpeg</url>
            <title>Stories by Arun Saxena on Medium</title>
            <link>https://medium.com/@asaxena_43463?source=rss-99f899e3817a------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Fri, 20 Nov 2026 23:11:35 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@asaxena_43463/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Qualification doesn’t belong in your filters]]></title>
            <link>https://techwitharun.com/qualification-doesnt-belong-in-your-filters-75e8e8373b8e?source=rss-99f899e3817a------2</link>
            <guid isPermaLink="false">https://medium.com/p/75e8e8373b8e</guid>
            <category><![CDATA[sales-operations]]></category>
            <category><![CDATA[clay]]></category>
            <category><![CDATA[gtm-engineering]]></category>
            <category><![CDATA[field-notes]]></category>
            <category><![CDATA[data-quality]]></category>
            <dc:creator><![CDATA[Arun Saxena]]></dc:creator>
            <pubDate>Thu, 13 Aug 2026 13:41:43 GMT</pubDate>
            <atom:updated>2026-08-14T07:11:10.203Z</atom:updated>
            <content:encoded><![CDATA[<h4>Field Notes #2</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*9HWSYGxrqV7c5iZ_5qBPGw.png" /></figure><p>I spent about ninety minutes building a list of fifty companies in Clay, most of it on the filters. Adding conditions, checking the preview, removing conditions, adding different ones. The list got cleaner. Then it stopped getting cleaner, and I could not work out why.</p><p>The answer turned out to be structural, and it changed how I think about where qualification logic should live.</p><h3>The ceiling</h3><p>Here is the sequence, as logged. Same result limit of 50 throughout, only the filter set changing:</p><ul><li>494 companies matched</li><li>81,472</li><li>63,766</li><li>42,098</li><li>22,934</li><li>11,658</li></ul><p>Six iterations, an order of magnitude of movement in the matched pool. TechCrunch, VentureBeat, Entrepreneur Media and ByteByteGo sat near the top of the preview through almost all of it.</p><p>That is not a filtering problem. It is a sampling problem.</p><p>At a limit of 50 against a pool of 11,658, I am seeing 0.4% of what matched. Against 81,472 I was seeing 0.06%. At those rates the question is not <em>which companies match</em> but <em>which 50 the ranking hands me</em>, and the ranking is not something my filters address. Narrowing the pool from 81K to 12K removes 69,000 companies I was never going to see anyway.</p><p>Filters reshape the pool. They only reshape the sample when they specifically exclude whatever is currently at the top. Everything else is effort with no observable effect — which is exactly what it felt like, for about forty minutes, before I understood what I was looking at.</p><p>The practical consequence: filter effort has a hard ceiling, and it arrives earlier than you would expect. Past that point you are not improving your list, you are rearranging a population you cannot see.</p><h3>Why the taxonomy can’t carry it</h3><p>The obvious response is better filters rather than more of them. Use industry properly. Get specific.</p><p>TechCrunch, VentureBeat and Entrepreneur Media all self-report as Software Development on LinkedIn.</p><p>They are not being dishonest. They are technology media companies, they employ engineers, they build products, and Software Development is the nearest available bucket in a taxonomy that has to cover every company on the platform. The same value is doing very different work for each of them.</p><p>This matters more than it first appears. LinkedIn’s industry list is a shared vocabulary that neither you nor Clay controls, populated by self-selection, and revised periodically — Computer Software, which I went looking for, no longer exists as a filterable value.</p><p>You cannot build qualification on a field with those properties. It is a coarse population filter and nothing more.</p><p>The same holds for anything AI-inferred. Early on I let Clay’s assistant construct my filters and ended up with AI · Business types contains B2B OR AI · Business types is empty. That second clause is a recall-over-precision decision — let through everything the classifier could not label — and it is the reason a venture capital firm and a software deals marketplace ended up in my list of B2B SaaS companies. The choice was reasonable. It was also made for me, and I did not notice until I read the filter back.</p><p>Worth knowing which of your conditions are attribute lookups and which are model judgments. They behave differently and they fail differently.</p><h3>Where the fragility actually lives</h3><p>Then I ran an enrichment and got the finding that reorganised my thinking.</p><p>One company, adplist.org, in a single row of my table:</p><ul><li><strong>Size:</strong> 2–10 employees</li><li><strong>Employee Count:</strong> 2,353</li><li><strong>Annual Revenue:</strong> 500M–1B</li></ul><p>Three fields, three mutually exclusive descriptions of the same organisation, all sourced through the same platform, all sitting side by side.</p><p>This is not a one-off. My first session produced WhatsApp at 51–200 employees, which I filed as an amusing outlier. Two instances make it a pattern, and the pattern has a specific shape: these fields come from different providers with different collection methods and different refresh cadences, and Clay presents them in one row with no indication that they disagree.</p><p>Nothing in the interface flags a contradiction. The cells render identically whether they agree or not.</p><p>So: any logic conditioned on a single enrichment field is structurally fragile, not because the field is often wrong, but because you have no signal telling you when it is. A rule reading Employee Count &gt; 1000 would classify adplist.org as an enterprise account. A rule reading Size == &quot;2-10&quot; would classify it as a micro-business. Both are defensible. Both are drawn from the same row.</p><p>The unit of truth cannot be a field. It has to be agreement across fields.</p><p>There is a cost dimension too. One row in my test returned Country and nothing else — no revenue, no employee count — and was charged the full 0.5 credits, same as a complete record. Cost is per call, not per field returned. Which means a hit rate quoted as a single number is close to meaningless: 80% coverage on a five-field enrichment could be five fields on 80% of rows, or one field on all of them. Those are very different deliverables at identical cost.</p><h3>The technique that did work</h3><p>The filter that finally cleaned my list was annual revenue, $5M–$10M. Media brands and mismatched-size companies dropped out immediately.</p><p>It is worth being precise about why, because the reason is not the obvious one.</p><p>Revenue on these platforms is a modelled estimate, not filed data — inferred from headcount, category and similar signals. As a statement of fact about a specific company it is unreliable.</p><p>But that unreliability is exactly what made it useful. The model breaks down for companies whose LinkedIn footprint does not match their real shape — a media brand with a large following and a small commercial operation, or a global product with a thinly maintained page. Those are precisely the companies contaminating my list. The estimate fails for them in a way it does not fail for a straightforward 30-person SaaS business, and that differential failure is a signal.</p><p>I was not filtering on revenue. I was filtering on <em>whether the revenue model behaves coherently for this company</em>, which is a proxy for <em>whether this company is the shape I think it is</em>.</p><p>Deliberately using a weak signal for something other than its stated purpose is a legitimate technique. It requires knowing how the signal is constructed and where it degrades, which is a different kind of knowledge from knowing what the field is called.</p><p>The corollary matters as much: the same field would be indefensible in a report. Same data, same table, appropriate in one position and not the other.</p><h3>Where this leaves the architecture</h3><p>Putting the four findings together:</p><p>Filters cannot do qualification, because sampling puts a ceiling on their effect and the underlying fields are too coarse to carry the weight.</p><p>Single-field conditions cannot do qualification, because fields disagree and nothing surfaces the disagreement.</p><p>What is left is a scoring layer that sits after enrichment and derives a qualification signal from cross-field agreement. Not “is Employee Count above 1000,” but “do headcount, revenue band and size bucket describe the same organisation, and how confident should I be if they don’t.”</p><p>Two design constraints follow:</p><p><strong>It has to be reproducible from raw fields.</strong> A scoring column built as a formula off raw enrichment outputs can be traced backwards: this company scored 7 because these four fields held these values. A scoring column built as an AI judgment produces a number you cannot defend when asked why their biggest prospect was excluded. In Clay specifically, formulas are also free — they consume no credits at all, on any number of rows — while AI enrichment columns are metered per row. So the defensible option is also the cheap one, which is a rare alignment and worth taking.</p><p><strong>It has to encode disagreement rather than hide it.</strong> If three fields describing company scale contradict each other, the score should reflect lower confidence, not silently pick whichever field the rule happened to reference. That is the difference between a system that degrades visibly and one that fails quietly.</p><p>Neither of these is difficult. They are just decisions that get made by default if nobody makes them deliberately — and the default is a rule referencing whichever field was most convenient at the time.</p><h3>What I have not built</h3><p>I have not built the scoring layer. That is the next rung, and I will write up the credit numbers and the failure modes when I have them.</p><p>What I have done is establish, from my own table, why the filter-first instinct hits a wall: the sampling ceiling, the taxonomy limits, and the cross-field contradictions. The architecture argument follows from those, but the implementation is still ahead of me.</p><p>I would be interested to know whether anyone has found a reliable way to surface cross-field disagreement inside Clay itself, rather than reconstructing it afterwards in a formula. If that exists, I have not found it.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=75e8e8373b8e" width="1" height="1" alt=""><hr><p><a href="https://techwitharun.com/qualification-doesnt-belong-in-your-filters-75e8e8373b8e">Qualification doesn’t belong in your filters</a> was originally published in <a href="https://techwitharun.com/">Arun Saxena’s Tech Terrain</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Azure Video Indexer]]></title>
            <link>https://medium.com/@asaxena_43463/azure-video-indexer-2c6468545721?source=rss-99f899e3817a------2</link>
            <guid isPermaLink="false">https://medium.com/p/2c6468545721</guid>
            <dc:creator><![CDATA[Arun Saxena]]></dc:creator>
            <pubDate>Wed, 12 Aug 2026 09:00:08 GMT</pubDate>
            <atom:updated>2026-08-12T09:00:08.450Z</atom:updated>
            <content:encoded><![CDATA[<h4>Unravel insights in your videos with AI</h4><h3>Introduction</h3><p>Azure Video Indexer (AVI) is a composite service layered on other AI services like:</p><ol><li>Face</li><li>Translator</li><li>Vision</li><li>Speech</li></ol><p>AVI uses 30+ AI models to gain rich insights. There are many scenarios we can use this service:</p><ol><li>Search: We can index all the insights and search our video library. For example, we can search for all videos in which Mr X spoke about topic Y in January 2023.</li><li>Content creation: We can use the various markers extracted as insights to locate sections of the video and then stitch them together to create reels or trailers.</li><li>Content Moderation: Validate and block content that&#39;s flagged as unsafe.</li><li>Recommendations: Improve end-user engagement by highlighting relevant video moments.</li></ol><p>We can get the following insights from videos:</p><ol><li>Detect faces — of already indexed celebrities and other faces that we can custom-train AVI on.</li><li>OCR</li><li>Identification of objects like chairs etc.</li><li>Scene segmentation — a scene is a series of frames in the video that are semantically related.</li><li>Shot detection is a series of frames from the same motion-picture camera. We can also tag the shot as close-up, multiple people, etc.</li></ol><p>Insights from Audio:</p><ol><li>Transcription with language detection</li><li>Captioning</li><li>Noise reduction</li><li>Content moderation</li><li>Translation</li><li>Audio effects like sirens, barks, clapping, etc.</li><li>Sentiment analysis</li></ol><h3>Scope of this article</h3><p>This article will upload videos to an Azure storage account and list them to Video Indexer. We will then store the results of the video analysis in DB and use Video Indexer API to show it on our web page</p><p>We will be using the following Azure services:</p><ol><li>BLOB Storage service to store the videos</li><li>Azure Functions to write application logic in. All our API endpoints will also be here.</li><li>Video Indexer service to transcribe our video and analyze sentiments.</li><li>SQL Service to store the results of Video Analysis</li></ol><h3>Generate SAS URL Endpoint</h3><p>To upload a video to BLOB storage, we first need to get a Shared Access Signature (SAS) token and use it in the URL to push the video into the BLOB.</p><p>The SAS token encapsulates a set of permissions and a valid time interval. This means that we can perform activities on storage BLOB from a specific start time to a specific end time.</p><p>Regarding SAS-based access, we can provide access to the entire storage account (Account Access) or only to a specific service (one of BLOB, Queue, Tables, File).</p><p>Since we are only interested in BLOBs, we will create a Service SAS. You can get more information here:</p><p><a href="https://learn.microsoft.com/en-us/rest/api/storageservices/create-service-sas">Create a service SAS - Azure Storage</a></p><p>While creating a service SAS token, we can assign granular permissions. For example, we want to allow video upload only if a file with the same name does not exist.</p><p>Read more about permissions and other settings here:</p><p><a href="https://learn.microsoft.com/en-us/rest/api/storageservices/create-service-sas#specify-permissions">Create a service SAS - Azure Storage</a></p><p>Here is the Azure Function that generates this service SAS URL for us that allows up to only writing the file to the container:</p><pre>import { app, HttpRequest, HttpResponseInit, InvocationContext } from &quot;@azure/functions&quot;;<br>import { BlobSASPermissions, ContainerSASPermissions, generateBlobSASQueryParameters } from &quot;@azure/storage-blob&quot;;<br>import { randomstring } from &quot;../common/randomstring&quot;;<br>import { error } from &quot;console&quot;;<br><br>/**<br> * The UI client will make a call to this function to get a SAS URL to the BLOB to upload it.<br> * We will be generating a random file name with the specified extension. This avoids file name related issues<br> * In order to upload tht BLOB you will need to use the URL along with a header<br> * x-ms-blob-type BlockBlob<br> * Make it a PUT request<br> * The url will also allow you to ONLY create the blob and not overwrite it.<br> * @param request - No parameter expected.<br> * @param context<br> * @returns<br> */<br>export async function sasurl(/service/request: HttpRequest, context: InvocationContext): Promise&lt;HttpResponseInit&gt; {<br> try {<br>  var ext = request.query.get(&quot;ext&quot;); // the UI supplies extension of the video file<br>  if (ext === null) throw &quot;ExtAbsent&quot;; // if extension is not supplied.<br><br><br>  const { BlobServiceClient, StorageSharedKeyCredential } = require(&quot;@azure/storage-blob&quot;);<br>  const accountName = process.env.PORTFOLIO_STORAGE_ACCOUNT; // Storage account name<br>  const accountKey = process.env.PORTFOLIO_STORAGE_ACCOUNT_KEY; // secret storage account key<br><br>  if (typeof accountName === &quot;undefined&quot; || typeof accountKey === &#39;undefined&#39;)<br>   throw &quot;Account Name or Account Key missing&quot;;<br><br><br>  const sharedKeyCredential = new StorageSharedKeyCredential(accountName, accountKey);<br><br>  const blobServiceClient = new BlobServiceClient(<br>   `https://${accountName}.blob.core.windows.net`,<br>   sharedKeyCredential<br>  );<br><br>  const containerName = process.env.PORTFOLIO_STORAGE_CONTAINER_NAME; // the container where we wish to upload video<br><br>  if(typeof containerName === &#39;undefined&#39;)<br>   throw &quot;Container Name is not provided&quot;<br><br>  const fileName = randomstring(16) + &quot;.&quot; + ext; // generate a random filename of 10 chars.<br>  <br>  const containerClient = await blobServiceClient.getContainerClient(containerName);<br>  const blobClient = await containerClient.getBlockBlobClient(fileName);<br><br>  const durationInMins = process.env.SAS_TOKEN_DURATION_MINS; // The duration in minutes that the SAS URL needs to be active.<br>  <br>  if(typeof durationInMins === &#39;undefined&#39;)<br>   throw &#39;Duration for SAS token not provided&#39;<br><br>  const sasOptions = {<br>   containerName: containerName,<br>   permissions: BlobSASPermissions.parse(&quot;w&quot;), // allows only write operations <br>   startsOn: new Date(),<br>   blobName: fileName,<br>   expiresOn: new Date(new Date().valueOf() + parseInt(durationInMins) * 60 * 1000), // this setting is supplied in millisecs<br>  };<br><br>  const sasToken = generateBlobSASQueryParameters(sasOptions, sharedKeyCredential).toString();<br><br>  console.log(`${blobClient.url}?${sasToken}`);<br><br>  return { body: `${blobClient.url}?${sasToken}` };<br>  <br> } catch (error) {<br>  if (error === &quot;ExtAbsent&quot;) {<br>   context.log(&quot;The extension is missing&quot;);<br>   return {<br>    status: 400,<br>    body: &quot;Extension is not provided&quot;,<br>   };<br>  } else {<br>   context.log(error);<br>   return {<br>    status: 500,<br>    body: &quot;Unknown server error&quot;,<br>   };<br>  }<br> }<br>}<br><br>app.http(&quot;sasurl&quot;, {<br> methods: [&quot;GET&quot;, &quot;POST&quot;],<br> authLevel: &quot;anonymous&quot;,<br> handler: sasurl,<br>});</pre><p>We are using a random file generator. The UI will need to upload the file using the same name. This way, we control file names nicely and avoid all the sticky issues with user-specified file names. We will, however, save the user&#39;s original file name in the DB to provide a familiar name to the user.</p><p>Here is the code for the file name generator:</p><pre>export function randomstring(len: number): string{<br>    const arr = &#39;ABCDEFGHIJKLMNOPQRSTabcdefghijklmnopqrst1234567890-_!()&#39;<br>    var retArr = []<br>    for (var i = 0; i &lt; len; i++){<br>        retArr[i] = arr[Math.floor(Math.random()*arr.length)]<br>    }<br>    return retArr.join(&#39;&#39;)<br><br>}</pre><p>If you examine the returned URL, you will see the following:</p><pre>sv: 2024-08-04 // signed version - the version of the shared access signature<br>sr: b // the required service, b for blob.<br>st: 2024-09-10T11:24:27Z // signed start, the instant when the URL becomes valid<br>se: 2024-09-10T11:54:27Z // signed end, the instant when the URL becomes invalid<br>sp: w // signed permissions - only create, no replacement allowed.<br>sig: &lt;signed signature&gt; // Hash based Message Authentication Code HMAC</pre><h3>Upload the Video</h3><p>Using this, our user interface will upload the video. Apart from the SAS service URL, the UI will need to supply one additional header:</p><pre>x-ms-blob-type: BlockBlob</pre><p>The HTTP method to use is PUT.</p><p>Thats it! The video will be uploaded.</p><p>Next, we need to inform another Azure function about the successful upload so it can create an entry in the database.</p><p>We could have done it in the first function but didn&#39;t, just in case the video upload failed.</p><h3>Make a DB entry for the Video</h3><p>To start using AVI, we first need to create an account. You can get started with a free trial account:</p><p><a href="https://learn.microsoft.com/en-in/azure/azure-video-indexer/create-account?tabs=portal#trial-account">Create an Azure AI Video Indexer (VI) account</a></p><p>or if you already have a paid subscription, you can use that to make an Azure AI Video Indexer resource:</p><p><a href="https://learn.microsoft.com/en-in/azure/azure-video-indexer/create-account?tabs=portal#paid-account">Create an Azure AI Video Indexer (VI) account</a></p><p>When creating an AVI resource, it&#39;s a good idea to link it with Azure OpenAI to unlock next-gen AI features like summarizing your video with OpenAI. This is optional, and you do it if you need to.</p><p>Feel free to use system-assigned managed identities, as that frees you from cleaning up managed identities. Here, Azure will manage the entire lifecycle of the identity that this resource will use to access other Azure services.</p><p>AVI will need access to a storage account at minimum, and Azure will prompt you to allow the system-managed identity (the one that Azure creates on your behalf) to access the storage account that you select when creating the AVI resource.</p><p>Next, we must set up a way to access this resource using API. In the Microsoft world, it means:</p><ol><li>First, get an access token (AT1) granted by Entra ID. This access token identifies you in the Microsoft world and has a way of encapsulating your identity. We will see what this means in a bit.</li><li>Next, using AT1, we need the access token (AT2) to connect to our AVI&#39;s API and consume its services.</li></ol><h4>Access Token 1 (AT1)</h4><p>We must announce who we are in MS World to get this token. The WatchGuard responsible for identifying you is Entra ID. You must create an App Registration in Entra ID using the Azure portal. This is your digital identity in MS World. Do that and note its secret key.</p><pre>URL: https://login.microsoftonline.com/&lt;tenant ID&gt;/oauth2/token<br><br>// In the body of the request pass the following in x-wwwform-urlencoded <br><br>Key                  Value<br>grant_type            client_credentials<br>client_id             &lt;The id of the app you created in entra ID&gt;<br>client_secret         &lt;The secret you noted down when you generated keys for the app&gt;<br>resource              https://management.azure.com/ </pre><p>You will get a JWT access token as a response.</p><p>Next, you will use it (AT1) to generate AT2.</p><h4>Access Token 2 (AT2)</h4><p>Now that MS recognizes us, we can request an access token to Video Indexer. The token will encapsulate our access level based on the Entra app registration that we did.</p><pre>URL: https://management.azure.com/subscriptions/&lt;subs id&gt;/resourceGroups/&lt;RG name&gt;/providers/Microsoft.VideoIndexer/accounts/&lt;AVI Name&gt;/generateAccessToken?api-version=2024-01-01<br><br>// set the authorization header to bearer and pass the access token<br><br>// In the body pass (POST) the following JSON<br><br>{<br>  &quot;permissionType&quot;:&quot;Contributor&quot;,<br>    &quot;scope&quot;:&quot;Account&quot;<br>}<br><br></pre><p>You should be rewarded with an access token to the AVI resource. We will use this to access the APIs.</p><p>NOTE: We are using ARM-based AVI, and the API to generate is different. Check here:</p><p><a href="https://learn.microsoft.com/en-us/rest/api/videoindexer/generate/access-token?view=rest-videoindexer-2024-01-01&amp;tabs=HTTP#code-try-0">Generate - Access Token - REST API (Azure Azure Video Indexer)</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2c6468545721" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Clay’s run controls are four layers deep. Most people configure one.]]></title>
            <link>https://techwitharun.com/clays-run-controls-are-four-layers-deep-most-people-configure-one-e2713f08974a?source=rss-99f899e3817a------2</link>
            <guid isPermaLink="false">https://medium.com/p/e2713f08974a</guid>
            <category><![CDATA[software-engineering]]></category>
            <category><![CDATA[sales-operations]]></category>
            <category><![CDATA[gtm-engineering]]></category>
            <category><![CDATA[clay]]></category>
            <category><![CDATA[field-notes]]></category>
            <dc:creator><![CDATA[Arun Saxena]]></dc:creator>
            <pubDate>Wed, 12 Aug 2026 08:19:16 GMT</pubDate>
            <atom:updated>2026-08-12T08:19:16.634Z</atom:updated>
            <content:encoded><![CDATA[<p><em>Field Notes #1</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*FQAt8aW4Co1VvWbqogkweQ.png" /></figure><p>I changed one setting and watched six rows of enriched data disappear.</p><p>Not deleted, as far as I can tell. Just gone from the cells where they had been sitting a minute earlier — company names, employee counts, revenue bands, all of it replaced by the words “Run condition not met.”</p><p>The change log recorded what I’d done. It showed the settings diff cleanly:</p><pre>&quot;runAsButton&quot;: true<br>    ↓<br>&quot;conditionalRunFormulaText&quot;: &quot;{{Test Batch}} == true&quot;</pre><p>It said nothing about the data.</p><p>That gap is what this post is about. Not the specific bug, if it even is one. The gap between what Clay’s run-control layer actually does and what its interface tells you it’s doing.</p><h3>The Four Layers</h3><p>Clay decides whether a given cell runs by consulting four separate controls. They sit in a hierarchy, and each one can silently override everything below it.</p><p><strong>Layer 1 — the run condition. </strong>A formula on the column: only run this cell if the formula resolves true. Mine was <em>{{Test Batch}} == true</em>, gated on a checkbox column I’d added.</p><p>This is a hard block, not a queue. A row failing the condition doesn’t display “Click to run” and wait for you. It displays “Run condition not met,” and clicking it does nothing. The gate isn’t advisory.</p><p><strong>Layer 2 — column-level auto-run.</strong> Whether satisfying the condition actually triggers execution, or merely makes the row eligible.</p><p>These two are independent, and I lost twenty minutes to assuming they weren’t. With auto-run off, ticking the checkbox on a row changed its state from blocked to eligible — and then nothing happened. The row sat there, condition satisfied, waiting for a manual click. Condition is permission. Auto-run is the trigger. Configuring one does not configure the other.</p><p><strong>Layer 3 — table-level auto-run. </strong>A master switch above both, in Table Settings.</p><p>With it off, nothing fires automatically anywhere in the table regardless of how any individual column is configured. A column can have a correct condition, auto-run enabled, and every dependency satisfied, and still sit completely inert because of a toggle two screens away that you set a week ago and forgot.</p><p><strong>Layer 4 — “Keep existing results.” </strong>A checkbox beneath table-level auto-run, easy to miss, that reads:</p><blockquote>Only errored, empty, or new cells will run.</blockquote><p>This is the one nobody talks about, and it’s doing the most work. It’s the reason my toggling didn’t produce a bill. It quietly protects every cell that already holds a valid result from being re-run and re-charged when something upstream changes.</p><p>It’s a good default. It’s also invisible unless you go looking, which means most people running Clay on a client’s data have no idea whether it’s on.</p><h3>What it costs to find this out</h3><p>I ran a single enrichment — Clay’s own Enrich Company, mapped to a domain column — one row at a time, checking the usage dashboard between each.</p><ul><li>After 1 row: 1 credit, 1 action</li><li>After 2 rows: 1 credit, 2 actions</li><li>After 3 rows: 2 credits, 3 actions</li><li>After 4 rows: 2 credits, 4 actions</li></ul><p>Actions increment exactly once per row. Credits increment every second row.</p><p>The listed price for that enrichment is 0.5 credits and 1 action per row, so the actual behaviour matches the advertised rate. But if you had only run one row and looked, you’d have concluded you were charged 1 credit against a listed 0.5 and gone looking for a discrepancy that isn’t there. The dashboard rounds cumulative totals to whole numbers.</p><p>Read the total across several rows before you decide a listed rate is wrong. I nearly logged a pricing bug that doesn’t exist.</p><h3>Two things that cost nothing</h3><p><strong>A blank required input is skipped, not attempted.</strong></p><p>I cleared the domain on one row and ran it. The cell returned “Missing input” — not an error, not a failed call, not a wasted credit. Usage was unchanged. Clay checks that required inputs are present before it calls the provider.</p><p>This is the safe outcome and worth confirming rather than assuming, because the alternative — firing a provider call with a null identifier and paying for the miss — is exactly how enrichment bills get surprising at volume.</p><p>Caveat: one enrichment type, one required field, manual run mode. I haven’t tested this under scheduled runs or with enrichments that take multiple optional inputs.</p><p><strong>Adding an output field later is free.</strong></p><p>I’d configured Enrich Company to return three fields. After running it on four rows, I went back and enabled a fourth — employee count. It populated all four already-enriched rows instantly. Usage: unchanged.</p><p>So Clay caches the full provider response at enrichment time and the field toggles just control which parts of it surface as columns. It isn’t re-calling anything.</p><p>The practical consequence is that you don’t need to get your field selection right up front. Enable what you need now, add more later at no cost. That’s a genuinely useful thing to know before you plan a large run, and I’ve not seen it stated anywhere.</p><h3>What this means on someone else’s table</h3><p>Three things I’d now do differently on a client engagement:</p><p><strong>Check all four layers before answering “will this run automatically?”</strong> The honest answer requires looking at the condition, the column toggle, the table toggle, and the keep-existing-results checkbox. Any one of them can make the other three irrelevant. “I set up a run condition” tells you almost nothing about whether the pipeline is live.</p><p><strong>Don’t switch a column’s run mode on a table with data in it.</strong> Not without warning someone first. Whatever happened to my six rows — cleared, hidden, invalidated, I genuinely can’t tell from the interface — is not something you want a client watching in real time while you say “that shouldn’t have happened.”</p><p><strong>Don’t treat the change log as an audit trail for data.</strong> It records configuration changes faithfully and says nothing about their consequences. Those are different questions, and only one of them is answered.</p><h3>What I’m still unsure about</h3><p>Whether that data was actually deleted or just hidden pending re-evaluation. Toggling the condition back off didn’t restore it. The change log doesn’t distinguish. I couldn’t determine this from the interface, which is itself the finding.</p><p><em>If someone knows, I’d like to.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e2713f08974a" width="1" height="1" alt=""><hr><p><a href="https://techwitharun.com/clays-run-controls-are-four-layers-deep-most-people-configure-one-e2713f08974a">Clay’s run controls are four layers deep. Most people configure one.</a> was originally published in <a href="https://techwitharun.com/">Arun Saxena’s Tech Terrain</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[About the Author]]></title>
            <link>https://techwitharun.com/about-the-author-fd4eb8feb5f9?source=rss-99f899e3817a------2</link>
            <guid isPermaLink="false">https://medium.com/p/fd4eb8feb5f9</guid>
            <dc:creator><![CDATA[Arun Saxena]]></dc:creator>
            <pubDate>Mon, 07 Oct 2024 04:35:22 GMT</pubDate>
            <atom:updated>2026-08-12T08:36:59.803Z</atom:updated>
            <content:encoded><![CDATA[<p>Arun Saxena has been building software for thirty years and is currently CTO at a B2B SaaS company. Before that he led technology strategy and operations at a large enterprise services firm, delivering systems for Fortune 500 clients.</p><p>He writes <strong>Field Notes</strong> — a working engineer’s account of learning Clay and GTM engineering properly: what things cost to run, what breaks them, and what the documentation doesn’t say. Earlier posts here cover LangChain, Azure AI, and Python.</p><p>All experiments use public data.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=fd4eb8feb5f9" width="1" height="1" alt=""><hr><p><a href="https://techwitharun.com/about-the-author-fd4eb8feb5f9">About the Author</a> was originally published in <a href="https://techwitharun.com/">Arun Saxena’s Tech Terrain</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Essential Python — XIV Decorators with Azure Function]]></title>
            <link>https://medium.com/@asaxena_43463/azure-function-python-fc5f54f69ded?source=rss-99f899e3817a------2</link>
            <guid isPermaLink="false">https://medium.com/p/fc5f54f69ded</guid>
            <category><![CDATA[azure-functions]]></category>
            <category><![CDATA[python-decorators]]></category>
            <category><![CDATA[python]]></category>
            <dc:creator><![CDATA[Arun Saxena]]></dc:creator>
            <pubDate>Mon, 16 Sep 2024 13:54:58 GMT</pubDate>
            <atom:updated>2024-09-18T10:13:20.285Z</atom:updated>
            <content:encoded><![CDATA[<h4>Getting started WITH some understanding</h4><h4>Introduction</h4><p>Azure functions are great serverless computing resources. I have used them extensively with Javascript, TypeScript, and C#. It was always easy to get set up and running; however, working with Python recently gave me a few hiccups.</p><p>This article is for folks who use a Mac, want to use AZ functions, and need to understand what happens underneath.</p><p>Here is a step-by-step guide to setting up your dev environment and creating your first default HTTP trigger.</p><h4>Step 1: Create the function app</h4><p>Create the function app in the Azure portal. I am using:</p><ol><li>Azure Functions runtime: ~4</li><li>Consumption plan</li><li>Python version:3.11</li><li>Operating system: Linux</li></ol><h4>Step 2: Set up the code repository on GitHub</h4><p>Set up a new code repository in your GitHub account with a readme.md and .gitignore file.</p><h4>Step 2a: Create CodeSpace</h4><p>I prefer using CodeSpace as it saves me all the hassle of ensuring my environment stays nice and clean.</p><p>Head over to your new CodeSpace and check the version of Python. In my case, it was 3.12, whereas when I wrote this article, the maximum supported version was 3.11.</p><p>To check what all versions are installed:</p><pre>$ cd /usr/local/python<br>$ ls<br><br>3.11.9    3.12.1    current</pre><p>This shows that I have both 3.11 and 3.12 installed. Symlink “<em>current</em>” points to 3.12.1.</p><p>More on Python version support here:</p><p><a href="https://github.com/microsoft/vscode-azurefunctions/issues/4179">Python version 3.12 not supported for creating azure functions in vs code · Issue #4179 · microsoft/vscode-azurefunctions</a></p><h4>Step 3: Create a virtual Environment</h4><p>To make 3.11 available to our function we create a virtual environment using <em>venv</em>.</p><pre>$ pwd<br> /usr/local/python<br><br>$ cd 3.11.9/bin<br><br>$ python3 # From within this folder python3 will resolve to version 3.11 as needed.<br>Python 3.11.9 ....<br><br># this will create a virtual environment in your project folder where we will create our function app<br>$ ./python3 -m venv /workspace/&lt;your project folder&gt;/.venv<br><br>$ cd /workspace/&lt;your project folder&gt;<br><br>$ source .venv/bin/activate<br>(.venv) $ # activates the virtual environment and changes the prompt.</pre><h4>Step4: Install the Extensions</h4><ol><li>Install Azure Functions Extension</li><li>Install Python extension</li><li>Prettier — I use this for formatting</li></ol><p>AZ functions extension integrates with core tools that allow us to debug and run our functions locally.</p><p>Press F1 or Cmd+Shift+P to get the command menu. Run <em>Azure Functions: Install and update Core Tools</em>.</p><p>Next, from the command palette, run <em>Azure Functions: Create new project…</em></p><p>Select the following:</p><ol><li>Python (Programming model V2)</li><li>We already have a virtual environment, so if asked to choose the Python interpreter, please skip</li><li>Select HTTP trigger</li><li>Give some name to your trigger</li><li>Auth level — ANONYMOUS. Default is FUNCTION</li></ol><p>In the local.settings.json file, update the AzureWebJobsStorage setting as</p><pre>&quot;AzureWebJobsStorage&quot;: &quot;UseDevelopmentStorage=true&quot;,</pre><h4>Step 5: Run the function locally</h4><p>Execute the command:</p><pre>$ func start</pre><p>There are many options available with this command:</p><p><a href="https://learn.microsoft.com/en-us/azure/azure-functions/functions-core-tools-reference?tabs=v2">Azure Functions Core Tools reference</a></p><p>This will start the function locally in your codespace on GitHub. <em>Command-click</em> the link presented:</p><pre>Found Python version 3.11.9 (python3).<br><br>Azure Functions Core Tools<br>Core Tools Version:       4.0.6280 Commit hash: N/A +421f0144b42047aa289ce691dc6db4fc8b6143e6 (64-bit)<br>Function Runtime Version: 4.834.3.22875<br><br>[2024-09-16T12:46:39.349Z] Worker process started and initialized.<br><br>Functions:<br><br>        http_trigger:  http://localhost:7071/api/http_trigger<br><br>For detailed output, run func with --verbose flag.</pre><p>This will open a new tab in the same browser with the URL now pointing to your codespace on GitHub.</p><p>to the end of the URL append:</p><pre>/api/http_trigger </pre><p>Here <em>http_trigger</em> is the name of my function. The function should execute successfully.</p><h4>Step 6: Use the URL in Postman</h4><p>If you use the URL in Postman, you will get an authentication failure (assuming that your GitHub repo is private). This is because we are trying to access the repo without authentication.</p><p>It worked from the browser tab because it carried that auth cookie. We can reuse this in Postman.</p><p>From the browser tab where the function worked, copy the Copy cURL URL of the network call to GitHub (you will quickly get this in developer tools).</p><p>Paste it in Postman&#39;s address bar; it should all work well.</p><h4>Step 7: Understanding the code</h4><p>I always believe knowing a little of what goes on beneath can lead to power and a superior programming experience. If we can figure out how our code works in the Azure Functions world, it&#39;s possible we may be able to use it better or at the very least gain some understanding of better coding practices.</p><p>So lets try…</p><p>Our journey begins with understanding decorators.</p><h4>Step 7a: Basic Simple Decorator</h4><p>Here is a basic decorator that we are all familiar with:</p><pre>def decorator(func): # 2<br>    print(f&#39;ID of myFunc passed as func to decorator {id(func):#x}&#39;) # 2a<br>    def innerDecorator(*args, **kwargs):<br>        print(&quot;Inner decorator&quot;)<br>        func() #2b<br>        print(&quot;After decorating&quot;)<br><br>    print(f&quot;ID of inner decorator {id(innerDecorator):#x}&quot;) # 3<br>    return innerDecorator # 4<br><br><br>@decorator # 1<br>def myFunc(): # 5<br>    print(&quot;This is my func&quot;)<br><br>print(f&#39;ID of myFunc {id(myFunc):#x}&#39;) # 7<br><br>myFunc() # 8</pre><p>And here is the output</p><pre>ID of myFunc passed as func to decorator 0x1054ce520<br>ID of Inner Decorator 0x1054cf7e0<br>ID of myFunc 0x1054cf7e0<br>Inner decorator<br>This is my func<br>After decorating</pre><p>So what is happening here:</p><ol><li>When the interpreter comes to line #1, it executes from line #2.</li><li>Before that, it creates a pointer to object <em>myFunc</em> and passes that to the decorator in the parameter <em>func</em></li><li>The ID of this object is printed on line #2a</li><li>It then creates the function object <em>innerDecorator</em></li><li>Prints the ID of the <em>innerDecorator</em> in line #3</li><li>On line #4, it returns this <em>innerDecorator</em>.</li><li>This returned function is what gets assigned to <em>myFunc</em>. Note that from then on, <em>myFunc</em> does not point to the function you wrote but to the function returned by the decorator.</li><li>Like<em> #5, myFunc </em>points to the object returned by line<em> #4, </em>which is the<em> innerDecorator</em>.</li><li>This can be easily verified. We are printing the IDs (memory address of the objects) on lines #7 and #3. You will notice both are the same.</li><li>More importantly, notice that the ID of the <em>myFunc</em> object that was passed to the decorator and printed on line #2a is different.</li><li>Finally, we execute <em>myFunc()</em> on line #8. This is nothing by executing the function <em>innerDecorator</em>.</li></ol><p><em>But why are we doing all this?</em></p><p>We want to decorate our function myFunc with some additional functionality. This is what Azure Functions also tries to do—take our function and embellish it with more features, functionality, and attributes.</p><p>So, an object of our function is passed to the decorator. The decorator then does all the adornments it wants, including a call to our function, and finally returns a <em>callable</em>. A callable is an object that implements the <em>__call_</em> method.</p><p>However, you will notice that we have absolutely no control over how a decorator decorates our function. What if we wanted to pass it some values, expect it to respect them, and decorate accordingly? Think…</p><p>This raises some questions:</p><ol><li>What if we don&#39;t include a call to our <em>func</em> like #2b?</li><li>What if we don&#39;t return any callable as we did on #4</li><li>What if we want to pass some parameters to our decorator? This is very useful as it allows the myFunc writer to control how the decorator decorates our function.</li></ol><h4>Step 7b: Let&#39;s not call our function and see!</h4><p>Lets omit the call to our function:</p><pre>def decorator(func):<br>    print(f&quot;ID of myFunc passed as func to decorator {id(func):#x}&quot;)<br><br>    def innerDecorator(*args, **kwargs):<br>        print(&quot;Inner decorator&quot;)<br>        # func() &lt;-- we have commented this out<br>        print(&quot;After decorating&quot;)<br><br>    print(f&quot;ID of Inner Decorator {id(innerDecorator):#x}&quot;)<br>    return innerDecorator<br><br><br>@decorator<br>def myFunc():<br>    print(&quot;This is my func&quot;)<br><br><br>print(f&quot;ID of myFunc {id(myFunc):#x}&quot;)<br><br>myFunc()</pre><p>This is the output</p><pre>ID of myFunc passed as func to decorator 0x1054ce160<br>ID of Inner Decorator 0x1054cc400<br>ID of myFunc 0x1054cc400<br>Inner decorator<br>After decorating</pre><p>Notice that myFunc and none of our code is executed as we expect.</p><h4>Step 7c: Let&#39;s not return any function and see!</h4><p>Next, let&#39;s not even return a pointer to innerDecorator:</p><pre>def decorator(func):<br>    print(f&quot;ID of myFunc passed as func to decorator {id(func):#x}&quot;)<br><br>    def innerDecorator(*args, **kwargs):<br>        print(&quot;Inner decorator&quot;)<br>        # func()<br>        print(&quot;After decorating&quot;)<br><br>    print(f&quot;ID of Inner Decorator {id(innerDecorator):#x}&quot;)<br>    # return innerDecorator &lt;-- this is removed<br><br><br>@decorator<br>def myFunc():<br>    print(&quot;This is my func&quot;)<br><br><br>print(f&quot;ID of myFunc {id(myFunc):#x}&quot;)<br>print(type(myFunc)) # New line<br>myFunc()</pre><p>This results in:</p><pre>ID of myFunc passes as func to decorator 0x1054cd9e0<br>ID of Inner Decorator 0x1054ce160<br>ID of myFunc 0x10280a1a0<br>&lt;class &#39;NoneType&#39;&gt;<br>---------------------------------------------------------------------------<br>TypeError                                 Traceback (most recent call last)<br>Cell In[10], line 20<br>     15     print(&quot;This is my func&quot;)<br>     18 print(f&quot;ID of myFunc {id(myFunc):#x}&quot;)<br>---&gt; 20 myFunc()<br><br>TypeError: &#39;NoneType&#39; object is not callable</pre><p>Well, the program will run fine until we call myFunc(). This is because we returned no callable. The type of myFunc after the execution of the decorator is NoneType; therefore, when we try to invoke it, we get the error.</p><h4>Step 7d: Let&#39;s pass some parameters to the decorator</h4><p>Now that we understand some basic operations of decorators let&#39;s say we want to control how the decorator works. Let&#39;s say we want to pass a parameter name to our decorator and expect it to print a greeting with the name.</p><pre>def decorator(*args, **kwargs):  # &lt;-- lets first see what all we are passed<br>    print(&quot;DECORATOR args&quot;, args, &quot;\nkwargs&quot;, kwargs)<br>    # print(f&quot;ID of myFunc passes as func to decorator {id(func):#x}&quot;) &lt;-- removed as it wants func<br><br>    def innerDecorator(*args, **kwargs):<br>        print(&quot;INNER DECORATOR args&quot;, args, &quot;\nkwargs&quot;, kwargs) # and what all is passed to the innerDecorator<br>        # print(f&#39;Greeting from inner decorator for {name}&#39;) &lt;-- lets keep this commented until we figure whats happening.<br>        # func() # &lt;-- stays out as we dont even have it<br>        print(&quot;After decorating&quot;)<br><br>    print(f&quot;ID of Inner Decorator {id(innerDecorator):#x}&quot;)<br>    return innerDecorator # &lt;-- comes back<br><br><br>@decorator(name=&quot;Bob Dylan&quot;) # &lt;-- pass a value to the named parameter<br>def myFunc():<br>    print(&quot;This is my func&quot;)<br><br><br>print(f&quot;ID of myFunc {id(myFunc):#x}&quot;)<br><br>print(type(myFunc))<br><br>myFunc()</pre><p>The output is:</p><pre># &lt;Decorator executing and returning innerdecorator&gt;<br>DECORATOR args () <br>kwargs {&#39;name&#39;: &#39;Bob Dylan&#39;}<br>ID of Inner Decorator 0x1054cf060<br><br>#&lt;innerDecorator executing!!!&gt;<br>INNER DECORATOR args (&lt;function myFunc at 0x1055072e0&gt;,) <br>kwargs {}<br>After decorating<br><br>#&lt;The rest of the program&gt;<br>ID of myFunc 0x10280a1a0<br>&lt;class &#39;NoneType&#39;&gt;<br>---------------------------------------------------------------------------<br>TypeError                                 Traceback (most recent call last)<br>Cell In[21], line 24<br>     20 print(f&quot;ID of myFunc {id(myFunc):#x}&quot;)<br>     22 print(type(myFunc))<br>---&gt; 24 myFunc()<br><br>TypeError: &#39;NoneType&#39; object is not callable</pre><p>Lets see whats happening here:</p><ol><li>We have modified both decorator and innerDecorator by simply printing all the positional and keyword arguments.</li><li>When decorator executes we can see that there are NO POSITIONAL ARGUMENTS only the single <em>name</em> keyword argument</li><li>Next the ID inner decorator is printed as we expect</li><li>But what happens next is a surprise!</li><li>Instead of printing ID of <em>myFunc</em>, the inner decorator is being executed</li><li>The positional and keyword arguments are printed</li><li><strong>NOTE that myFunc is passed to <em>innerDecorator</em>! </strong>You can check its ID that is printed</li><li>Finally the the remaining program is executed. The ID of <em>myFunc</em> is printed.</li><li>BUT note that the ID of myFunc printed in innerDecorator and in the main program are different!</li><li>Also its type is <em>NoneType</em> which is not a callable and therefore the error.</li></ol><p>Here is what happens:</p><ol><li>When @decorator is used with parameters as (name=”Bob Dylan”) then Python executes the decorator passing it only the specified arguments</li><li>The function pointer to the decorated function is NOT passed.</li><li>Python expects the decorator to return a pointer to a function, which in out case in innerDecorator</li><li>Python then passes the pointer to the decorated function myFunc to this returned function pointer!</li><li>This can be easily verified by the fact that positional argument to innerDecorator is myFunc</li><li>Python expects that innerDecorator will also expect a pointer to a function that will then be the object that myFunc points to</li><li>But since we return no function pointer, by default it points to an object of NoneType</li><li>Since NoneType is not a callable — the error</li></ol><p>Ok then lets return a pointer to a function from innerDecorator and see:</p><pre>def decorator(name):  # PASS-1 Called with Name<br><br>    def innerDecorator(func): # PASS-2 Called with a pointer to myFunc<br>        print(f&#39;ID of func in Inner Decorator: {id(func):#x}&#39;)<br>    <br>        def returnedInner():<br>            print(f&#39;Greeting from returned inner function for {name}&#39;) <br>            func() <br>            print(&quot;After decorating&quot;)<br>            <br>    <br>        print(f&#39;ID of returned inner: {id(returnedInner):#x}&#39;)<br>        return returnedInner # PASS-2 ends by returning a pointer to inner function<br>    <br>    print(f&quot;ID of Inner Decorator {id(innerDecorator):#x}&quot;)<br>    return innerDecorator # PASS-1 ends by returning a pointer to inner function<br><br><br>@decorator(name=&quot;Bob Dylan&quot;)  # &lt;-- pass a value to the named parameter<br>def myFunc():<br>    print(&quot;This is my func&quot;)<br><br><br>print(f&quot;ID of myFunc {id(myFunc):#x}&quot;)<br><br>print(type(myFunc))<br><br>myFunc()</pre><p>The output is:</p><pre>#PASS-1 decorator executes<br>ID of Inner Decorator 0x106453a60<br><br>#PASS-2 innerDecorator executes<br>ID of func in Inner Decorator: 0x10650e5c0<br>ID of returned inner: 0x10650e8e0 # &lt;-- this object is what myFunc points to <br><br><br># Rest of the program executes<br>ID of myFunc 0x10650e8e0 # &lt;-- ID of returned inner and myFunc are same<br>&lt;class &#39;function&#39;&gt;<br><br># Now returnedInner that is pointed to by myFunc executes<br>Greeting from returned inner function for Bob Dylan<br>This is my func<br>After decorating</pre><p>So this is what really goes on:</p><ol><li>When a decorator is used by passing it some parameters or even without with <em>()</em> symbol, then it is invoked</li><li>It is not passed a pointer to our function</li><li>Python expects the decorator to pass a pointer to a function.</li><li>Python then invokes this returned function and passes it a pointer to our function</li><li>Python expects this function to in turn return a pointer to <em>another</em> function</li><li>Python assigns a reference to this finally returned function to our function name — myFunc</li><li>From this point on whenever myFunc is executed it executes the function that was finally returned!</li></ol><h4>Step 7e: Now, let&#39;s see what Azure Functions does to our function!</h4><p>Your function code should look like this:</p><pre>import azure.functions as func<br>import logging<br><br>app = func.FunctionApp(http_auth_level=func.AuthLevel.ANONYMOUS)<br><br>@app.route(route=&quot;http_trigger&quot;) # &lt;-- the decorator with parameters<br>def http_trigger(req: func.HttpRequest) -&gt; func.HttpResponse:<br>    logging.info(&#39;Python HTTP trigger function processed a request.&#39;)<br><br>    name = req.params.get(&#39;name&#39;)<br>    if not name:<br>        try:<br>            req_body = req.get_json()<br>        except ValueError:<br>            pass<br>        else:<br>            name = req_body.get(&#39;name&#39;)<br><br>    if name:<br>        return func.HttpResponse(f&quot;Hello, {name}. This HTTP triggered function executed successfully.&quot;)<br>    else:<br>        return func.HttpResponse(<br>             &quot;This HTTP triggered function executed successfully. Pass a name in the query string or in the request body for a personalized response.&quot;,<br>             status_code=200<br>        )</pre><p>We are using a function <em>route</em> of an object of type <em>FunctionApp</em> as a decorator with parameters.</p><p>Lets get into the code and see. You can access the module <em>azure.functions</em> here:</p><pre>.venv/lib/python3.11/site-packages/azure/functions</pre><p>Here, you will see the top-level <em>__init__.py</em> file. In this file, near the top, is the following code:</p><pre>from .decorators import (FunctionApp, Function, Blueprint,<br>                         DecoratorApi, DataType, AuthLevel,<br>                         Cardinality, AccessRights, HttpMethod,<br>                         AsgiFunctionApp, WsgiFunctionApp,<br>                         ExternalHttpFunctionApp, BlobSource)</pre><p>This class <em>FunctionApp</em> resides in the module <em>decorators,</em> which is available in the subfolder decorators under …/azure/functions.</p><p>In the __init__.py file of this module you will find an entry for <em>FunctionApp</em>:</p><pre>__all__ = [<br>    &#39;FunctionApp&#39;,  # &lt;--<br>    &#39;Function&#39;,<br>    &#39;FunctionRegister&#39;,<br>    &#39;DecoratorApi&#39;,<br>    &#39;TriggerApi&#39;,<br>    &#39;BindingApi&#39;,<br>    &#39;SettingsApi&#39;,<br>    &#39;Blueprint&#39;,<br>    &#39;ExternalHttpFunctionApp&#39;,<br>    &#39;AsgiFunctionApp&#39;,<br>    &#39;WsgiFunctionApp&#39;,<br>    &#39;DataType&#39;,<br>    &#39;AuthLevel&#39;,<br>    &#39;Cardinality&#39;,<br>    &#39;AccessRights&#39;,<br>    &#39;HttpMethod&#39;,<br>    &#39;BlobSource&#39;<br>]</pre><p>Now, head over to the file <em>function_app.py</em> that defines this class. You will find that the constructor of the class is defined as:</p><pre>def __init__(self,<br>                 http_auth_level: Union[AuthLevel, str] = AuthLevel.FUNCTION):</pre><p>You will notice that <em>http_auth_level</em> is what we are passing to the constructor in our code. This determines what keys, if any, that need to be passed to access this function:</p><p>Also notice that the class derives from a bunch of other classes:</p><pre>class FunctionApp(FunctionRegister, TriggerApi, BindingApi, SettingsApi):</pre><p>Here:</p><ol><li>The FunctionRegister class (interface as it derives from ABC) is for declaring the TOP LEVEL function app class, which will be indexed by Python runtime.</li><li>TriggerAPI is an interface containing trigger decorators.</li><li>BindingApi interface to for binding decorators</li><li>SettingApi interface to extend the decorators for function settings.</li></ol><p>Of interest here is the TriggerAPI class. Check its definition:</p><pre>class TriggerApi(DecoratorApi, ABC):<br>    &quot;&quot;&quot;Interface to extend for using existing trigger decorator functions.&quot;&quot;&quot;</pre><p>This class is available in the same file and defines a decorator <em>route</em> that we are using.</p><pre>def route(self,<br>              route: Optional[str] = None, #&lt;-- the route param we passed goes here<br>              trigger_arg_name: str = &#39;req&#39;,<br>              binding_arg_name: str = &#39;$return&#39;,<br>              methods: Optional[<br>                  Union[Iterable[str], Iterable[HttpMethod]]] = None,<br>              auth_level: Optional[Union[AuthLevel, str]] = None,<br>              trigger_extra_fields: Dict[str, Any] = {},<br>              binding_extra_fields: Dict[str, Any] = {}<br>              ) -&gt; Callable[..., Any]: #&lt;-- it needs to return a callable that will be passed our function<br></pre><p>We also know that a decorated function is like a wrapper around a decorated function, passing the function to another function.</p><p>This is the body of function route:</p><pre>        @self._configure_function_builder<br>        def wrap(fb):<br>            def decorator():<br>                fb.add_trigger(trigger=HttpTrigger(<br>                    name=trigger_arg_name,<br>                    methods=parse_iterable_param_to_enums(methods, HttpMethod),<br>                    auth_level=parse_singular_param_to_enum(auth_level,<br>                                                            AuthLevel),<br>                    route=route, **trigger_extra_fields))<br>                fb.add_binding(binding=HttpOutput(<br>                    name=binding_arg_name, **binding_extra_fields))<br>                return fb<br><br>            return decorator()<br><br>        return wrap #&lt;-- the function pointer which will be passed our function</pre><p>The function _configure_function_builder is defined in class <em>DecoratorAPI</em> that is an ancestor of <em>TriggerAPI</em>.</p><p>Python takes the function <em>wrap</em> and passes it to the decorator.</p><pre>def _configure_function_builder(self, wrap) -&gt; Callable[..., Any]:<br>        &quot;&quot;&quot;Decorator function on user defined function to create and return<br>         :class:`FunctionBuilder` object from :class:`Callable` func.<br>        &quot;&quot;&quot;<br><br>        def decorator(func):<br>            fb = self._validate_type(func)<br>            self._function_builders.append(fb)<br>            return wrap(fb)<br><br>        return decorator</pre><p>This returns a decorator that performs some tasks and eventually returns wrap.</p><p>So what happens is:</p><ol><li>The function wrap gets a pointer to our function.</li><li>This wrap is passed to _configure_function_builder as its decorator</li><li>Python expects it to return a function pointer which will then be the object that wrap will point to</li><li>_configure_function_builder returns decorator which is what wrap now points to</li><li>It gets a pointer to our function, creates a function builder from it and then returns a function pointer wrap with this builder as parameter.</li></ol><p>Now that we are done processing the decorator of wrap, the subsequent flow is:</p><ol><li>Wrap creates a decorator (this is the returnedInner in our example earlier)</li><li>Wrap returns a pointer to this decorator</li><li>Now when AzureFuntime executes our function — this decorator is what will execute.</li><li>It adds HttpTrigger and HttpBinding objects to the function and then returns a pointer to our function</li><li>So when our function executes these objects fully initialised are available to us.</li></ol><p>I know that was fairly more complex than our simple examples. But I will encourage you to build on our simple example for a case of decorator within a decorator to verify how all this works.</p><p>This brings us to an end. I hope you had a good time reading :)</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=fc5f54f69ded" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[LangChain — IV: Embeddings]]></title>
            <link>https://techwitharun.com/embeddings-47f827bbcf57?source=rss-99f899e3817a------2</link>
            <guid isPermaLink="false">https://medium.com/p/47f827bbcf57</guid>
            <category><![CDATA[langchain]]></category>
            <category><![CDATA[embedding]]></category>
            <category><![CDATA[skip-gram]]></category>
            <category><![CDATA[bag-of-words-nlp]]></category>
            <category><![CDATA[word2vec]]></category>
            <dc:creator><![CDATA[Arun Saxena]]></dc:creator>
            <pubDate>Mon, 26 Aug 2024 07:46:43 GMT</pubDate>
            <atom:updated>2024-08-28T06:53:41.053Z</atom:updated>
            <content:encoded><![CDATA[<p>Conceptual background | algorithms | Sample code | LangChain Classes</p><h3>Introduction</h3><p>Word embeddings are dense vector representations of words, sentences, or text in a continuous space. We can also represent images with vectors.</p><p>The vectors are created to capture the semantic meaning of the words or sentences, as the case may be.</p><p>Once the model is created and validated with test data, we can use it for classification and prediction. Tasks such as the following are often achieved using such embedding models:</p><ol><li>Machine translation — prediction</li><li>Sentiment analysis — classification</li><li>Text classification — classification</li><li>Search using semantic similarity — classification</li></ol><p>Dense vectors are just a set of numbers that represent probability distribution along various dimensions. Let&#39;s understand this better with the help of visuals.</p><h3>Envisioning vectorization of an image</h3><p>Let&#39;s create attribute vectors for two very commonly appearing objects: Orange and Sunflower. We have selected 10 attributes and assigned them values on a scale of 1 to 10.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*MvmAP_1d-6ALVXcMwikrRQ.png" /><figcaption>Attribute values for Sunflower and Orange</figcaption></figure><p>Plotting a radar chart of these values shows their attribute distribution along the 10 dimensions flattened on a 2D surface. This representation makes it easy to visualize their relative smears.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*xNOGMlmYj1ZCELgULIBg3Q.jpeg" /><figcaption>Attribute (dimension) wise distribution for Sunflower and Orange</figcaption></figure><p>Each orange and sunflower can be plotted as a single line vector in a 10D space (which is impossible for me to imagine). A matrix of numbers, such as the attribute values, represents a vector. The dot product of vectors gives the extent of similarity between the vectors. Embedding models use this proximity measure to generate semantic search results or classify objects.</p><p>We can also imagine this proximity or similarity as the radar graphs’ overlap percentage.</p><h3>Use the Model to generate embeddings.</h3><p>Let&#39;s see this in action with LangChain. As you will expect, LangChain provides wrappers to a number of embedding models, so we have a consistent interface for integrating with any of them.</p><pre>import os<br>import configparser<br>from langchain_cohere import CohereEmbeddings<br>from langchain_core.messages import HumanMessage, SystemMessage<br><br># set up config parser<br>config = configparser.ConfigParser()<br>config.read(&quot;../config.ini&quot;)  # holds secrets and keys<br># config.read(&quot;config.ini&quot;)<br><br># load the Default section.<br>default = config[&quot;DEFAULT&quot;]<br><br><br># set the environment variables needed by LangSmith<br>os.environ[&quot;LANGCHAIN_TRACING_V2&quot;] = default.get(&quot;LANGCHAIN_TRACING_V2&quot;)<br>os.environ[&quot;LANGCHAIN_API_KEY&quot;] = default.get(&quot;LANGCHAIN_API_KEY&quot;)<br>os.environ[&quot;LANGCHAIN_ENDPOINT&quot;] = default.get(&quot;LANGCHAIN_ENDPOINT&quot;)<br>os.environ[&quot;LANGCHAIN_PROJECT&quot;] = default.get(&quot;LANGCHAIN_PROJECT&quot;)<br><br># load Cohere config<br>cohereConfig = config[&quot;cohere&quot;]<br><br>os.environ[&quot;COHERE_API_KEY&quot;] = cohereConfig.get(&quot;COHERE_API_KEY&quot;)<br><br># select the model.<br>embedding = CohereEmbeddings(model=&quot;embed-english-v3.0&quot;)<br><br>#generate embedding for &quot;king&quot;<br>e = embedding.embed([&quot;king&quot;], input_type=&quot;search_query&quot;)<br>print(e)<br></pre><p>The output is:</p><pre>[[0.024230957, -0.024551392, 0.02520752, -0.040008545, -0.06311035, <br>...,<br> 0.03656006, -0.0013208389, 0.04586792]]</pre><p>This is a 384 dimension embedding of the word King.</p><h3>Predicting words with embeddings</h3><p>LLMs use embeddings to generate the most probable words given a sequence of words.</p><p>For example, <em>What a wonderful _____. </em>The model will output a list of words that can follow with a probability score. In this case, the model may generate <em>a day,</em> <em>dress,</em> etc. The model may further fine-tune the probability distribution by considering a larger context. For example, if the words before this phrase discussed the weather, the word <em>day</em> will have a higher probability.</p><p>The embedding model must be trained with data to generate these probability distributions. Once trained, it is ready to make predictions. The predictions and their quality depend on</p><ol><li>The dataset used</li><li>The extent of training done.</li><li>The algorithm used parameters, etc.</li></ol><p>Language models can be trained on running text, which we have an abundance of, like Wikipedia, all the books, etc. This is a huge advantage of language models. Further, since the learning is unsupervised, it&#39;s scalable. All of this results in highly robust embedding models.</p><h3>One Hot Vectors</h3><p>One hot vector is a scheme for encoding vectors for words. Suppose our vocabulary consists of <em>N</em> words; then<em>, </em>we define a<em> length-N vector </em>with an index from<em> 0 to N-1 </em>for each word. To obtain a one-hot vector representation for the word, we set each element in the vector to 0 except one. In this way, each word is represented by a vector and can be used in neural networks.</p><pre>WORD              VECTOR <br>Bird              [0,0,0,0,1,0,0,........0]<br>Banana            [0,1,0,0,0,0,0,0,0.....0]<br>... and so on</pre><p>It&#39;s easy to see that this schema is relatively inefficient:</p><ol><li>For any meaningful size vocabulary containing 10s of thousands of words, the vectors will be extremely long and inefficient</li><li>No contextual information is stored within the vector. Thus, by looking at a word, we can never find words similar to it.</li></ol><h3>Word2Vec</h3><p>This tool addresses the issues with one-hot vectors. It maps each word to a <em>fixed-length</em> vector. These vectors also consider the context and similarity between words in a manner we saw earlier. There are two main models in this tool:</p><ol><li>Continuous bag of words</li><li>Skip-gram model</li></ol><p>Both obtain meaningful representations of words, considering some of the surrounding words. Both of them are unsupervised models.</p><h3>Skip-gram model</h3><p>The skip-gram model starts with the target word and then gives the probability distribution of 2 words, both the left and right of the target word.</p><p>For example, “<em>The chef loves cooking pasta”</em>. This model selects loves as the center word and gives the conditional probability for generating context words “<em>The chef”</em> and “<em>cooking pasta</em>.”</p><p>The context words are generated independently of each other. Let&#39;s envision this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/720/1*kXHisTDu5D9SZx2MWYEKZQ.gif" /><figcaption>Skip-gram model with independent probability for two words before and after.</figcaption></figure><p>In the skip-gram model, each word in the vocabulary has 2 vectors associated with it.</p><h3>Continuous Bag of Words model</h3><p>This model is similar to the Skip-Gram model with one key difference. While Skip-Gram determines the probability distribution of context words given a target word, the CBOW model determines the probability distribution of target words given the context.</p><p>The same sentence, “The chef loves cooking pasta,” in the CBOW model will predict the probability of the target word:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/720/1*OtYfNAjvqn04hfV3KgUnxg.gif" /><figcaption>The CBOW model is used to predict the probability of a target word.</figcaption></figure><p>The training process for these models results in a matrix containing an entry for each word in the model and two vectors for each word (target and context).</p><h3>The training process</h3><p>Let us briefly look at the training process. We begin with an untrained model that is sure to give random predictions. The prediction is then compared with the correct output, and the difference (loss) is fed into the model to update its weights and biases. This process continues for all the samples in our data set, which marks the end of an epoch. Multiple epochs are needed to complete the training of a model.</p><p>When we say embeddings matrix, we mean the embedding vector of each word in our vocabulary. Let&#39;s assume that our vocabulary is just:</p><p>“The chef loves cooking pasta.”</p><p>Our embedding matrix consists of V=5 (V for vocabulary) embedding vectors. Each vector will have D = 6 (say) values (dimensions).</p><pre>Embedding Matrix VxD<br><br>Word(V)    Vector(V)<br>The        [.2, .6, -.4, -.9, .7]<br>Chef       [.2, .5, .1, -.11, -.91]<br>Loves      [-.63, .52, .31, -.91, .01.]<br>Cooking    [...]<br>Pasta      [...]</pre><p>In an untrained model, these values are provided randomly, and the training process iteratively updates them.</p><p>Let&#39;s assume we start the training process and supply the untrained model with the word “loves,” we want it to predict “chef” as one of the context words.</p><p>Embedding models are neural nets with multiple layers. The first layer is the embedding matrix, followed by other neural layers such as dense, convolution, or recurrent.</p><p>In the case of Word2Vec, apart from the embedding layer, there may be an optional projection layer followed by the output layer.</p><p>When the model receives an input word, it picks the embedding and loads it to the first layer. The output is then computed, and the loss function is fed back to the model to adjust the word&#39;s weights (embedding).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/720/1*rJIn1KzJ_--_ddAHTvC_6w.gif" /><figcaption>Embedding model training process.</figcaption></figure><p>Generating the output prediction is computationally rather intensive, as a probability must be generated for all the words in the vocabulary. Any fairly large vocabulary will mean a lot of computation for each sample. In production, there have been many optimizations to this process to make it efficient. But for illustrative purposes, this should suffice.</p><h3>BERT</h3><pre># We are Extending Embeddings Class of langchain and implementing<br><br>import os<br>import configparser<br>import torch<br>from transformers import BertModel, BertTokenizer<br>from langchain.embeddings.base import Embeddings<br>from langchain.evaluation import load_evaluator, EmbeddingDistance<br><br># Set up config parser<br>config = configparser.ConfigParser()<br>config.read(&quot;../config.ini&quot;)  # Holds secrets and keys<br><br># Load the Default section.<br>default = config[&quot;DEFAULT&quot;]<br><br># Set the environment variables needed by LangSmith (if required)<br>os.environ[&quot;LANGCHAIN_API_KEY&quot;] = default.get(&quot;LANGCHAIN_API_KEY&quot;)<br>os.environ[&quot;LANGCHAIN_ENDPOINT&quot;] = default.get(&quot;LANGCHAIN_ENDPOINT&quot;)<br>os.environ[&quot;LANGCHAIN_PROJECT&quot;] = default.get(&quot;LANGCHAIN_PROJECT&quot;)<br><br># Modify BERTEmbeddings class to inherit from LangChain&#39;s Embeddings base class<br>class BERTEmbeddings(Embeddings):<br>    def __init__(self, model_name: str = &quot;bert-base-uncased&quot;):<br>        self.tokenizer = BertTokenizer.from_pretrained(model_name)<br>        self.model = BertModel.from_pretrained(model_name)<br><br>    def embed_documents(self, texts, input_type=&quot;search_query&quot;):<br>        # Tokenize input text<br>        inputs = self.tokenizer(texts, return_tensors=&#39;pt&#39;, padding=True, truncation=True, max_length=512)<br><br>        # Generate embeddings<br>        with torch.no_grad():<br>            outputs = self.model(**inputs)<br><br>        # Pooling by taking the mean of the last hidden state<br>        embeddings = outputs.last_hidden_state.mean(dim=1)<br>        return embeddings.tolist()<br><br>    def embed_query(self, text: str) -&gt; list:<br>        return self.embed_documents([text])[0]<br><br># Instantiate the BERT embedding class<br>embedding_bert = BERTEmbeddings(model_name=&quot;bert-base-uncased&quot;)<br><br># Load the evaluator<br>evaluator = load_evaluator(<br>    &quot;pairwise_embedding_distance&quot;, <br>    embeddings=embedding_bert, <br>    distance_metric=EmbeddingDistance.EUCLIDEAN<br>)<br><br># Print evaluator details<br>print(evaluator.__repr__())<br><br># Evaluate string pairs<br>result_1 = evaluator.evaluate_string_pairs(prediction=&quot;man&quot;, prediction_b=&quot;pencil&quot;)<br>result_2 = evaluator.evaluate_string_pairs(prediction=&quot;marker&quot;, prediction_b=&quot;pencil&quot;)<br><br>print(result_1)</pre><h3>Plotting vectors and comparing their similarity</h3><p>Let&#39;s see how we can use LangChain to produce embeddings.</p><pre># We need this visualization<br>! pip install cohere altair -q</pre><p>Next, we import the required libraries:</p><pre>import cohere # the model<br>import pandas as pd<br>import numpy as np<br>import altair as alt<br>from sklearn.decomposition import PCA<br>from sklearn.metrics.pairwise import cosine_similarity<br>from sklearn.cluster import KMeans<br><br>co = cohere.Client(&quot;YOUR_API_KEY&quot;)<br><br># Ensure you have a file with words accessible.<br>df = pd.read_csv(&#39;../words.csv&#39;,names=[&quot;word&quot;]) <br><br># my words file has the following words<br>Man<br>son<br>Father<br>Brother<br>Orange<br>Banana<br>Mother<br>Woman<br>Child<br>India<br>China<br>Uncle<br>Aunt<br>pune<br>mumbai<br>jalna<br>USA</pre><pre># Get text embeddings. We use cohere embedding model<br>def get_embeddings(texts, model=&#39;embed-english-v3.0&#39;, input_type=&quot;search_document&quot;):<br>    output = co.embed(<br>        model=model,<br>        input_type=input_type,<br>        texts=texts)<br>    return output.embeddings<br><br># Embed the dataset<br>df[&#39;query_embeds&#39;] = get_embeddings(df[&#39;word&#39;].tolist())<br>df.head()<br>print(df)</pre><pre># Function to return the principal components. arr = array of embedding, n = number of principle component<br>def get_pc(arr, n):<br>    pca = PCA(n_components=n)<br>    embeds_transform = pca.fit_transform(arr)<br>    return embeds_transform<br><br># Reduce embeddings to 10 principal components to aid visualization<br>print(df[&#39;query_embeds&#39;],&quot;\n&quot;)<br><br>embeds = np.array(df[&#39;query_embeds&#39;].tolist())<br>print(embeds,&quot;\n&quot;)<br><br>embeds_pc = get_pc(embeds, 15)<br>print(&quot;number of dimensions after reduction :&quot;,len(embeds_pc[0]))</pre><pre># Set sample size to visualize<br>sample = 15<br><br># Reshape the data for visualization purposes<br>source = pd.DataFrame(embeds_pc)[:sample]<br>source = pd.concat([source,df[&#39;word&#39;]], axis=1)<br>source = source.melt(id_vars=[&#39;word&#39;])</pre><pre># Configure the plot<br>chart = alt.Chart(source).mark_rect().encode(<br>    x=alt.X(&#39;variable:N&#39;, title=&quot;Embedding&quot;,sort=None),<br>    y=alt.Y(&#39;word:N&#39;, title=&#39;&#39;,sort=None,axis=alt.Axis(labelLimit=500)),<br>    color=alt.Color(&#39;value:Q&#39;, title=&quot;Value&quot;, scale=alt.Scale(<br>                range=[&quot;#FFFFFF&quot;, &quot;#000000&quot;]))<br>)</pre><pre>result = chart.configure(background=&#39;#ffffff&#39;<br>        ).properties(<br>        width=1200,<br>        height=400,<br>        title=&#39;Embeddings with 10 dimensions&#39;<br>       ).configure_axis(<br>      labelFontSize=15,<br>      titleFontSize=12)<br><br># Show the plot<br>result</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*1yQxgRkBj3jp9XZUZTLHSw.png" /><figcaption>Greyscale representation of word vectors</figcaption></figure><p>You will notice a level of similarity in the more significant dimensions that come to the left. Note that Orange and Banana are similar in the first dimension and, similarly, the four male words before that. Next, let&#39;s try to plot these vectors in 2D space and see if similar words come close to each other.</p><pre><br># Function to generate the 2D plot<br>def generate_chart(df,xcol,ycol,lbl=&#39;on&#39;,color=&#39;basic&#39;,title=&#39;&#39;):<br>    chart = alt.Chart(df).mark_circle(size=500).encode(<br>        x=<br>        alt.X(xcol,<br>              scale=alt.Scale(zero=False),<br>              axis=alt.Axis(labels=False, ticks=False, domain=False)<br>             ),<br>        y=<br>        alt.Y(ycol,<br>              scale=alt.Scale(zero=False),<br>              axis=alt.Axis(labels=False, ticks=False, domain=False)<br>             ),<br>        color= alt.value(&#39;#333293&#39;) if color == &#39;basic&#39; else color,<br>        tooltip=[&#39;word&#39;]<br>    )<br>    <br>    if lbl == &#39;on&#39;:<br>        text = chart.mark_text(align=&#39;left&#39;, baseline=&#39;middle&#39;,dx=15, size=13,color=&#39;black&#39;).encode(text=&#39;word&#39;, color= alt.value(&#39;black&#39;))<br>    else:<br>        text = chart.mark_text(align=&#39;left&#39;, baseline=&#39;middle&#39;,dx=10).encode()<br>        <br>    result = (chart + text).configure(background=&quot;#FDF7F0&quot;).properties(<br>        width=800,<br>        height=500,<br>        title=title<br>    ).configure_legend(orient=&#39;bottom&#39;, titleFontSize=18,labelFontSize=18)<br>    <br>    return result</pre><pre># Reduce embeddings to 2 principal components to aid visualization<br>embeds_pc2 = get_pc(embeds, 2)<br><br># Add the principal components to dataframe<br>df_pc2 = pd.concat([df, pd.DataFrame(embeds_pc2)], axis=1)<br><br># Plot the 2D embeddings on a chart<br>df_pc2.columns = df_pc2.columns.astype(str)<br><br>generate_chart(df_pc2.iloc[:15],&#39;0&#39;,&#39;1&#39;,title=&#39;2D Embeddings&#39;)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/829/1*zS9Kb6e0DZsw13HsB5pGbA.png" /></figure><p>Note that Orange and Banana are close. For some reason, Man, Woman, and child are together, and father and mother come together.</p><h3>Envision, add &amp; subtract vectors.</h3><p>An arrow represents vectors. The length of the arrow denotes its magnitude, and its direction is the direction of the vector.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/690/1*NqJqsRixezrzWBotxgyERw.gif" /><figcaption>We are adding and subtracting vectors.</figcaption></figure><p>Let&#39;s assume that vector A is “Dog” and vector B is “Tall.” Then, A + B will mean a tall dog, perhaps a Great Dane. On the other hand, B will mean “short,” so A-B is a short dog, like a Spitz.</p><p>Let&#39;s see this in action. In this demo, we plot the vectors of a few words and then perform addition and subtraction on them.</p><p>We will show that the “closeness” or “distance” between the words “more” and “less” is similar to “high” and “low.”</p><p>If that is true, then subtracting “less” from “more” should have the same magnitude as subtracting “low” from “high.”</p><pre>import numpy as np<br>from sklearn.metrics.pairwise import cosine_similarity<br>import matplotlib.pyplot as plt<br>from sklearn.decomposition import PCA<br><br><br># Load GloVe embeddings into a dictionary<br>embeddings = {}<br>with open(&#39;glove.6B.100d.txt&#39;, &#39;r&#39;, encoding=&#39;utf-8&#39;) as f:<br>    for line in f:<br>        values = line.split()<br>        word = values[0]<br>        vector = np.asarray(values[1:], dtype=&#39;float32&#39;)<br>        embeddings[word] = vector<br><br></pre><pre># we will use it later to calculate the cosine similarity between vectors<br>def cosine_similarity(vec1, vec2):<br>    # Compute the dot product<br>    dot_product = np.dot(vec1, vec2)<br>    <br>    # Compute the magnitudes (norms) of the vectors<br>    norm_vec1 = np.linalg.norm(vec1)<br>    norm_vec2 = np.linalg.norm(vec2)<br>    <br>    # Compute the cosine similarity<br>    similarity = dot_product / (norm_vec1 * norm_vec2)<br>    return similarity<br></pre><pre># It will take list of 4 words, and consider vector between first 2 word and last 2.<br>def custom_cosine_similarity_finder(words):<br>    vector1 = embeddings[words[0]] - embeddings[words[1]]<br>    vector2 = embeddings[words[2]] - embeddings[words[3]]<br>    similarity = cosine_similarity(vector1, vector2)<br>    print(f&quot;Cosine similarity: \n &#39;{words[0]}-{words[1]}&#39; and &#39;{words[2]}-{words[3]}&#39;: {similarity}&quot;)</pre><pre>def grap_plot_for_vectors(words):<br>    # PCA to reduce dimensions to 2D for visualization<br>    vectors = np.array([embeddings[word] for word in words])<br>    pca = PCA(n_components=2)<br>    reduced_vectors = pca.fit_transform(vectors)<br><br>    # Calculate the direction of the vectors in 2D<br>    vector_1_2d = reduced_vectors[0] - reduced_vectors[1]  # japanese - japan<br>    vector_2_2d = reduced_vectors[2] - reduced_vectors[3]  # chinese - china<br><br>    # Plotting the words and vectors<br>    plt.figure(figsize=(8, 6))<br>    plt.scatter(reduced_vectors[:, 0], reduced_vectors[:, 1])<br><br>    for i, word in enumerate(words):<br>        plt.text(reduced_vectors[i, 0] + 0.01, reduced_vectors[i, 1] + 0.01, word, fontsize=12)<br><br><br>    # Draw arrows (vectors) between word pairs<br>    plt.arrow(reduced_vectors[1, 0], reduced_vectors[1, 1], vector_1_2d[0], vector_1_2d[1],<br>            head_width=0.05, head_length=0.1, fc=&#39;blue&#39;, ec=&#39;blue&#39;, label=f&quot;{words[0]}-&gt;{words[1]}&quot;)<br>    plt.arrow(reduced_vectors[3, 0], reduced_vectors[3, 1], vector_2_2d[0], vector_2_2d[1],<br>            head_width=0.05, head_length=0.1, fc=&#39;green&#39;, ec=&#39;green&#39;, label=f&#39;{words[2]} -&gt; {words[2]}&#39;)<br><br>    # Draw lines to show that the vectors are parallel<br>    plt.plot([reduced_vectors[1, 0], reduced_vectors[0, 0]], [reduced_vectors[1, 1], reduced_vectors[0, 1]], &#39;b--&#39;, label=f&quot;{words[0]}-{words[1]}&quot;)<br>    plt.plot([reduced_vectors[3, 0], reduced_vectors[2, 0]], [reduced_vectors[3, 1], reduced_vectors[2, 1]], &#39;g--&#39;, label=f&#39;{words[2]} -&gt; {words[2]}&#39;)<br><br>    plt.legend()<br>    plt.xlabel(&#39;PCA Dimension 1&#39;)<br>    plt.ylabel(&#39;PCA Dimension 2&#39;)<br>  <br>    plt.grid(True)<br>    plt.show()</pre><pre>words = [&#39;king&#39;, &#39;man&#39;, &#39;queen&#39;, &#39;woman&#39;]<br>grap_plot_for_vectors(words)<br>custom_cosine_similarity_finder(words)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/736/1*fvN1WCrfXipWZFx54Fjwuw.png" /></figure><p><em>Cosine similarity: ‘king-man’ and ‘queen-woman’: 0.758131206035614</em></p><pre>words = [&#39;dark&#39;, &#39;darker&#39;, &#39;bright&#39;, &#39;brighter&#39;]<br>grap_plot_for_vectors(words)<br>custom_cosine_similarity_finder(words)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/729/1*6JUrl8I1v9MqWXFdLqRW1w.png" /></figure><p><em>Cosine similarity: ‘dark-darker’ and ‘bright-brighter’: 0.6938932538032532</em></p><p>Now consider:</p><pre>DARK - DARKER + BRIGHTER = BRIGHT</pre><p>This equation is more or less evident from the diagram. There is a loss of accuracy because of reducing dimensions and flatenning to 2D.</p><h3>LangChain support for embeddings</h3><p>Let&#39;s use LangChain to generate embedding and check their similarity using the API provided.</p><pre>import os<br>import configparser<br>from langchain_cohere import CohereEmbeddings<br>from langchain_core.messages import HumanMessage, SystemMessage<br><br># set up config parser<br><br>config = configparser.ConfigParser()<br>config.read(&quot;../config.ini&quot;)  # holds secrets and keys<br># config.read(&quot;config.ini&quot;)<br><br># load the Default section.<br>default = config[&quot;DEFAULT&quot;]<br><br><br># set the environment variables needed by LangSmith<br># os.environ[&quot;LANGCHAIN_TRACING_V2&quot;] = default.get(&quot;LANGCHAIN_TRACING_V2&quot;)<br>os.environ[&quot;LANGCHAIN_API_KEY&quot;] = default.get(&quot;LANGCHAIN_API_KEY&quot;)<br>os.environ[&quot;LANGCHAIN_ENDPOINT&quot;] = default.get(&quot;LANGCHAIN_ENDPOINT&quot;)<br>os.environ[&quot;LANGCHAIN_PROJECT&quot;] = default.get(&quot;LANGCHAIN_PROJECT&quot;)<br><br># load Cohere config<br>cohereConfig = config[&quot;cohere&quot;]<br><br>os.environ[&quot;COHERE_API_KEY&quot;] = cohereConfig.get(&quot;COHERE_API_KEY&quot;)<br><br># select the model.<br>embedding = CohereEmbeddings(model=&quot;embed-english-v3.0&quot;)<br><br>e = embedding.embed([&quot;king&quot;], input_type=&quot;search_query&quot;)<br>print(e)<br><br>print(&#39;setup done successfully&#39;)</pre><pre>from langchain.evaluation import load_evaluator, EmbeddingDistance<br>list (EmbeddingDistance)<br><br>evaluator = load_evaluator(&quot;pairwise_embedding_distance&quot;,  embeddings=embedding, distance_metric=EmbeddingDistance.EUCLIDEAN)<br>print(evaluator.__repr__())</pre><pre>print(&quot;eraser-pencil&quot;,evaluator.evaluate_string_pairs(prediction=&quot;eraser&quot;, prediction_b=&quot;pencil&quot;))<br>print(&quot;marker-pencil&quot;,evaluator.evaluate_string_pairs(prediction=&quot;marker&quot;, prediction_b=&quot;pencil&quot;))<br><br># Lower the value, closer are the terms.</pre><p><em>eraser-pencil {‘score’: </em>6.96883508453162<em>} <br>marker-pencil {‘score’: </em>3.8628910432419983<em>}</em></p><p>Since marker and pencil are both writing instruments, the score is lower which means they are semantically closer.</p><p>We have so far dealt with individual words. But we can generate embeddings for sentences or paragraphs also. The process involves getting the vectors of individual words and then using them to get a semantic distribution of the sentence. Lets see this with an example:</p><pre>print(&quot;s1 - Tiger is king of the jungle\ns2 - Tiger is a very dengerous animal \n\t&quot;,<br>    evaluator.evaluate_string_pairs(prediction=&quot;Tiger is king of the jungle.&quot;, <br>    prediction_b=&quot;Tiger is a very dengerous animal.&quot;))<br><br>print(&quot;\ns3 - Metro trains run in city\ns4 - Tiger is king of the jungle\n\t&quot;,<br>    evaluator.evaluate_string_pairs(prediction=&quot;Metro trains run in city&quot;, <br>    prediction_b=&quot;Tiger is king of the jungle&quot;))<br></pre><p>s1 — Tiger is king of the jungle <br>s2 — Tiger is a very dengerous animal <br>s1-s2 : {‘score’: 0.8319432460785122}</p><p>s3 — Metro trains run in city <br>s4 — Tiger is king of the jungle <br>s3-s4 : {‘score’: 1.1784714313923719}</p><p>The sentences S1 and S2 are semantically much closer than S3 and S4 and this is evident in the proximity score where a lower score means higher co-relation.</p><p>LangChain provides an abstract class</p><h4>langchain_core.embeddings.embeddings.Embeddings</h4><p>We derive from this interface when building concrete classes for the various embedding models supported by LangChain.</p><p>There are 4 methods in this interface:</p><ol><li><strong>aembed_documents:</strong> Async embedding of a list of documents.</li><li><strong>embed_documents:</strong> Embed a list of documents in blocking mode.</li><li><strong>aembed_query:</strong> Async embed query text</li><li><strong>embed_query:</strong> Emed query text</li></ol><p>LangChain also supports caching results from models, this saves you the cost associated with every round trip to the model.</p><p>The module:</p><h4>langchain.embedding.cache</h4><p>It is a wrapper around embedded and provides embedding in a key-value store. The text is hashed and the hash is used as the key.</p><p>In this module, there is a class:</p><h4>CacheBackedEmbeddings(Embeddings)</h4><p>This abstract class works with any store that implements this interface for keys and values of type string and List[float], respectively.</p><h3>References</h3><h4>Embeddings basics</h4><p><a href="https://jalammar.github.io/illustrated-word2vec/">The Illustrated Word2vec</a></p><h4>LangChain Embedding</h4><p><a href="https://python.langchain.com/v0.2/api_reference/core/embeddings/langchain_core.embeddings.embeddings.Embeddings.html#langchain_core.embeddings.embeddings.Embeddings">Embeddings - 🦜🔗 LangChain documentation</a></p><h4>Caching Embeddings</h4><p><a href="https://python.langchain.com/v0.2/api_reference/_modules/langchain/embeddings/cache.html#CacheBackedEmbeddings">langchain.embeddings.cache - 🦜🔗 LangChain documentation</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=47f827bbcf57" width="1" height="1" alt=""><hr><p><a href="https://techwitharun.com/embeddings-47f827bbcf57">LangChain — IV: Embeddings</a> was originally published in <a href="https://techwitharun.com/">Arun Saxena’s Tech Terrain</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[LangChain — III: Prompts]]></title>
            <link>https://techwitharun.com/langchain-iii-prompts-2df826c0ec3d?source=rss-99f899e3817a------2</link>
            <guid isPermaLink="false">https://medium.com/p/2df826c0ec3d</guid>
            <category><![CDATA[langchain-chain]]></category>
            <category><![CDATA[langchain-messages]]></category>
            <category><![CDATA[langchain-agents]]></category>
            <category><![CDATA[langchain]]></category>
            <dc:creator><![CDATA[Arun Saxena]]></dc:creator>
            <pubDate>Tue, 13 Aug 2024 12:56:42 GMT</pubDate>
            <atom:updated>2024-08-19T08:41:56.003Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/720/1*aoixEaUlzf9zTqvUgbADIg.gif" /><figcaption>LangChain Prompts and their structure</figcaption></figure><h3>LangChain — III: Prompts</h3><h4>Unravel the world of prompting LLMs using LangChain</h4><p>In the previous articles, we saw:</p><ol><li><a href="https://medium.com/azure-monitor-from-a-programmers-perspective/langchain-i-626ea02d5d5c">Introduction to LangChain and using it to quickly create a chatbot that asks LLMs a bunch of puzzles.</a></li><li><a href="https://medium.com/azure-monitor-from-a-programmers-perspective/langchain-ii-basic-chatbot-unpacked-a60510b9ac6b">Doing a deep dive into the LangChain framework and the structures involved in creating a basic chatbot. It should give you a very good handle on some of the frequent classes and how they are organized.</a></li></ol><p>This article will examine the world of prompts within LangChain. Prompts are usually constructed at runtime from different sources, and LangChain makes it easier to address complex prompt generation scenarios.</p><h3>Prompt Template Classes</h3><blockquote>Purpose: Provide a mechanism to construct prompts for models. Prompts come in varying shapes and sizes to suit different business needs. Prompt template classes address all these needs and help produce a Prompt Value class that&#39;s used as a prompt to models.</blockquote><p>Prompts are usually part of a chain in projects. Thus, they all implement the <em>Runnable</em> interface (class) and can be <em>invoked. </em>The whole purpose of all the Classes in this article is to provide a structured and scalable way to produce <em>PromptValues</em> from <em>PromptTemplates</em> when invoked.</p><p>This <em>PromptValue</em> is then passed to the LLM Model for response.</p><p>Here are the classes that make up the LangChain prompts module.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/1*LEcJicqexHhudNTyx_GeAA.jpeg" /><figcaption>Hierarchy of classes comprising prompts.</figcaption></figure><h4>BasePromptTemplate</h4><p>The base class for all prompt templates. Since it is abstract and one of its derived classes is usually used.</p><h4>BaseChatPromptTemplate</h4><p>Base Class → BasePromptTemplate</p><p>This Abstract Base Class (ABC) is the base class for all chat prompt templates.</p><h4><strong>ChatPromptTemplate</strong></h4><p><em>Base Class → BaseChatPromptTemplate</em></p><p>The class we usually instantiate when dealing with chat prompts.</p><h4>StringPromptTemplate</h4><p>This is the base class for all templates that expose the functionality of f-strings in prompt manipulation. This is an abstract base class. We typically use one of the derived classes to validate input data and use it in format strings to generate a prompt.</p><h4>PromptTemplate</h4><p><em>Base Class → StringPromptTemplate</em></p><p>This is a basic f-string style template that comprises a string with several replacement parameters. We pass the parameters to generate the template at generation time.</p><h4>FewShotChatMessagePromptTemplate</h4><p><em>Base Classes → BaseChatPromptTemplate + FewShotMixin</em></p><p>Chat Prompt that supports few-shot examples in a conversational chat setting. Supports multiple messages with examples.</p><h4>FewShotPromptTemplate</h4><p><em>Base Classes → StringPromptTemplate+ FewShotMixin</em></p><p>Basic prompt with support for examples</p><h4>FewShotPromptWithTemplates</h4><p><em>Base Class → StringPromptTemplate</em></p><p>A prompt template with support for examples built into the prompt.</p><h4>ImagePromptTemplate</h4><p><em>Base Class → BasePromptTemplate</em></p><p>Image prompt template for multimodal model</p><h4>PipelinePromptTemplate</h4><p>A template is used to combine multiple prompt templates into a final prompt.</p><h3>Prompt Value Classes</h3><p>When we invoke a prompt template, a prompt value is returned. We usually invoke a template at runtime and pass values for all the replacement variables. Prompt Value represents the actual prompt ready to be passed to the model. It comprises several back-and-forth messages, examples, and images.</p><p>Here is how these classes are laid out:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/906/1*cgFHGbY02uTTZPbjOVKEuQ.gif" /><figcaption>Prompt Value class diagram</figcaption></figure><h4>PromptValue</h4><p>The abstract base class of all prompt values. It defines 2 important functions:</p><ol><li>to_messages: Give a List[BaseMessage] of all the messages in the prompt.</li><li>to_string: Gives a string representation of the prompt.</li></ol><h4>ChatPromptValue</h4><p>Base Class → PromptValue</p><p>A type of prompt value that is made of chat messages.</p><h4>ChatPromptValueConcrete</h4><p>Base Class → ChatPromptValue</p><p>It can restrict the type of chat message classes that can be part of it. The values are <em>AIMessage, SystemMessage, HumanMessage, ChatMessage, FunctionMessage, and ToolMessage.</em></p><h4>ImagePromptValue</h4><p><em>Base Class → PromptValue</em></p><p>A wrapper around the ImageURL class that represents an image prompt.</p><h4>ImageURL</h4><p><em>Derives from no class</em></p><p>Holds the URL of the image and its level of detail — high or low.</p><h3>Message Prompt Template Classes</h3><p>Prompts employ messages prompt templates to create and manipulate messages at runtime. They DO NOT derive from Runnable as they are not designed to be used as part of a processing chain. Several classes are available to support different message constructs. Here is a class diagram of the message classes.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/730/1*s4M7AMRYb9pH1iCmAeRP9A.jpeg" /><figcaption>Hierarchy of classes in Prompt Messages library</figcaption></figure><p>There are two fundamental kinds of messages: system messages, which are generated by the system, and human messages, which are input by the user. The content of the System message may be generated by the AI model or provided as part of the message template by the developer.</p><p>Each of these messages, in turn, can be comprised entirely of text or may include both text and image.</p><p>Message Prompt Template classes are designed to support a templatized approach to creating the underlying message.</p><p>Let&#39;s begin by looking at a basic text message.</p><h4>ChatMessagePromptTemplate</h4><p><em>Base Class → BaseStringMessagePromptTemplate</em></p><p>The concrete class constructs plain text messages. We usually use one of the factory methods to create prompt messages.</p><p><em>FromTemplate</em> is a very common factory method. This method is defined in the BaseStringMessagePromptTemplate (base class) and invoked from a derived class, which in our case is ChatMessagePromptTemplate (derived class).</p><p>The base class mandates that, at a minimum, a string template of the prompt be passed with placeholder variables for dynamic text (string) values. In addition, the derived class needs an additional parameter role of type string to construct the message.</p><pre><br>class BaseStringMessagePromptTemplate(BaseMessagePromptTemplate, ABC):<br>    &quot;&quot;&quot;Base class for message prompt templates that use a string prompt template.&quot;&quot;&quot;<br><br>    prompt: StringPromptTemplate<br>    &quot;&quot;&quot;String prompt template.&quot;&quot;&quot;<br>    additional_kwargs: dict = Field(default_factory=dict)<br>    &quot;&quot;&quot;Additional keyword arguments to pass to the prompt template.&quot;&quot;&quot;<br>    <br>    # We use this factory method from derived ChatMessagePromptTemplate<br>    @classmethod<br>    def from_template(<br>        # self gets assigned to cls and is of type of the calling class.<br>        cls: Type[MessagePromptTemplateT],<br>        template: str,<br>        template_format: str = &quot;f-string&quot;,<br>        partial_variables: Optional[Dict[str, Any]] = None,<br>        **kwargs: Any, #NOTE we need to pass any parameters for the derived class here. <br>    ) -&gt; MessagePromptTemplateT:<br>.....<br><br><br># ChatMessagePromptTemplate needs the param &#39;role&#39;<br>class ChatMessagePromptTemplate(BaseStringMessagePromptTemplate):<br>    &quot;&quot;&quot;Chat message prompt template.&quot;&quot;&quot;<br><br>    role: str<br>    &quot;&quot;&quot;Role of the message.&quot;&quot;&quot;<br>....   <br>    </pre><pre>from langchain_core.prompts.chat import (<br>    ChatMessagePromptTemplate,<br>)<br><br># template parameter is required by the base class BasseStringMessagePromptTemplate as a string for creating the prompt<br># role is required by ChatMessagePromptTemplate<br>chatMessagePrompt = ChatMessagePromptTemplate.from_template(<br>    template=&quot;Please give me flight options for {from_city} to {to_city}&quot;,<br>    role=&quot;travel agent&quot;,<br>)<br><br>print(f&quot;The type of Prompt Message Template is \n\t{type(chatMessagePrompt)}&quot;)<br><br># Use the prompt message template to generate the message by providing values for all the replacement variables.<br>baseMessage = chatMessagePrompt.format(from_city=&quot;New Delhi&quot;, to_city=&quot;Mumbai&quot;)<br><br>print(<br>    f&quot;The type of message is: \n\t{type(baseMessage)}, \n\tand its __repr__ value is:  {baseMessage.__repr__()}&quot;<br>)  </pre><p><a href="https://github.com/langchain-ai/langchain/blob/master/libs/core/langchain_core/prompts/chat.py">langchain/libs/core/langchain_core/prompts/chat.py at master · langchain-ai/langchain</a></p><p>The output is:</p><pre>The type of Prompt Message Template is <br>        &lt;class &#39;langchain_core.prompts.chat.ChatMessagePromptTemplate&#39;&gt;<br>The type of message is: <br>        &lt;class &#39;langchain_core.messages.chat.ChatMessage&#39;&gt;, <br>        and its __repr__ value is:  ChatMessage(content=&#39;Please give me flight options for New Delhi to Mumbai&#39;, role=&#39;travel agent&#39;)</pre><h4>BaseMessagePromptTemplate</h4><p>The abstract base for all message templates in a prompt.</p><h4>BaseStringMessagePromptTemplate</h4><p><em>Base Class → BaseMessagePromptTemplate</em></p><p>If you need control over the types of variables contained in the template, the names of input variables, and the output parser that will be used to parse the response from the Model. Use all these features if you create a set of foundation classes for your application and want your team to follow a specific class for creating prompt messages. In this case, derive your class from BaseStringMessagePromptTemplate and have greater control</p><h4>_StringImageMessagePromptTemplate</h4><p><em>Base Class → BaseMessagePromptTemplate</em></p><p>This class is not directly to be used in our code (starts with _) as the implementation may change over time. We derive from this class when we need to work with messages that can be a combination of string and image (URL)</p><h4>HumanMessagePromptTemplate</h4><p><em>Base Class → _StringImageMessagePromptTemplate</em></p><p>A message that represents the User input. This can be a combination of text and image.</p><h4>SystemMessagePromptTemplate</h4><p><em>Base Class → _StringImageMessagePromptTemplate</em></p><p>Represents a message that is not sent to the user, only to the model.</p><h4>MessagePlaceholder</h4><p><em>Base Class → BaseMessagePromptTemplate</em></p><p>It acts like a proxy for an actual message, which is supplied at runtime.</p><h3>Message Classes</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/640/1*NmFGpuAIon0mJqle0G1-BA.gif" /><figcaption>LangChain Message Classes</figcaption></figure><p>Finally, we have a set of classes for the messages themselves. They are what prompt message templates produce to be used as part of the chainable prompt sent to the Model.</p><h4>BaseMessage</h4><p>The abstract base of all messages</p><h4>ChatMessage</h4><p>A message that can be assigned to an arbitrary speaker. This is the class that <em>ChatMessagePromptTemplate </em>returns. I have never really come across the need to directly use this class.</p><h4>BaseMessageChunk</h4><p>A part of a message that can be concatenated with other chunks to create the full message.</p><h4>AIMessage</h4><p>This is the message that the model returns. It contains the model&#39;s raw response, usage data, and other metadata that LangChain adds.</p><h4>AIMessageChunk</h4><p>A chunk of response from Model. We use this when streaming responses from the model and updating the UI.</p><h4>UsageMetadata</h4><p>Both AIMessageChunk and AIMessage include in their response usage metadata. It gives the following:</p><ol><li><em>input_tokens: number</em></li><li><em>output_tokens: number</em></li><li><em>total_tokens: number</em></li></ol><h4>ChatMessageChunk</h4><p>Create the message from chunks</p><h4>HumanMessage</h4><p>As the name implies, messages from human/user to the model. We use this class very often when representing user input to a chatbot.</p><h4>HumanMessageChunk</h4><p>Used when composing a Human Message from chunks.</p><h4>SystemMessage</h4><p>These messages prime Model behavior, such as giving instructions, context, etc.</p><h4>SystemMessageChunk</h4><p>When creating a system message from chunks.</p><h4>Message Examples</h4><p>In this example, we will use the Cohere command-r-plus model and demonstrate how messages are constructed and passed to the model:</p><pre>import os<br>import configparser<br>from langchain_cohere import ChatCohere<br>from langchain_core.messages import HumanMessage, SystemMessage<br><br># set up config parser<br>config = configparser.ConfigParser()<br>config.read(&quot;../config.ini&quot;)  # holds secrets and keys<br><br># load the Default section.<br>default = config[&quot;DEFAULT&quot;]<br><br><br># set the environment variables needed by LangSmith<br>os.environ[&quot;LANGCHAIN_TRACING_V2&quot;] = default.get(&quot;LANGCHAIN_TRACING_V2&quot;)<br>os.environ[&quot;LANGCHAIN_API_KEY&quot;] = default.get(&quot;LANGCHAIN_API_KEY&quot;)<br>os.environ[&quot;LANGCHAIN_ENDPOINT&quot;] = default.get(&quot;LANGCHAIN_ENDPOINT&quot;)<br>os.environ[&quot;LANGCHAIN_PROJECT&quot;] = default.get(&quot;LANGCHAIN_PROJECT&quot;)<br><br># load Cohere config<br>cohereConfig = config[&quot;cohere&quot;]<br><br>os.environ[&quot;COHERE_API_KEY&quot;] = cohereConfig.get(&quot;COHERE_API_KEY&quot;)<br><br># select the model.<br>model = ChatCohere(model=&quot;command-r-plus&quot;)<br><br>messageForModel = [<br>    SystemMessage(&quot;You are a chat bot who is an expert in political science&quot;),<br>    HumanMessage(content=&quot;What is the capital of India&quot;),<br>]<br><br># Or we could have created individual messages as:<br>sysMessage = SystemMessage(&quot;You are a chat bot who is an expert in political science&quot;)<br>humanMessage = HumanMessage(content=&quot;What is the capital of India&quot;)<br><br># Models such as Cohere, OpenAI extend BaseChatModel from LangChainCore<br># They are runnable and hence have an invoke method.<br># The invoke takes in a List and returns object of BaseMessage<br><br># ~# Both of the following product the same results.<br># invoke params:<br># str | List[dict | tuple | BaseMessage] | PromptValue<br><br># Here we pass a List of Base Message that are eval at runtime<br>response = model.invoke(messageForModel)<br><br># Here we pass the actual Base Messages themselves.<br>response = model.invoke([sysMessage, humanMessage])<br><br>print(response.__repr__())</pre><p>The response is:</p><pre>AIMessage(content=&#39;The capital of India is New Delhi, which is a union territory within the country and is also the seat of the Government of India. It is located within the National Capital Territory of Delhi and is one of the largest cities in India in terms of population.&#39; <br>additional_kwargs={&#39;documents&#39;: None, &#39;citations&#39;: None, <br>&#39;search_results&#39;: None, <br>&#39;search_queries&#39;: None, <br>&#39;is_search_required&#39;: None, <br>&#39;generation_id&#39;: &#39;78f266b4-26f2-4f22-8bed-8d86b0c49673&#39;, <br>&#39;token_count&#39;: {&#39;input_tokens&#39;: 87, &#39;output_tokens&#39;: 51}} <br>response_metadata={&#39;documents&#39;: None, &#39;citations&#39;: None, <br>&#39;search_results&#39;: None, <br>&#39;search_queries&#39;: None, <br>&#39;is_search_required&#39;: None, <br>&#39;generation_id&#39;: &#39;78f266b4-26f2-4f22-8bed-8d86b0c49673&#39;, <br>&#39;token_count&#39;: {&#39;input_tokens&#39;: 87, &#39;output_tokens&#39;: 51}} <br>id=&#39;run-a29c21af-aa75-4dc2-9e94-404d5d1eded9-0&#39; <br>usage_metadata={&#39;input_tokens&#39;: 87, &#39;output_tokens&#39;: 51, &#39;total_tokens&#39;: 138})</pre><p>Notice that the response from <em>ChatModel</em> is an <em>AIMessage</em> containing an object of type <em>UsageMetadata</em> that gives information about the number of tokens consumed in the call.</p><h4>Tool/Function Call Messages</h4><p>LangChain also provides a set of classes for making calls to tools. Tools are an interface for your application to interact with a line of business applications. They are usually functions annotated with <em>@tool</em> and are called based on the response from the model. There are classes to make calls to tools, collect responses from tools, and pass them to models, handling errors.</p><h3>Samples — Prompt Templates, Message prompt templates, Prompt Values and Messages</h3><p>Let&#39;s use these classes and get a feel for them.</p><blockquote>Note: LangChain heavily uses the Pydantic library. One pattern is using the Field function to define and initialise fields on host classes. So we (or LangChain) defines a class, defines a field in the class and sets its value to Field function of Pydantic. This function takes a function pointer as its argument. Next when an instance of the class (That has Pydantic BaseModel in its hierarchy) is being created it takes all the keyword arguments passed and looks for keys with name same as the field defined. If it finds a match, it takes the value of the keyword argument and passes it to the function pointer passes to Field. The result is set as the value of the field.</blockquote><p>Let’s see an example.</p><h4>ImagePromptTemplate</h4><blockquote>Purpose: To provide a mechanism to create a prompt that comprises of a single image. The image can be from a URL or a file on your computer</blockquote><pre>import os<br>import configparser<br>from langchain_cohere import ChatCohere<br>from langchain_core.messages import HumanMessage, SystemMessage<br><br># set up config parser<br>config = configparser.ConfigParser()<br># config.read(&quot;../config.ini&quot;)  # holds secrets and keys<br>config.read(&quot;config.ini&quot;)<br><br># load the Default section.<br>default = config[&quot;DEFAULT&quot;]<br><br><br># set the environment variables needed by LangSmith<br>os.environ[&quot;LANGCHAIN_TRACING_V2&quot;] = default.get(&quot;LANGCHAIN_TRACING_V2&quot;)<br>os.environ[&quot;LANGCHAIN_API_KEY&quot;] = default.get(&quot;LANGCHAIN_API_KEY&quot;)<br>os.environ[&quot;LANGCHAIN_ENDPOINT&quot;] = default.get(&quot;LANGCHAIN_ENDPOINT&quot;)<br>os.environ[&quot;LANGCHAIN_PROJECT&quot;] = default.get(&quot;LANGCHAIN_PROJECT&quot;)<br><br># load Cohere config<br>cohereConfig = config[&quot;cohere&quot;]<br><br>os.environ[&quot;COHERE_API_KEY&quot;] = cohereConfig.get(&quot;COHERE_API_KEY&quot;)<br><br># select the model.<br>model = ChatCohere(model=&quot;command-r-plus&quot;)<br><br># ~# Image Prompt Template<br><br>from langchain_core.prompts.image import ImagePromptTemplate<br><br><br># ImagePromptTemplate class has a field &quot;template&quot;. This is the program flow.<br><br># init method of ImagePromptTemplate accepts (**kwargs). This method checks:<br># 1. if &quot;input_variable&quot; key is not in kwargs then it adds it to the dict with value - []<br># 2. Checks if &quot;input_variables&quot; has any of the keys &quot;url&quot;, &quot;path&quot; or &quot;detail&quot;<br># 3. If it does then error is thrown.<br># 4. Calls init of super.<br><br># A common pattern in good Python programming is to set a field in the class and<br># have Pydantic BaseModel Class take the task of assigning them values.<br># Pydantic Field has a parameter called &quot;default_factory&quot; which is a callable (defines a __call__() method.)<br># Now while setting the field in the class, like template in our case we initialize it as such:<br># template: dict = Field(default_factory=dict)<br># What is this does is, sets the callable to dict.<br><br># Now when super(init) gets executed, the base class hierarchy invokes init of BaseModel in pydantic site_packages -&gt; main.py file.<br># Here its supplied with the instance of our newly created object of type ImagePromptTemplate which has a attribute &quot;template&quot;<br># that is defined as Field function with callable set to dict()<br># So then BaseModel goes thru all the fields in kwargs it will find a template key whose value is a dictionay.<br># Since our object also has attribute template there is a match<br># And Pydantic will execute dict({&quot;url&quot;: &quot;the prompt&quot;}) and pass the result to template attribute.<br># You can check in debug and see that template is indeed set to this value;<br><br>imagePrompt = ImagePromptTemplate(template={&quot;url&quot;: &quot;the prompt&quot;})<br><br>print(imagePrompt.__repr__())</pre><p>And the output is:</p><pre># This when passed to eval() will create an ImagePromptTemplate.<br><br>ImagePromptTemplate(input_variables=[], template={&#39;url&#39;: &#39;the prompt&#39;})<br></pre><p>And here is a debugger output showing the field.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*EzGfgrM-xIHBmfWn8lGeQw.png" /><figcaption>The template field in our object is set.</figcaption></figure><p>Note: Here, we gave a dummy value for the URL. Usually, the template field is set with replacement variables whose values are passed when we invoke the format method. This then replaces the keyword arguments with the same-named variables in the template string. See below:</p><p>Lets look at this in more detail:</p><pre># Image prompt template works with ImageURL class<br># https://api.python.langchain.com/en/latest/prompt_values/langchain_core.prompt_values.ImageURL.html#langchain_core.prompt_values.ImageURL<br># It has url and detail as the 2 attributes<br><br>imagePrompt = ImagePromptTemplate()<br>imagePrompt.template = {<br>    &quot;url&quot;: &quot;http://yourdomain/images/{imageName}.pic&quot;,<br>    &quot;detail&quot;: &quot;low&quot;,<br>}<br><br># Invoking the prompt template produces its PromptValue class<br>promptVal = imagePrompt.invoke(input={&quot;imageName&quot;: &quot;pic1.jpeg&quot;})<br><br># Here is what PromptVal looks like<br>print(promptVal.__repr__())<br><br># And here is what the template class is.<br>print(imagePrompt.__repr__())</pre><p>And the output is:</p><pre>ImagePromptValue(image_url={&#39;detail&#39;: &#39;low&#39;, &#39;url&#39;: &#39;http://yourdomain/images/pic1.jpeg.pic&#39;})<br>ImagePromptTemplate(input_variables=[], template={&#39;url&#39;: &#39;http://yourdomain/images/{imageName}.pic&#39;, &#39;detail&#39;: &#39;low&#39;})</pre><p>We have been looking at images with URL. But what if the image is available locally on your computer:</p><pre>imagePrompt = ImagePromptTemplate()<br><br># The format method has an additional parameter &quot;path&quot; to the path of the local file.<br># invoking format on the prompt template will pick the image and covert into base64 encoded string<br># It will create an ImageURL object by replacing the attribute name &quot;path&quot; to &quot;url&quot;<br>imageURL = imagePrompt.format(path=&quot;PromptMessages.jpg&quot;, detail=&quot;low&quot;)<br><br># Now we can assign this ready and formatted dict to the template<br>imagePrompt.template = imageURL<br><br># When this prompt is used as part of chain its invoked to produce the prompt value thats passed to model<br>imagePromptValue = imagePrompt.invoke(input={})<br>print(imagePromptValue.__repr__())</pre><p>The output is:</p><pre>ImagePromptValue(image_url={&#39;detail&#39;: &#39;low&#39;, <br>&#39;url&#39;: &#39;data:image/jpeg;base64,/9j/4AAQSkZJRgABAQAAAQAD/4g.....&lt;long string&gt; </pre><p>You can see the pattern:</p><ol><li>You set a template string with replacement values for URL at design time.</li><li>At runtime, you can pass the URL or a part of it and format the template string to get the ImageURL dictionary. Use this as the updated template and then invoke.</li><li>Or (recommended) you can call the invoke function on the prompt template, passing the replacement values.</li></ol><h4>ChatPromptTemplate</h4><blockquote>Purpose: A prompt template to support conversations with models. Conversations can include many different kind of messages — AI, Human, system, tool or function. This is where we use the classes for Message Prompt Templates and Message classes.</blockquote><p>Let&#39;s look at a quick example:</p><pre>## Chat Prompt Template - using a list of string.<br>This represents a conversation context of a chat. The conversation can contain many back and forth messages. This template threrefore has a colleciton of messages. We can supply the messages using tuples or creating message objects using Message Prompt Template.<br><br>We can provide any of the following type of messages:<br>1. system - Like instructions or a note or context. Anything that we as the developer want the model to use for understanding its role, its tasks and expectations and give sufficient context.<br>2. human - Encapsulates any responses or messages by the user<br>3. ai - Encapsulates any responses by the model. <br>4. placeholder - A placeholder where the prompt will receive an optional list of messages at run time.<br>5. assistant - Gets created as an AI message internally<br><br>Apart from this there are message templates to represent messages to tools and their responses that go back to model. We shall cover them when we get to tools.<br><br>Below is an example of creating a chat prompt using string.</pre><pre>from langchain_core.prompts import ChatPromptTemplate<br><br>template = ChatPromptTemplate(<br>    [<br>        (&quot;system&quot;, &quot;You are a helpful AI bot. Your name is {name}.&quot;),<br>        (&quot;human&quot;, &quot;Hello, how are you doing?&quot;),<br>        (&quot;ai&quot;, &quot;I&#39;m doing well, thanks!&quot;),<br>        (&quot;human&quot;, &quot;{user_input}&quot;)<br>    ]<br>)<br><br>print(&#39;Template Messages: &#39;,template.messages.__repr__())<br>print(&#39;Template Object: &#39;, template.__repr__())<br><br># Invoking executes the template and places <br>promptValue = template.invoke(input=<br>  {&quot;name&quot;:&quot;Arun&quot;, <br>  &quot;user_input&quot;:&quot;How is the weather today?&quot;})<br><br>print(&#39;Prompt Value: &#39;,promptValue.__repr__())</pre><p>The output is:</p><pre>Template Messages:  [<br>  SystemMessagePromptTemplate(prompt=PromptTemplate(input_variables=[&#39;name&#39;], template=&#39;You are a helpful AI bot. Your name is {name}.&#39;)), <br>  HumanMessagePromptTemplate(prompt=PromptTemplate(input_variables=[], template=&#39;Hello, how are you doing?&#39;)), <br>  AIMessagePromptTemplate(prompt=PromptTemplate(input_variables=[], template=&quot;I&#39;m doing well, thanks!&quot;)), <br>  HumanMessagePromptTemplate(prompt=PromptTemplate(input_variables=[&#39;user_input&#39;], template=&#39;{user_input}&#39;))]<br><br><br>Template Object:  <br>  ChatPromptTemplate(input_variables=[&#39;name&#39;, &#39;user_input&#39;], <br>  messages=[<br>    SystemMessagePromptTemplate(prompt=PromptTemplate(input_variables=[&#39;name&#39;], template=&#39;You are a helpful AI bot. Your name is {name}.&#39;)), <br>    HumanMessagePromptTemplate(prompt=PromptTemplate(input_variables=[], template=&#39;Hello, how are you doing?&#39;)), <br>    AIMessagePromptTemplate(prompt=PromptTemplate(input_variables=[], template=&quot;I&#39;m doing well, thanks!&quot;)), <br>    HumanMessagePromptTemplate(prompt=PromptTemplate(input_variables=[&#39;user_input&#39;], template=&#39;{user_input}&#39;))])<br><br><br>Prompt Value:  ChatPromptValue(messages=<br>  [SystemMessage(content=&#39;You are a helpful AI bot. Your name is Arun.&#39;), <br>  HumanMessage(content=&#39;Hello, how are you doing?&#39;), <br>  AIMessage(content=&quot;I&#39;m doing well, thanks!&quot;), <br>  HumanMessage(content=&#39;How is the weather today?&#39;)])</pre><p>It&#39;s evident what&#39;s happening here. The messages we supplied as tuples are used to create objects of the respective type of messages. Notice how all the Template Messages output are a List of objects of the respective Message Prompt Template.</p><p>Next, these message objects are passed to ChatPromptTemplate as a list of objects to create the template object.</p><p>Finally, when we invoke, we supply the “input” parameter, which is a dictionary containing the values of all the replacement variables. The output of invoke is a Prompt Value object, which is used as input for the model. All replace variables are resolved and set in the prompt value object.</p><p>Next let&#39;s do the same thing using Message Prompt templates:</p><pre>from langchain_core.prompts import SystemMessagePromptTemplate, HumanMessagePromptTemplate, AIMessagePromptTemplate, ChatPromptTemplate, PromptTemplate<br>from langchain_core.messages import HumanMessage, AIMessage<br><br># We use message prompt template as we have a replacement variable.<br>systemMessage = SystemMessagePromptTemplate.from_template(&quot;You are a helpful AI bot. Your name is {name}.&quot;)<br><br># No need to use message prompt template as no replacement variable.<br>humanMessage1 = HumanMessage(&quot;Hello, how are you doing?&quot;)<br><br>aimessage = AIMessage(&quot;I&#39;m doing well, thanks!&quot;)<br><br>humanMessage2 = HumanMessagePromptTemplate.from_template(&quot;{user_input}&quot;)<br><br>template = ChatPromptTemplate.from_messages([<br>    systemMessage, humanMessage1, aimessage, humanMessage2<br>])<br><br>promptValue = template.invoke(input={&quot;name&quot;:&quot;Arun&quot;, &quot;user_input&quot;:&quot;How is the weather today?&quot;})<br><br>print(&#39;Prompt Value: &#39;, promptValue.__repr__())</pre><p>As you will expect, the output is the same:</p><pre>Prompt Value:  ChatPromptValue(<br>  messages=[<br>    SystemMessage(content=&#39;You are a helpful AI bot. Your name is Arun.&#39;), <br>    HumanMessage(content=&#39;Hello, how are you doing?&#39;), <br>    AIMessage(content=&quot;I&#39;m doing well, thanks!&quot;), <br>    HumanMessage(content=&#39;How is the weather today?&#39;)])</pre><p>Note: Message Prompt Template classes are wrappers around Prompt Template classes. As you can see above, they take a parameter called <em>“prompt,”</em> which is nothing but a prompt template.</p><p>The convenience method <em>.from_template</em> saves you the hassle of providing input_variables separately and automatically creates them.</p><p><em>What is your chat that includes an Image?</em></p><p>That is rather simple. As we know by now, a Chat Prompt Template comprises a list of messages composed from Message Prompt Templates. As we also know that these message prompt templates are wrappers over prompt templates, it&#39;s only logical that we should be able to supply a Chat Prompt Template with an Image prompt template as one of the messages.</p><p>If you check the source code here:</p><p><a href="https://github.com/langchain-ai/langchain/blob/master/libs/core/langchain_core/prompts/chat.py#L816">langchain/libs/core/langchain_core/prompts/chat.py at master · langchain-ai/langchain</a></p><p>You will notice that <em>ChatPromptTemplate’s __init__ message</em> parameter is defined as:</p><pre># The message param of ChatPromptTemplate __init__ method<br>messages: Sequence[MessageLikeRepresentation]<br><br># MessageLikeRepresentation is defined as:<br>MessageLikeRepresentation = Union[<br>    MessageLike,<br>    Tuple[<br>        Union[str, Type],<br>        Union[str, List[dict], List[object]],<br>    ],<br>    str,<br>]<br><br># And MessageLike as:<br>MessageLike = Union[BaseMessagePromptTemplate, <br>  BaseMessage, BaseChatPromptTemplate]<br><br># HumanMessagePromptTemplate implements BaseMessagePromptTemplate via<br># class __StringImageMessagePromptTemplate<br># Here is how this is defined:<br><br>class _StringImageMessagePromptTemplate(BaseMessagePromptTemplate):<br>    &quot;&quot;&quot;Human message prompt template. This is a message sent from the user.&quot;&quot;&quot;<br><br>    prompt: Union[<br>        StringPromptTemplate, List[Union[StringPromptTemplate, ImagePromptTemplate]]<br>    ]<br>    &quot;&quot;&quot;Prompt template.&quot;&quot;&quot;<br><br># And you can see that prompt supports a list with ImagePromptTemplate as a possible value</pre><p>Let&#39;s see this in action:</p><pre>from langchain_core.prompts.image import ImagePromptTemplate<br>from langchain_core.prompts import HumanMessagePromptTemplate, ChatPromptTemplate<br>from langchain_core.messages import HumanMessage<br><br><br># Lets create the Image Prompt Template<br>imagePrompt = ImagePromptTemplate(input_variables=[&quot;imageName&quot;], template={&quot;url&quot;:&quot;http://yourdomain.com/{imageName}&quot;, &quot;detail&quot;:&quot;low&quot;})<br><br># Just to check we are doing this right - lets invoke and see if the variables are replaced<br>promptValue = imagePrompt.invoke(input={&quot;imageName&quot;:&quot;good_pic.jpeg&quot;})<br>print(&#39;Image Prompt&#39;,imagePrompt.__repr__())<br>print(&#39;Prompt Value&#39;, promptValue.__repr__())<br>print(&quot;ImageURL object&quot;, promptValue.image_url)<br><br><br># Now lets wrap our Image Prompt Template within our Human Message Prompt Template<br>humanMessage1 = HumanMessagePromptTemplate(prompt=[imagePrompt])<br>print(&#39;Human Message1&#39;, humanMessage1.__repr__())<br><br># And since there is no replacement variable we create the message directly.<br>humanMessage2 = HumanMessage(&quot;Describe this image in brief&quot;)<br>print(&#39;Human Message2&#39;, humanMessage2.__repr__())<br><br># Now we can use both these in our Chat Prompt Template<br>chatPrompt = ChatPromptTemplate.from_messages([<br>    humanMessage1, humanMessage2<br>])<br>print(&#39;Chat prompt: &#39;, chatPrompt.__repr__())<br><br># And invoke will substitute the values and prepare the prompt to be sent to model<br>promptValue = chatPrompt.invoke(input={&quot;imageName&quot;:&quot;my_pic.png&quot;})<br>print(&#39;Prompt Value: &#39;,promptValue.__repr__())<br><br></pre><p>The output is:</p><pre>Image Prompt ImagePromptTemplate(<br>  input_variables=[&#39;imageName&#39;], <br>  template={&#39;url&#39;: &#39;http://yourdomain.com/{imageName}&#39;, &#39;detail&#39;: &#39;low&#39;})<br><br>Prompt Value ImagePromptValue(<br>  image_url={&#39;detail&#39;: &#39;low&#39;, &#39;url&#39;: &#39;http://yourdomain.com/good_pic.jpeg&#39;})<br><br>ImageURL object {&#39;detail&#39;: &#39;low&#39;, &#39;url&#39;: &#39;http://yourdomain.com/good_pic.jpeg&#39;}<br><br>Human Message1 HumanMessagePromptTemplate(<br>  prompt=[ImagePromptTemplate(input_variables=[&#39;imageName&#39;], template={&#39;url&#39;: &#39;http://yourdomain.com/{imageName}&#39;, &#39;detail&#39;: &#39;low&#39;})])<br><br>Human Message2 HumanMessage(content=&#39;Describe this image in brief&#39;)<br><br>Chat prompt:  ChatPromptTemplate(<br>  input_variables=[&#39;imageName&#39;], <br>  messages=[<br>      HumanMessagePromptTemplate(prompt=[ImagePromptTemplate(input_variables=[&#39;imageName&#39;], template={&#39;url&#39;: &#39;http://yourdomain.com/{imageName}&#39;, &#39;detail&#39;: &#39;low&#39;})]), <br>      HumanMessage(content=&#39;Describe this image in brief&#39;)])<br><br>Prompt Value:  ChatPromptValue(<br>  messages=[<br>      HumanMessage(content=[{&#39;type&#39;: &#39;image_url&#39;, &#39;image_url&#39;: {&#39;url&#39;: &#39;http://yourdomain.com/my_pic.png&#39;, &#39;detail&#39;: &#39;low&#39;}}]), <br>      HumanMessage(content=&#39;Describe this image in brief&#39;)])</pre><p>In this way, you can use Images in your prompts.</p><h3>Prompts with examples/shots</h3><p>We sometimes want to provide examples of expected responses along with the prompt. This can lead to much better results in many cases. LangChain supports support for multiple scenarios examples.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2bSZrOtN9pMKrJATkQXWvA.jpeg" /><figcaption>Traverse the right path to Example Selectors</figcaption></figure><p>Here is how it works:</p><ol><li>At the top level, we have 2 choices: either work with an Example selector class or provide examples</li><li>Use Example Selector when there is a common bank of examples and we want to draw from them. This class implements methods to add to the bank and select relevant examples from it using a strategy.</li><li>If you do not have a bank of examples, provide them on the fly.</li><li>If using an example selector, initialize it by providing examples and example prompts.</li><li>Examples are usually provided consistently, making it possible to supply a prompt template (PromptTemplate) string.</li><li>When sending the examples to the model, a prefix string should be added before them. This will usually set the context.</li><li>A suffix where we ask the question or query whose response we want from the model.</li></ol><p>It is possible that the built-in 3 strategies do not fit your needs. In that case, extend <em>BaseExampleSelector</em> and implement your strategy by overriding the <em>addExample</em> and <em>selectExamples</em> methods.</p><p>Here is a class diagram covering all the classes in the Example selector collection.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/789/1*4z_sKz70mTTLCdbehp2hHw.jpeg" /><figcaption>Example Selector Classes</figcaption></figure><h4>MaxMarginalSimilarityExampleSelector</h4><p>Base Class → _VectorStoreExampleSelector → BaseExampleSelector</p><p>Selects examples from vector store based on cosine similarity of examples to the input, iteratively adding them while also penalizing those that are very, very close to the selected examples.</p><h4>SemanticSimilarityExampleSelector</h4><p>Base Class → _VectorStoreExampleSelector → BaseExampleSelector</p><p>This class also selects examples based on cosine similarity but does not focus on making the examples diverse by penalizing close proximity to selected examples.</p><h3>Sample code — Prompts with examples</h3><p>Let&#39;s first look at how we provide examples without using the selector:</p><pre>from langchain_core.prompts import PromptTemplate, FewShotPromptTemplate<br>examples = [<br>    {<br>        &quot;type&quot;:&quot;animals&quot;,<br>        &quot;que&quot;:&quot;Do lions hunt in a pack?&quot;,<br>        &quot;ans&quot;:&quot;Yes lions, unlike tigers hunt as a group&quot;<br>    },<br>    {<br>        &quot;type&quot;:&quot;corporates&quot;,<br>        &quot;que&quot;:&quot;Who is the founder of Tesla Motors?&quot;,<br>        &quot;ans&quot;: &quot;Elon Musk&quot;<br>    },<br>    {<br>        &quot;type&quot;: &quot;geography&quot;,<br>        &quot;que&quot;:&quot;What are the mountain ranges in west India?&quot;,<br>        &quot;ans&quot;: &quot;They are western ghats&quot;<br>    },<br>    {<br>        &quot;type&quot;:&quot;synonyms&quot;,<br>        &quot;que&quot;:&quot;stoic&quot;,<br>        &quot;ans&quot;:&quot;quiet&quot;<br>    }<br>]<br><br>prompt = FewShotPromptTemplate(<br>    examples=examples,<br>    example_prompt = PromptTemplate.from_template(&quot;Input: {que} -&gt; Output: {ans}&quot;),<br>    prefix = &quot;Please answer the following question based on the examples provided.&quot;,<br>    suffix = &quot;{user_question}&quot;,<br>    input_variables=[&quot;user_question&quot;]<br>)<br><br>promptValue = prompt.format(user_question=&quot;What is the capital of Japan&quot;)<br><br>print(promptValue.__repr__())<br><br></pre><p>The response is:</p><pre>&#39;Please answer the following question based on the examples provided.\n\n<br>Input: Do lions hunt in a pack? -&gt; Output: Yes lions, unlike tigers hunt as a group\n\n<br>Input: Who is the founder of Tesla Motors? -&gt; Output: Elon Musk\n\n<br>Input: What are the mountain ranges in west India? -&gt; Output: They are western ghats\n\n<br>Input: stoic -&gt; Output: quiet\n\n<br>Input: What is the capital of Japan -&gt; Output: &#39;</pre><p>There is nothing wrong here; if you want to use a fixed set of examples, this suffices. But in systems of any size, you may want to reuse a set of examples and have a way of selecting from them. Let&#39;s see that:</p><pre>from typing import Any, Dict, List<br>from langchain_core.example_selectors import LengthBasedExampleSelector, BaseExampleSelector<br>from langchain_core.prompts import PromptTemplate, FewShotPromptTemplate<br><br># Our set of examples<br>examples = [<br>    {<br>        &quot;type&quot;:&quot;animals&quot;,<br>        &quot;que&quot;:&quot;Do lions hunt in a pack?&quot;,<br>        &quot;ans&quot;:&quot;Yes lions, unlike tigers hunt as a group&quot;<br>    },<br>    {<br>        &quot;type&quot;:&quot;corporates&quot;,<br>        &quot;que&quot;:&quot;Who is the founder of Tesla Motors?&quot;,<br>        &quot;ans&quot;: &quot;Elon Musk&quot;<br>    },<br>    {<br>        &quot;type&quot;: &quot;geography&quot;,<br>        &quot;que&quot;:&quot;What are the mountain ranges in west India?&quot;,<br>        &quot;ans&quot;: &quot;They are western ghats&quot;<br>    },<br>    {<br>        &quot;type&quot;:&quot;synonyms&quot;,<br>        &quot;que&quot;:&quot;stoic&quot;,<br>        &quot;ans&quot;:&quot;quiet&quot;<br>    }<br>    <br>]<br><br># Our custom selector that returns specified type of examples<br>class MyExampleSelector(BaseExampleSelector):<br>    def __init__(self, examples):<br>        self.examples = examples<br>    def add_example(self, example: Dict[str, str]) -&gt; Any:<br>        return super().add_example(example)<br><br>    def select_examples(self, input_variables) -&gt; List[dict]:<br>        selected_examples = []<br>        typeOfQue = input_variables[&quot;typeOfQue&quot;]<br>        for example in self.examples:<br>            if example[&quot;type&quot;] == typeOfQue:<br>                selected_examples.append(example)<br><br>        return selected_examples<br><br>        <br># selector = LengthBasedExampleSelector(examples=examples, example_prompt= PromptTemplate.from_template(&quot;Input: {que} -&gt; Output: {ans}&quot;), max_length=8)<br>selector = MyExampleSelector(examples=examples)<br><br># Prompt template<br>prompt = FewShotPromptTemplate(<br>    example_selector=selector, # now instead of using examples directly we use selector<br>    example_prompt = PromptTemplate.from_template(&quot;Input: {que} -&gt; Output: {ans}&quot;),<br>    prefix = &quot;Please answer the following question based on the examples provided.&quot;,<br>    suffix = &quot;Input: {user_question} -&gt; Output: &quot;,<br>    input_variables=[&quot;user_question&quot;, &quot;typeOfQue&quot;]<br>)<br><br><br><br>promptValue = prompt.format(user_question=&quot;What is the capital of Japan&quot;, typeOfQue=&quot;geography&quot;)<br>print(promptValue.__repr__())</pre><p>The response is:</p><pre><br>&#39;Please answer the following question based on the examples provided.\n\n<br>Input: What are the mountain ranges in west India? -&gt; Output: They are western ghats\n\n<br>Input: What is the capital of Japan -&gt; Output: &#39;</pre><p>Notice that only the examples of type geography are selected. This is how the values are passed along:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*jxt_9A0iU15NgM0iPwMGuw.jpeg" /><figcaption>How FewShotPromptTemplate works with custom Example Selector.</figcaption></figure><p>This brings us to the end of this rather long blog. Hope you all find it interesting and informative.</p><h3>References</h3><ul><li><a href="https://api.python.langchain.com/en/latest/core_api_reference.html#module-langchain_core.prompts">langchain_core 0.2.29 - 🦜🔗 LangChain 0.2.12</a></li><li><a href="https://api.python.langchain.com/en/latest/language_models/langchain_core.language_models.chat_models.BaseChatModel.html">langchain_core.language_models.chat_models.BaseChatModel - 🦜🔗 LangChain 0.2.12</a></li><li><a href="https://api.python.langchain.com/en/latest/_modules/langchain_core/language_models/chat_models.html#BaseChatModel">langchain_core.language_models.chat_models - 🦜🔗 LangChain 0.2.12</a></li><li><a href="https://api.python.langchain.com/en/latest/prompts/langchain_core.prompts.chat.ChatMessagePromptTemplate.html#langchain_core.prompts.chat.ChatMessagePromptTemplate">langchain_core.prompts.chat.ChatMessagePromptTemplate - 🦜🔗 LangChain 0.2.12</a></li><li><a href="https://github.com/langchain-ai/langchain/blob/master/libs/core/langchain_core/language_models/chat_models.py">langchain/libs/core/langchain_core/language_models/chat_models.py at master · langchain-ai/langchain</a></li></ul><h4>ImagePromptTemplate</h4><p><a href="https://github.com/langchain-ai/langchain/blob/master/libs/core/langchain_core/prompts/image.py">langchain/libs/core/langchain_core/prompts/image.py at master · langchain-ai/langchain</a></p><h4>ChatPromptTemplate</h4><p><a href="https://github.com/langchain-ai/langchain/blob/master/libs/core/langchain_core/prompts/chat.py">langchain/libs/core/langchain_core/prompts/chat.py at master · langchain-ai/langchain</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2df826c0ec3d" width="1" height="1" alt=""><hr><p><a href="https://techwitharun.com/langchain-iii-prompts-2df826c0ec3d">LangChain — III: Prompts</a> was originally published in <a href="https://techwitharun.com/">Arun Saxena’s Tech Terrain</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[LangChain — II: Basic Chatbot Unpacked]]></title>
            <link>https://techwitharun.com/langchain-ii-basic-chatbot-unpacked-a60510b9ac6b?source=rss-99f899e3817a------2</link>
            <guid isPermaLink="false">https://medium.com/p/a60510b9ac6b</guid>
            <category><![CDATA[chatbot-development]]></category>
            <category><![CDATA[langchain-chain]]></category>
            <category><![CDATA[llm-chatbot]]></category>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[langchain]]></category>
            <dc:creator><![CDATA[Arun Saxena]]></dc:creator>
            <pubDate>Thu, 25 Jul 2024 10:24:12 GMT</pubDate>
            <atom:updated>2024-07-25T10:32:21.029Z</atom:updated>
            <content:encoded><![CDATA[<h3>LangChain — II: Basic Chatbot Unpacked</h3><h4>Get to know the underlying LangChain framework involved in a basic chat.</h4><p>In <a href="https://medium.com/azure-monitor-from-a-programmers-perspective/langchain-i-626ea02d5d5c">the previous article, we set up the environment and got a feel for LangChain. Without knowing the underlying framework, we called GPT and got it to answer some basic puzzles.</a></p><p>In this article, we will create a basic chatbot rooted in the <a href="https://python.langchain.com/v0.2/docs/tutorials/chatbot/">same tutorial LangChain provides</a>. But here is the difference:</p><p>Once you have read and explored the threads that this article provides, you should start appreciating the underlying framework and the value that it adds.</p><p>I have broken down the simple tutorial into its logical parts and tried to explain the mechanics of classes and their hierarchy, how all of them come together to deliver functionality, and, more importantly, once you start going deeper, you will start discovering natural ways to apply the framework for a variety of business use cases.</p><h3>The most basic chat</h3><p>Here is the code for the most basic chat:</p><pre>import os<br>import configparser<br><br># import the parser to extract the string output from the model response<br>from langchain_core.output_parsers import StrOutputParser<br><br># Import the implementation of ChatCohere(BaseCohere, BaseChatModel)<br>from langchain_cohere import ChatCohere<br><br># Base Chat takes in a list containing an instance of HumanMessage<br>from langchain_core.messages import HumanMessage<br><br><br><br><br># set up config parser<br>config = configparser.ConfigParser()<br>config.read(&quot;config.ini&quot;)<br><br># load the Default section.<br>default = config[&quot;DEFAULT&quot;]<br><br># set the environment variables needed by LangSmith<br>os.environ[&quot;LANGCHAIN_TRACING_V2&quot;] = default.get(&quot;LANGCHAIN_TRACING_V2&quot;)<br>os.environ[&quot;LANGCHAIN_API_KEY&quot;] = default.get(&quot;LANGCHAIN_API_KEY&quot;)<br>os.environ[&quot;LANGCHAIN_ENDPOINT&quot;] = default.get(&quot;LANGCHAIN_ENDPOINT&quot;)<br>os.environ[&quot;LANGCHAIN_PROJECT&quot;] = default.get(&quot;LANGCHAIN_PROJECT&quot;)<br><br># load Cohere config<br>cohereconfig = config[&quot;cohere&quot;]<br><br># set Cohere key in env variable<br>os.environ[&quot;COHERE_API_KEY&quot;] = cohereconfig.get(&quot;COHERE_API_KEY&quot;)<br><br># select the model<br>cohereModel = ChatCohere(name=&quot;command-r&quot;)<br><br># create a parser instance to extract the content string from AIMessage response of model<br>parser = StrOutputParser()<br><br># create the chain of Runnables<br>chain = cohereModel | parser<br><br># pass the HumanMessage to the first component of the chain of Runnables<br>response = chain.invoke([HumanMessage(content=&quot;hi i am bob&quot;)])<br><br>print(response)<br><br><br>#=== OUTPUT ===<br>Hello Bob, I am Coral. How can I help you today?</pre><p>Here is a simplified and minimal class UML diagram showing the relationships between the classes of our minimal chat application.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/798/1*i0ioGAxxe8MVQRv46omtrg.jpeg" /><figcaption>Classes involved in the basic Cohere chat using LangChain.</figcaption></figure><p>Let&#39;s now explore some key classes in developing our chat application.</p><h4>Runnable</h4><p>Runnable is a core class extensively used in the formation of chains. Chains are a mechanism for combining components that implement the Runnable class.</p><p>Many LangChain components, including chat models (like ChatCohere), LLMs, output parsers, prompts, etc., are derived from Runnable. In our example, you can see that BaseLanguageModel, the base of all chat model classes, derives from RunnableSerializable, a class that can be serialized to JSON and utilized for chaining.</p><p>A runnable class comprises a unit of work that can be assembled by invoking, batching, or streaming the components that comprise the chain. They all have sync, async, batch, and streaming support.</p><p>The statement:</p><pre>chain = cohereModel | parser</pre><p>constructs the chain. Alternatively, we can also construct a chain by using the class RunnableSequence:</p><pre>chain = RunnableSequence(cohereModel, parser)</pre><pre><br>chain.invoke([HumanMessage(content=&quot;hi i am bob&quot;)])</pre><p>This statement calls the <em>invoke</em> method of the first component, <em>cohereModel</em>, with the <em>HumanMessage</em> as an argument. The output is a <em>BaseMessage</em> object.</p><p>This baseMessage object is passed as input to the <em>StrOutputParser&#39;s</em> <em>invoke</em> method, which returns the most likely string representation of the response.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/401/1*K1F-m4gImEUO0AELkpQuKg.jpeg" /><figcaption>Chain of Runnables.</figcaption></figure><p>When the chain is invoked <em>chain.invoke(…)</em>, it calls the invoke method of <em>the cohereModel object, passing it the parameters it received as input. The response of ChatCohere-type objects is an instance of BaseMessage, which is then passed to the chained object parser. The parser</em> then applies some logic and returns a likely string representation of the input <em>BaseMessage</em>.</p><p>Apart from invoke there are other important methods of Runnable:</p><ol><li><strong>invoke/ainvoke:</strong> call the component with input / async</li><li><strong>batch/abatch:</strong> Execute the component on a list of inputs / async. Batch runs invoke() in parallel using thread pool executor.</li><li><strong>stream/astream: </strong>stream back chunks of response / async</li></ol><h4>BaseChatModel</h4><p>This is the base class for all Chat Models like ChatGPT, Cohere, Claude, Mistral etc.</p><p>All classes that derive from BaseChatModel follow the same signature:</p><pre>def invoke(str | List[dict | tuple | BaseMessage] | PromptValue):<br>  # function definition<br>  pass</pre><p>We are passing on a list created by initializing the HumanMessage class.</p><pre>[HumanMessage(content=&quot;hi i am bob&quot;)]</pre><p>This creates an instance of <em>BaseMessage</em> and packs it into a list. This list is a valid input to the <em>invoke</em> method.</p><h4>BaseMessage</h4><p>We are using <em>HumanMessage</em>, derived from <em>BaseMessage</em>, which represents a message from a human. Typically, this is the user of the chat application.</p><p>The single required parameter for creating an object of <em>BaseMessage</em> type is <em>content</em>.</p><pre>param content: Union[str, List[Union[str,Dict]]]<br><br># Union, List, Dict are all defined in typing module<br># Union means one of the input types is expected. We are passing a string.</pre><h4>PromptValue</h4><p>PromptValue is also a legal input to the BaseChatModel invoke method. It is a wrapper class that can have prompts that are text, a list of messages, or, in the future, other media like images or speech.</p><h4>StrOutputParser</h4><p>This is one of the lower-level parsers available. It derives from BaseOutputParser. The invoke method is defined here.</p><pre>def invoke(<br>        self, input: Union[str, BaseMessage], config: Optional[RunnableConfig] = None<br>    ) -&gt; T:<br><br># this says input can be either string or BaseMessage.<br><br>parser.input_schema.schema() # get JSON schema of the input<br>parser.output_schema.schema() # gets JSON schema of the output</pre><p>Both input and output schema are Pydantic models auto-generated from the Runnable structure.</p><p>Pydantic is a Python library that does data validation and setting management using Python-type annotations. It enforces hints at runtime and provides user-friendly errors.</p><p><a href="https://docs.pydantic.dev/1.10/">Welcome to Pydantic - Pydantic</a></p><p>Pydantic models are a way of defining schemas. Models are classes that inherit from pydantic.BaseModel and define fields as annotated attributes.</p><p>Runnable class is derived from this pydantic.BaseModel. If you check the source code of Runnable:</p><p><a href="https://api.python.langchain.com/en/latest/_modules/langchain_core/runnables/base.html#Runnable">langchain_core.runnables.base - 🦜🔗 LangChain 0.2.7</a></p><p>You will see a line:</p><pre>from langchain_core.pydantic_v1 import BaseModel, Field<br><br># and later there are get schema methods<br><br>def get_input_schema(<br>        self, config: Optional[RunnableConfig] = None<br>    ) -&gt; Type[BaseModel]:<br><br>def output_schema(self) -&gt; Type[BaseModel]:<br>        &quot;&quot;&quot;The type of output this Runnable produces specified as a pydantic model.&quot;&quot;&quot;<br>        return self.get_output_schema()</pre><p>In this way, a parser publishes its input and output schemas.</p><h3>Adding History to our basic chat</h3><p>At this stage, if the user were to ask the chat application another question related to the earlier conversation, things would fall apart. LLM models are frozen in time and do not persist in conversation details. It&#39;s up to us as developers to manage this conversation history and provide it to the model as part of the prompt each time.</p><p>Chat History is managed through a set of classes offered by LangChain Community. First, we need to install the module to use them.</p><pre>pip install langchain_community</pre><p>The module langchain_core contains the abstract base class for all instantiable chat history classes. Various vendors like Postgres, Redis, etc provide all the implementations. The ULM hierarchy is as:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/882/1*AK0G4D_QYwdzuVOVRKgIbg.jpeg" /><figcaption>Hierarchy of Chat Message History classes.</figcaption></figure><p>In this article, we will save the message to memory only. In production, we use some persistent stores. Since multiple chats can be going on at the same time, we need to ensure that we save each conversation separately.</p><p>BaseChatMessageHistory is the abstract base class of all message history classes and provides a template for managing messages. Various individual classes then implement the template in their context.</p><p>Let&#39;s try to use InMemoryChatHistory as an example.</p><pre>import os<br>import sys<br>import configparser<br>from langchain_core.runnables import RunnableSequence<br>from typing import TypeVar<br>import abc<br>import asyncio<br><br><br># Import the implementation of ChatCohere(BaseCohere, BaseChatModel)<br>from langchain_cohere import ChatCohere<br><br># Base Chat takes in a list containing an instance of HumanMessage<br>from langchain_core.messages import HumanMessage, SystemMessage<br><br>from langchain_core.chat_history import (<br>    BaseChatMessageHistory,<br>    InMemoryChatMessageHistory,<br>)<br><br># set up config parser<br>config = configparser.ConfigParser()<br>config.read(&quot;config.ini&quot;)<br><br># load the Default section.<br>default = config[&quot;DEFAULT&quot;]<br><br># set the environment variables needed by LangSmith<br>os.environ[&quot;LANGCHAIN_TRACING_V2&quot;] = default.get(&quot;LANGCHAIN_TRACING_V2&quot;)<br>os.environ[&quot;LANGCHAIN_API_KEY&quot;] = default.get(&quot;LANGCHAIN_API_KEY&quot;)<br>os.environ[&quot;LANGCHAIN_ENDPOINT&quot;] = default.get(&quot;LANGCHAIN_ENDPOINT&quot;)<br>os.environ[&quot;LANGCHAIN_PROJECT&quot;] = default.get(&quot;LANGCHAIN_PROJECT&quot;)<br><br># load Cohere config<br>cohereconfig = config[&quot;cohere&quot;]<br><br># set Cohere key in env variable<br>os.environ[&quot;COHERE_API_KEY&quot;] = cohereconfig.get(&quot;COHERE_API_KEY&quot;)<br><br># select the model<br>cohereModel = ChatCohere(name=&quot;command-r&quot;)<br><br># creating a store for our messages to retain chat history<br>store = InMemoryChatMessageHistory()<br><br><br># get the messages from the store and have the calling program wait for the messages to be delivered.<br># This is async because usually the store will have persistence and therefore the wait times.<br>async def getMessages():<br>    await asyncio.sleep(2)  # sleep 2 secs to mimic DB operation<br>    print(&quot;&gt;&gt;&gt; messages retrieved from DB&quot;)<br>    return await store.aget_messages()<br><br><br># Add the first user message to the store.<br>store.add_message(HumanMessage(&quot;Hi! I am Bob&quot;))<br><br><br># pass the converstaion history to the model. Wait for the messages to be delivered before calling the model.<br>response = cohereModel.invoke(asyncio.run(getMessages()))<br>print(response.content)<br><br># Add the system response to the store.<br>store.add_message(SystemMessage(response.content))<br><br><br># add the next user message to the conversation store.<br>store.add_message(HumanMessage(&quot;What is my name and its meaning?&quot;))<br><br># calling the model again with full conversation history<br>response = cohereModel.invoke(asyncio.run(getMessages()))<br><br># We get a response from the model that remembers the entire conversation.<br>print(response.content)<br><br></pre><pre>=== OUTPUT ===<br>&gt;&gt;&gt; messages retrieved from DB<br>Hello Bob! It&#39;s nice to meet you. How can I help you today?<br><br>&gt;&gt;&gt; messages retrieved from DB<br>Your name is Bob. It is a shortened form of the name Robert, <br>which means &quot;bright fame&quot; or &quot;famous brilliance.&quot; <br>The name Robert has English and German origins and has been in use for many centuries. <br>Is there anything else you would like to know about your name or its origin?</pre><p>While all is good, a few issues must be addressed here. Since Chat History classes do not have Runnable in their chain of ancestors, we cannot directly chain them. This means we need to execute the chain, fetch the response from the model, and then update the store. Not ideal.</p><p>Another problem with this approach is that we must implement the functions responsible for storing and retrieving messages in all chats. This is repetitive coding and should set our alarm bells ringing. Ideally, all this should be neatly encapsulated within the chat history class with the right semantics for the store—in memory or DB, etc.</p><h4>History with a Runnable</h4><p>To address these problems, the class RunnableWithMessageHistory is derived from Runnable. It takes the chain as the first argument and a pointer to a function (remember, functions are first-class citizens) that supports reading and writing to history as the second argument.</p><pre>import os<br>import configparser<br>import asyncio<br>from langchain_core.runnables.history import RunnableWithMessageHistory<br><br># Import the implementation of ChatCohere(BaseCohere, BaseChatModel)<br>from langchain_cohere import ChatCohere<br><br># Base Chat takes in a list containing an instance of HumanMessage<br>from langchain_core.messages import HumanMessage, SystemMessage<br><br>from langchain_core.chat_history import (<br>    BaseChatMessageHistory,<br>    InMemoryChatMessageHistory,<br>)<br><br># set up config parser<br>config = configparser.ConfigParser()<br>config.read(&quot;config.ini&quot;)<br><br># load the Default section.<br>default = config[&quot;DEFAULT&quot;]<br><br># set the environment variables needed by LangSmith<br>os.environ[&quot;LANGCHAIN_TRACING_V2&quot;] = default.get(&quot;LANGCHAIN_TRACING_V2&quot;)<br>os.environ[&quot;LANGCHAIN_API_KEY&quot;] = default.get(&quot;LANGCHAIN_API_KEY&quot;)<br>os.environ[&quot;LANGCHAIN_ENDPOINT&quot;] = default.get(&quot;LANGCHAIN_ENDPOINT&quot;)<br>os.environ[&quot;LANGCHAIN_PROJECT&quot;] = default.get(&quot;LANGCHAIN_PROJECT&quot;)<br><br># load Cohere config<br>cohereconfig = config[&quot;cohere&quot;]<br><br># set Cohere key in env variable<br>os.environ[&quot;COHERE_API_KEY&quot;] = cohereconfig.get(&quot;COHERE_API_KEY&quot;)<br><br># select the model<br>cohereModel = ChatCohere(name=&quot;command-r&quot;)<br><br><br># our store now is a dict with session ids as key<br>store = {}<br><br><br># the callable that provides an interface to our message store.<br>def get_session_history(session_id: str) -&gt; BaseChatMessageHistory:<br>    if session_id not in store:<br>        store[session_id] = InMemoryChatMessageHistory()<br>    return store[session_id]<br><br><br>config = {&quot;configurable&quot;: {&quot;session_id&quot;: &quot;abc2&quot;}}<br>with_message_history = RunnableWithMessageHistory(cohereModel, get_session_history)<br><br>response = with_message_history.invoke(<br>    [HumanMessage(content=&quot;Hi! I am Bob&quot;)], config=config<br>)<br><br>print(response.content)<br><br>response = with_message_history.invoke(<br>    [HumanMessage(content=&quot;Whats my name&quot;)], config=config<br>)<br><br>print(response.content)</pre><p>Here is a flowchart of this simple program:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/549/1*tkt-kJrxlb8J-MzW20qCYQ.jpeg" /><figcaption>Flowchart</figcaption></figure><h4>RunnableWithMessageHistory</h4><p>Manages chat message history for another runnable. It&#39;s a wrapper around another runnable, providing it with chat history before invocation and storing the model&#39;s response afterward.</p><p>Our program only passes the model as the base runnable for this wrapper. However, we usually pass a chain as the runnable. The wrapper simply calls the chain&#39;s invoke method, which calls the invoke on the first runnable in the chain.</p><p>When invoking this runnable, we must provide a configuration containing the appropriate parameters for the message history factory. By default, it takes a single config parameter <em>session_id</em>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/341/1*_vBBaJI0QKTfezinz-MI6w.jpeg" /><figcaption>RunnableWithMessageHistory.</figcaption></figure><h4>get_session_history</h4><p>This Callable object implements the __call()__ method. Callable is a class (in the typing module) used to indicate a callable or basically a method. It returns an object of type <em>BaseChatMessageHistory </em>for the session id. <em>RunnableWithMessageHistory </em>then ensures that the message store is updated<em> </em>with responses from the model.</p><p>The overall program now looks neat and clean, with minimal code. In the next article, we will look at the world of prompts!</p><h3>References</h3><h4>Runnable docs:</h4><p><a href="https://python.langchain.com/v0.1/docs/expression_language/interface/">LangChain Expression Language Cheatsheet | 🦜️🔗 LangChain</a></p><h4>Runnable Code:</h4><p><a href="https://api.python.langchain.com/en/latest/_modules/langchain_core/runnables/base.html#Runnable">langchain_core.runnables.base - 🦜🔗 LangChain 0.2.7</a></p><h4>HumanMessage Class</h4><p><a href="https://api.python.langchain.com/en/latest/messages/langchain_core.messages.human.HumanMessage.html">langchain_core.messages.human.HumanMessage - 🦜🔗 LangChain 0.2.7</a></p><h4>BaseMessage Class</h4><p><a href="https://api.python.langchain.com/en/latest/messages/langchain_core.messages.base.BaseMessage.html#langchain_core.messages.base.BaseMessage">langchain_core.messages.base.BaseMessage - 🦜🔗 LangChain 0.2.7</a></p><h4>ChatCohere Class</h4><p><a href="https://api.python.langchain.com/en/latest/chat_models/langchain_community.chat_models.cohere.ChatCohere.html">langchain_community.chat_models.cohere.ChatCohere - 🦜🔗 LangChain 0.2.7</a></p><h4>RunnableSequence:</h4><p><a href="https://api.python.langchain.com/en/latest/_modules/langchain_core/runnables/base.html#RunnableSequence">langchain_core.runnables.base - 🦜🔗 LangChain 0.2.7</a></p><h4>Typing Module</h4><p><a href="https://api.python.langchain.com/en/latest/_modules/typing.html">typing - 🦜🔗 LangChain 0.1.11</a></p><h4>BaseChatModel</h4><p><a href="https://api.python.langchain.com/en/latest/language_models/langchain_core.language_models.chat_models.BaseChatModel.html">langchain_core.language_models.chat_models.BaseChatModel - 🦜🔗 LangChain 0.2.12</a></p><h4>InMemoryChatHistory</h4><p><a href="https://api.python.langchain.com/en/latest/chat_history/langchain_core.chat_history.InMemoryChatMessageHistory.html#langchain_core.chat_history.InMemoryChatMessageHistory">langchain_core.chat_history.InMemoryChatMessageHistory - 🦜🔗 LangChain 0.2.10</a></p><h4>AIMessage</h4><p><a href="https://api.python.langchain.com/en/latest/messages/langchain_core.messages.ai.AIMessage.html#langchain_core.messages.ai.AIMessage">langchain_core.messages.ai.AIMessage - 🦜🔗 LangChain 0.2.10</a></p><h4>Runnable Interface</h4><p><a href="https://python.langchain.com/v0.1/docs/expression_language/interface/">LangChain Expression Language Cheatsheet | 🦜️🔗 LangChain</a></p><h4>Cohere Chat Model</h4><p><a href="https://api.python.langchain.com/en/latest/chat_models/langchain_cohere.chat_models.ChatCohere.html">langchain_cohere.chat_models.ChatCohere - 🦜🔗 LangChain 0.2.7</a></p><h4>RunnableBindingBase</h4><p><a href="https://api.python.langchain.com/en/latest/runnables/langchain_core.runnables.base.RunnableBindingBase.html#langchain_core.runnables.base.RunnableBindingBase">langchain_core.runnables.base.RunnableBindingBase - 🦜🔗 LangChain 0.2.11</a></p><h4>RunnableWithMessageHistory</h4><p><a href="https://api.python.langchain.com/en/latest/runnables/langchain_core.runnables.history.RunnableWithMessageHistory.html#langchain_core.runnables.history.RunnableWithMessageHistory">langchain_core.runnables.history.RunnableWithMessageHistory - 🦜🔗 LangChain 0.2.11</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a60510b9ac6b" width="1" height="1" alt=""><hr><p><a href="https://techwitharun.com/langchain-ii-basic-chatbot-unpacked-a60510b9ac6b">LangChain — II: Basic Chatbot Unpacked</a> was originally published in <a href="https://techwitharun.com/">Arun Saxena’s Tech Terrain</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[RAG — I: LangChain Basics]]></title>
            <link>https://techwitharun.com/langchain-i-626ea02d5d5c?source=rss-99f899e3817a------2</link>
            <guid isPermaLink="false">https://medium.com/p/626ea02d5d5c</guid>
            <category><![CDATA[langchain]]></category>
            <category><![CDATA[llm-applications]]></category>
            <category><![CDATA[groq]]></category>
            <category><![CDATA[rag-application]]></category>
            <category><![CDATA[langsmith]]></category>
            <dc:creator><![CDATA[Arun Saxena]]></dc:creator>
            <pubDate>Wed, 03 Jul 2024 13:29:24 GMT</pubDate>
            <atom:updated>2024-07-10T13:09:20.023Z</atom:updated>
            <content:encoded><![CDATA[<h3>RAG — I: LangChain basics</h3><h4>Using GPT 3.5 Turbo &amp; Cohere to solve logical reasoning puzzles :)</h4><p>In this article, we will demonstrate the basic usage of LangChain to interact with popular LLM GPT 3.5. We will ask it to solve a few logical reasoning puzzles and see how it performs. We will use chain-of-thought prompting to get a step-by-step analysis of how the LLM analyzed the problem.</p><h3>Installing and setting up LangChain</h3><p>We will be using the following products:</p><ol><li>LangChain:</li><li>LangSmith:</li></ol><p>Installing LangChain is easy:</p><pre>pip intall langchain</pre><p>This installs the basic LangChain. We normally use LangChain and its integrations with various models. They need to be installed separately.</p><p>We will be using JupyterLab for this and future articles on LangChain. If you are unfamiliar with it, now is a good time to learn it and set it up.</p><p>Here is the documentation:</p><p><a href="https://docs.jupyter.org/en/latest/">Project Jupyter Documentation - Jupyter Documentation 4.1.1 alpha documentation</a></p><p>Here are the installation steps</p><p><a href="https://jupyter.org/install">Project Jupyter</a></p><p>We will also use LangSmith in all our articles. Most applications make multiple calls to LLM before generating a response to the user. It&#39;s useful to inspect what is happening inside the chain or agent.</p><p>You need to sign up for this service. The free tier should get you started on all these articles.</p><p><a href="https://www.langchain.com/langsmith">LangSmith: AI Agent &amp; LLM Observability Platform</a></p><p>For this article we will set up tracing by default, thus all calls will get traced. To do this we need to get LangChain API key. Head over to your LangSmith account and click the settings icon at the bottom left of your home page:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*i9Q_ByrW0mtl86GiTPQ8kA.png" /><figcaption>Click the settings icon on your landing page.</figcaption></figure><p>From the settings page, generate a Personal API key. Save a copy of the key.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*9hY5uC5dnV5Avg59hqcTZg.png" /><figcaption>Generate a personal key and save it.</figcaption></figure><h3>Config File</h3><p>We will use the Python config parser module and consume all config settings from the config.ini file. A config file can have multiple sections that help you logically group settings.</p><pre>[DEFAULT]<br>LANGCHAIN_TRACING_V2 : true<br>LANGCHAIN_API_KEY : lsv2_&lt;LangSmith API Key&gt;6120<br>LANGCHAIN_ENDPOINT : https://api.smith.langchain.com<br>LANGCHAIN_PROJECT : pr-helloworld<br><br>[azure.openai]<br>AZURE_OPENAI_API_KEY : ff92 &lt;Azure Open AI Key&gt; 4d8b<br>AZURE_OPENAI_DEPLOYMENT_NAME : first<br>AZURE_OPENAI_API_VERSION : 2024-02-01<br>AZURE_OPENAI_ENDPOINT = https://arun1.openai.azure.com/</pre><p>We will discuss these settings along the way. But for now, notice the 2 sections <em>DEFAULT</em> and <em>azure.openai</em>. To enable tracing set the value of <em>LANGCHAIN_TRACING_V2</em> to <em>true</em>. Also paste the saved API key for <em>LANGCHAIN_API_KEY</em>.</p><p>The LangSmith endpoint is standard. And mention any name of your choice for the project. Its a place where all your traces will be collected.</p><h3>ConfigParser Module</h3><p>Here are some useful functions of this module.</p><pre>&gt;&gt;&gt; import configparser # imports the module<br>&gt;&gt;&gt; config = configparser.ConfigParser() # creates an instance of ConfigParser class<br>&gt;&gt;&gt; config.read(&#39;config.ini&#39;) # reads the config file. I have it placed in the same folder as the notebook<br>&gt;&gt;&gt; config.sections() # list the sections<br>[&#39;azure.openai&#39;] # DEFAULT is not included.<br><br>&gt;&gt;&gt; default = config[&#39;DEFAULT&#39;] # now default will have a dictionary of key value pairs in the default section<br>&gt;&gt;&gt; default.get(&#39;&lt;KEY NAME&gt;&#39;) # gets the value as string</pre><h3>Azure Openai</h3><h4>Install LangChain SDK for Openai</h4><p>In this article, we will use multiple language models with LangChain. Let&#39;s begin with Azure Openai.</p><p>First off, install LangChain SDK for Openai:</p><pre>pip install -qU langchain-openai<br><br># q - quiet<br># U - Upgrade if needed.</pre><p>Once this is done, we can use this SDK to call OpenAI.</p><h4>Set up Azure OpenAI service.</h4><p>Now, let us set up ChatGPT in Azure. In your Azure subscription home page, search for Azure Open AI. If you do not already have an open AI service, create one.</p><p>Once the service is created, navigate to it. From the left menu, navigate to Resource Management -&gt; Keys and Endpoint. Copy the endpoint and any one of the keys.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2n96FQMCObTV9DFPz0_WKg.png" /><figcaption>Note the endpoint and any one of the keys.</figcaption></figure><p>Now that we have a service let&#39;s deploy a language model. Click <em>Model Deployments</em> below <em>Keys and Endpoints</em>.</p><p>Clicking Manage Deployments will take you to Azure Open AI Studio. Head to deployments and create a new deployment. For this article, I am using GPT-35-Turbo.</p><p>Give your deployment a name — as I have given <em>first</em> below:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*qj_ShO-8ZbPhoT5jEZkfwQ.png" /><figcaption>Deploy a GPT model and give it a name.</figcaption></figure><p>To make a call to Azure Open AI we need the following:</p><ol><li>The endpoint of the Azure Open AI service</li><li>The secret key of the service.</li></ol><p>Both these get us to the Azure OpenAI service. Next, we need to identify the target deployment:</p><ol><li>The name of the deployment — <em>first</em></li><li>The API version.</li></ol><p>In Azure, API versions are usually denoted by dates. The latest version for the chat completions API of OpenAI GPT models is <em>2024–02–01</em>. For more, refer to the link in the references section.</p><h4>Putting it all together</h4><p>We are now ready to call GPT, consume the response and check the traces in LangSmith.</p><p>Refer to the notebook:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/a865823bd2124892d798cd8712e950ef/href">https://medium.com/media/a865823bd2124892d798cd8712e950ef/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/fc3b4babc4ab02c0b2caa61742065055/href">https://medium.com/media/fc3b4babc4ab02c0b2caa61742065055/href</a></iframe><h4>Output Parser</h4><p>You will notice that we are using output parsers to parse the response received from GPT. LangChain bundles responses from different LLMs in a consistent structure <em>AIMessage</em>. This contains the string response from the LLM and other metadata.</p><p>StrOutputParser implements the Runnable Interface. More about this parser and its base classes here:</p><p><a href="https://api.python.langchain.com/en/latest/output_parsers/langchain_core.output_parsers.string.StrOutputParser.html">langchain_core.output_parsers.string.StrOutputParser - 🦜🔗 LangChain 0.2.6</a></p><h4>Chaining</h4><p>In LangChain, the <strong><em>|</em></strong> operator creates a chain in which the first output is passed to the second, and so on. All the instances in the chain must implement the Runnable interface.</p><p>The runnable interface has a .invoke() method. The output of the first invoke call is passed as input to the invoke of the second instance, and so on.</p><pre># The following statements:<br>chain = model | parser<br>chain.invoke(messages)<br><br><br># Are equivalent to:<br><br>modelResult = model.invoke(messages)<br>textResponse = parser.invoke(modelResult)<br><br># This is the same as the more verbose Pipe<br>chain = model.pipe(parser)<br>chain.invoke(messages)</pre><h4>Prompt Template</h4><p>While preparing prompts for LLMs, we usually mix user input with some application logic and search data. Therefore, passing straight strings may often be useful. LangChain supports prompt templates that allow us to stitch together the prompt.</p><p>To demonstrate this, let&#39;s define a set of puzzles that a user can choose from and ask the LLM to solve. Additionally, we will allow the user to qualify the type of response as detailed, step-by-step or brief.</p><p>Create a file puzzles.py and place it in the same folder as the notebook:</p><pre>def puzzles(puzzleName):<br>    if puzzleName == &quot;tallBuilding&quot;:<br>        return &quot;&quot;&quot;A man who lives in a 30-story building decides to jump out of his window. <br>        He survives the fall with no injuries. How did that happen?&quot;&quot;&quot;<br>    elif puzzleName == &quot;hungryLions&quot;:<br>        return &quot;&quot;&quot;A man is condemned to death. He has to choose from three rooms to accept his punishment. <br>        The first room has a firing squad with guns loaded. The second room has a blazing fire. <br>        The third room is full of tigers that haven’t eaten for six months. Which room should he choose?&quot;&quot;&quot;<br>    elif puzzleName == &quot;walk10Flights&quot;:<br>        return &quot;&quot;&quot;A woman lives on the 30th floor and hates taking the stairs. <br>        Every day, she takes the elevator down to the lobby floor to go to work. <br>        When she comes home from work, she takes the elevator to the 20th floor and walks the rest of the way up, except on days when it rains. <br>        Those days, she takes the elevator all the way up. <br>        Why does she walk the last 10 flights of stairs if she hates it so much?&quot;&quot;&quot;<br>    elif puzzleName == &quot;screamDead&quot;:<br>        return &quot;&quot;&quot;A woman opened a door, screamed and was found dead a few minutes later. <br>        No gunshots were heard in the area. <br>        What happened to her?&quot;&quot;&quot;<br>    else:<br>        return &quot;&quot;&quot;Four men, one of whom was known to have committed a certain crime, <br>        made the following statements when questioned by the police:<br>        1. Archie: Dave did it.<br>        2. Dave: Tony did it.<br>        3. Gus: I didn’t do it.<br>        4. Tony: Dave lied when he said I did it.<br>        If only one of these four statements is true, who was the guilty man?&quot;&quot;&quot;</pre><p>These are our interesting puzzles. We will import them as a module in our notebook and then, based on user input, pick a puzzle from them. We have also templatized our system message.</p><p>To create a template prompt, we use the class <em>ChatPromptTemplate</em>. This class has multiple ways to create a prompt template. We are creating our template from messages.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/4fc2aa65052ac7c5e15d9df31b8dc7f0/href">https://medium.com/media/4fc2aa65052ac7c5e15d9df31b8dc7f0/href</a></iframe><h3>LangSmith</h3><p>After running all the cells in the notebook, head over to your LangSmith account. You should see a list of all the projects. Remember, you initially provided a name for the project as an environment variable. Go to that project in LangSmith.</p><p>In the project, you should see a list of all the Runs; by default, the latest run is on the top. Since we are running a sequence of runnables we will see the name as RunnableSequence. Click on any one of them:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*i26FjmpAaAMjjgUwR3CPJg.png" /><figcaption>Runnable Sequence</figcaption></figure><p>In the panel that opens up, from its left, click on RunnableSequence as shown above. On the right, you will see an overview of the entire run. Input is the dictionary we provide when invoking, and output is the overall output.</p><p>Remember, our chain had its object:</p><ol><li>ChatPromptTemplate <em>(called prompt_template)</em></li><li>AzureChatOpenAI <em>(called model)</em></li><li>StrOutputParser <em>(called parser)</em></li></ol><p>You see all these objects in LangSmith right below RunnableSequence. Clicking on any of them will give you the input and output of each of them.</p><h3>Cohere</h3><p>Let&#39;s set up Cohere and get it to answer the puzzle.</p><p>First, install LangChain SDK for Cohere:</p><pre>pip install -qU langchain-cohere</pre><p>Set up an account in Cohere, and get your API key from the dashboard. In config.ini add a section for cohere:</p><pre>[cohere]<br>COHERE_API_KEY : c2G4&lt;your_cohere_API_Key&gt;GlMm</pre><p>Next, we are going to use VS Code and get it to answer the same puzzle:</p><pre>import os<br>import configparser<br>import puzzles<br><br># set up config parser<br>config = configparser.ConfigParser()<br>config.read(&quot;config.ini&quot;)<br><br># load the Default section.<br>default = config[&quot;DEFAULT&quot;]<br><br><br># set the environment variables needed by LangSmith<br>os.environ[&quot;LANGCHAIN_TRACING_V2&quot;] = default.get(&quot;LANGCHAIN_TRACING_V2&quot;)<br>os.environ[&quot;LANGCHAIN_API_KEY&quot;] = default.get(&quot;LANGCHAIN_API_KEY&quot;)<br>os.environ[&quot;LANGCHAIN_ENDPOINT&quot;] = default.get(&quot;LANGCHAIN_ENDPOINT&quot;)<br>os.environ[&quot;LANGCHAIN_PROJECT&quot;] = default.get(&quot;LANGCHAIN_PROJECT&quot;)<br><br># load Cohere config<br>cohereConfig = config[&quot;cohere&quot;]<br><br>os.environ[&quot;COHERE_API_KEY&quot;] = cohereConfig.get(&quot;COHERE_API_KEY&quot;)<br><br><br>from langchain_cohere import ChatCohere<br><br># We are using command-r. Its their leading model for RAG applications.<br>coheremodel = ChatCohere(model=&quot;command-r&quot;)<br><br><br># import the parser and template modules of langchain core<br>from langchain_core.output_parsers import StrOutputParser<br>from langchain_core.prompts import ChatPromptTemplate<br><br><br>parser = StrOutputParser()<br><br>puzzle = puzzles.puzzles(&quot;tallBuilding&quot;)  # Based on user input pick a puzzle.<br><br><br>system_template = (<br>    &quot;solve the following puzzle. Please provide a {responseType} response.&quot;<br>)<br><br>prompt_template = ChatPromptTemplate.from_messages(<br>    [(&quot;system&quot;, system_template), (&quot;user&quot;, puzzle)]<br>)<br><br><br># prompt Template also implements runnable and can be easily chained.<br>chain = prompt_template | coheremodel | parser<br><br>str = chain.invoke({&quot;responseType&quot;: &quot;detailed&quot;})<br><br>print(str)<br><br># === OUTPUT ===<br>The man survived the fall because he fell on a pile of trash, <br>which broke his fall and prevented him from getting injured. <br>Alternatively, he might have jumped out of the window on the ground floor, <br>which would also explain his survival. <br><br>Does that solve the riddle? <br>Let me know if you would like further clarification on this puzzle!</pre><h3>Groq</h3><p>Groq is a very popular hosting cloud. It claims extremely efficient processing based on a different hardware architecture: Language Processing Units (LPU).</p><p>We will use Llama-3 8b for our purpose. Go ahead and create your account in GroqCloud and get your API keys. Update the config.ini file:</p><pre>[groq]<br>GROQ_API_KEY : gsk_AS84h&lt;your groq API key&gt;rbG</pre><p>Here is the Python code to make the request:</p><pre>import os<br>import configparser<br>import puzzles<br><br># set up config parser<br>config = configparser.ConfigParser()<br>config.read(&quot;config.ini&quot;)<br><br># load the Default section.<br>default = config[&quot;DEFAULT&quot;]<br><br><br># set the environment variables needed by LangSmith<br>os.environ[&quot;LANGCHAIN_TRACING_V2&quot;] = default.get(&quot;LANGCHAIN_TRACING_V2&quot;)<br>os.environ[&quot;LANGCHAIN_API_KEY&quot;] = default.get(&quot;LANGCHAIN_API_KEY&quot;)<br>os.environ[&quot;LANGCHAIN_ENDPOINT&quot;] = default.get(&quot;LANGCHAIN_ENDPOINT&quot;)<br>os.environ[&quot;LANGCHAIN_PROJECT&quot;] = default.get(&quot;LANGCHAIN_PROJECT&quot;)<br><br># load Groq config<br>groqConfig = config[&quot;groq&quot;]<br><br>os.environ[&quot;GROQ_API_KEY&quot;] = groqConfig.get(&quot;GROQ_API_KEY&quot;)<br><br><br>from langchain_groq import ChatGroq<br><br># We use the llama3 8b model<br>groqmodel = ChatGroq(model=&quot;llama3-8b-8192&quot;)<br><br>from langchain_core.output_parsers import StrOutputParser<br>from langchain_core.prompts import ChatPromptTemplate<br><br>parser = StrOutputParser()<br>puzzle = puzzles.puzzles(&quot;tallBuilding&quot;)  # Based on user input pick a puzzle.<br>system_template = (<br>    &quot;solve the following puzzle. Please provide a {responseType} response.&quot;<br>)<br><br>prompt_template = ChatPromptTemplate.from_messages(<br>    [(&quot;system&quot;, system_template), (&quot;user&quot;, puzzle)]<br>)<br><br><br># prompt Template also implements runnable and can be easily chained.<br>chain = prompt_template | groqmodel | parser<br><br>str = chain.invoke({&quot;responseType&quot;: &quot;detailed&quot;})<br><br>print(str)<br><br># === OUTPUT ===<br>Here&#39;s a breakdown of the puzzle:<br><br>* The man lives in a 30-story building, which means the building has 30 floors above the ground floor.<br>* The puzzle states that the man jumps out of his window, which implies that he is on one of the upper floors.<br>* However, the puzzle also states that he survives the fall with no injuries.<br>* This seems impossible, as falling from a 30-story building would likely result in severe injuries or even death.<br>* But the puzzle is trying to trick us into assuming that the man jumped from a high floor. Instead, we need to think outside the box and consider the possibility that the man lived on the ground floor.<br><br>When we re-examine the puzzle from this new perspective, it becomes clear that the man simply stepped out of his front door and onto the ground, rather than falling from a great height. This is why he survived the &quot;fall&quot; with no injuries.</pre><h3>Conclusion</h3><p>This article was to give you an introduction to LangChain. You will appreciate the consistent programming interface it provides for programming against various LLMs.</p><p>This is useful as there are a plethora of LLMs today, and such a framework takes a lot of grunt out.</p><p>In subsequent articles, we will delve deeper into LangChain and develop an RAG application.</p><blockquote>Disclaimer</blockquote><blockquote>The logos and trademarks used in this presentation are the property of their respective owners. Their inclusion here does not imply endorsement or affiliation with our organization. We respect intellectual property rights and have used these logos only for illustrative and informational purposes only.</blockquote><h3>References:</h3><p><a href="https://learn.microsoft.com/en-us/azure/ai-services/openai/reference#chat-completions">Azure OpenAI Service REST API reference - Azure OpenAI</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=626ea02d5d5c" width="1" height="1" alt=""><hr><p><a href="https://techwitharun.com/langchain-i-626ea02d5d5c">RAG — I: LangChain Basics</a> was originally published in <a href="https://techwitharun.com/">Arun Saxena’s Tech Terrain</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Essential Python — XII]]></title>
            <link>https://techwitharun.com/essential-python-xii-7a66bd48b705?source=rss-99f899e3817a------2</link>
            <guid isPermaLink="false">https://medium.com/p/7a66bd48b705</guid>
            <category><![CDATA[python]]></category>
            <dc:creator><![CDATA[Arun Saxena]]></dc:creator>
            <pubDate>Wed, 03 Jul 2024 06:48:48 GMT</pubDate>
            <atom:updated>2024-07-03T06:48:48.371Z</atom:updated>
            <content:encoded><![CDATA[<h3>Essential Python — XII</h3><h4>Everything is an object</h4><p>Everything in a Python program is represented by objects or relations between them. Even code is represented by __code__ object.</p><p>Every object has 3 key properties:</p><ol><li>ID—Every object has an identity. We can get the identity using the <em>id(object)</em>. In the case of CPython, the ID represents the memory location where the object is stored. We can use the ‘<em>is’</em> operator to compare 2 objects. It&#39;s True when the ID is the same. The ID of an object is immutable.</li><li>Type — This determines the behavior of the object. We can use <em>type(object)</em> to get the type (metaclass). The type is also an object like everything else in Python. The eventual base type of all objects is <em>object</em>. Like ID, the type of an object is also not changeable.</li><li>Value — Objects have values that determine the state of the object. Immutable objects like int and string are immutable as their value cannot be changed. Meanwhile, others, like lists, are mutable. Some objects, like tuples like containers for other objects, are considered immutable as the pointer references they hold to the contained objects cannot be changed. Other container objects like lists are mutable because we can change these pointers.</li></ol><h3>Container Tyes</h3><h4>Sequences</h4><p>They are <strong>finite ordered nonunique sets indexed by positive integers starting with 0</strong>. The built-in function len() gets the number of elements in the sequence. They support retrieving objects using index <em>sequence[index]</em>. Since they are ordered, slicing is also supported by <em>sequence[start:end:step]</em>.</p><p>Strings, Tuples, and Bytes are immutable sequences.</p><p>Lists and ByteArrays are mutable sequences</p><h4>Sets</h4><p>These are unordered, finite and unique set of immutable objects. Since they are unordered, they are not indexable. They are iterable like sequences and the built-in function len() returns the number of elements. Sets are usually used when we need to perform a number of mathematical operations like intersection, union etc.</p><p>Normal sets are mutable and are created with the set() function</p><p>Frozen sets are immutable and created with frozenset()</p><h4>Mappings</h4><p>A finite set of mutable objects indexed by arbitrary keys. There is only one built-in type called dictionary. They represent key-value pairs. The keys can be any immutable type.</p><p><a href="https://nbviewer.org/urls/usefulfilesstorage.blob.core.windows.net/%24web/Untitled.ipynb">Notebook on nbviewer</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7a66bd48b705" width="1" height="1" alt=""><hr><p><a href="https://techwitharun.com/essential-python-xii-7a66bd48b705">Essential Python — XII</a> was originally published in <a href="https://techwitharun.com/">Arun Saxena’s Tech Terrain</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>