<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Prashant Sahu on Medium]]></title>
        <description><![CDATA[Stories by Prashant Sahu on Medium]]></description>
        <link>https://medium.com/@prashant9501?source=rss-a54d16fb2f45------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*mDNWwjh3-oRuF31h</url>
            <title>Stories by Prashant Sahu on Medium</title>
            <link>https://medium.com/@prashant9501?source=rss-a54d16fb2f45------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sat, 21 Nov 2026 16:42:10 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@prashant9501/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[LLM Observability with LangSmith — Part 2: Eval Gates, Prompt Versioning & Choosing Your Stack]]></title>
            <link>https://pub.towardsai.net/llm-observability-with-langsmith-part-2-eval-gates-prompt-versioning-choosing-your-stack-e607473320b5?source=rss-a54d16fb2f45------2</link>
            <guid isPermaLink="false">https://medium.com/p/e607473320b5</guid>
            <category><![CDATA[observability]]></category>
            <category><![CDATA[langsmith]]></category>
            <category><![CDATA[traceability]]></category>
            <category><![CDATA[agentic-ai]]></category>
            <category><![CDATA[langfuse]]></category>
            <dc:creator><![CDATA[Prashant Sahu]]></dc:creator>
            <pubDate>Sat, 13 Jun 2026 22:01:01 GMT</pubDate>
            <atom:updated>2026-06-13T23:48:40.560Z</atom:updated>
            <content:encoded><![CDATA[<h4><em>In Part 1, we made an agent replayable and audit-proof. Now we make it regression-proof — and then decide, with a clear head, whether LangSmith is even the right tool for your team.</em></h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*4lkz8-U6X9osBqOQ1hzgOA.png" /></figure><blockquote><strong>📚 This is Part 2 of a two-part series.</strong> In <a href="https://pub.towardsai.net/llm-observability-with-langsmith-part-1-tracing-everything-building-audit-grade-callbacks-c477719af691"><strong>Part 1</strong></a> , Meera — a GenAI engineer at AcmeAI — shipped a LangGraph support agent and got stopped by three questions from Sanjay, the head of risk. We answered two of them: with LANGSMITH_TRACING=true and a project name, every conversation became a <strong>replayable trace</strong> (question #1 — the Air Canada defense), and a custom BaseCallbackHandler gave compliance a <strong>tamper-evident, PII-redacted, vendor-independent audit log</strong> running alongside LangSmith (question #2 — the two-lane pattern). We also proved the @traceable decorator traces <em>any</em> Python function, LangChain or not. If you haven&#39;t read it, start there — this part reuses the agent built in <a href="https://pub.towardsai.net/llm-observability-with-langsmith-part-1-tracing-everything-building-audit-grade-callbacks-c477719af691">Part 1</a>.</blockquote><p>One question is still open, and it’s the sneakiest of the three:</p><blockquote><strong><em>“If somebody tweaks a prompt next quarter, will we catch the regression </em>before<em> it ships?”</em></strong></blockquote><p>Right now the answer is no. Let’s fix that — and then zoom all the way out to the industry playbook, the LangSmith-vs-Langfuse decision, and the rest of the 2026 tooling field.</p><h3>1. The Regression Question: Datasets, Evaluators, Experiments</h3><p>The failure Sanjay is describing is silent. Someone “improves” the routing prompt — adds one clarifying sentence — and refund questions quietly start routing to <em>General</em>, where the responder can’t see billing documents. No error, no alert, just gradually worse answers discovered weeks later through complaints.</p><p>Code has had the antidote for decades: regression tests. LangSmith brings the same discipline to LLM behavior with three nouns:</p><ul><li>A <strong>dataset</strong>: a versioned collection of (input → expected output) examples. Your test fixture.</li><li>An <strong>evaluator</strong>: a function that scores one (input, expected, actual) triple. Your assertion.</li><li>An <strong>experiment</strong>: one full run of the dataset through your system with evaluators attached. Your test-suite execution — with results stored, charted, and diffable in the dashboard.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*JLE4SvIkp_VA4Gixj6AoTA.png" /><figcaption>Dataset, evaluator, experiment</figcaption></figure><p>Meera builds a routing benchmark, including the trap cases that bite real routers:</p><pre>from langsmith import Client<br>from langsmith.evaluation import evaluate<br><br>client = Client()<br>DATASET_NAME = &quot;acmeai-routing-eval&quot;<br>examples = [<br>    {&quot;inputs&quot;: {&quot;customer_query&quot;: &quot;Do you ship to Singapore?&quot;},<br>     &quot;outputs&quot;: {&quot;expected_category&quot;: &quot;General&quot;}},<br>    {&quot;inputs&quot;: {&quot;customer_query&quot;: &quot;My GPU appliance throws CUDA errors after the latest firmware&quot;},<br>     &quot;outputs&quot;: {&quot;expected_category&quot;: &quot;Technical&quot;}},<br>    {&quot;inputs&quot;: {&quot;customer_query&quot;: &quot;Why was my Visa charged twice this month?&quot;},<br>     &quot;outputs&quot;: {&quot;expected_category&quot;: &quot;Billing&quot;}},<br>    {&quot;inputs&quot;: {&quot;customer_query&quot;: &quot;How do I update my saved payment method?&quot;},<br>     &quot;outputs&quot;: {&quot;expected_category&quot;: &quot;Billing&quot;}},<br>    {&quot;inputs&quot;: {&quot;customer_query&quot;: &quot;I want a refund right now, your product is unusable&quot;},<br>     &quot;outputs&quot;: {&quot;expected_category&quot;: &quot;Billing&quot;}},   # angry, but still Billing - sentiment handles escalation<br>    {&quot;inputs&quot;: {&quot;customer_query&quot;: &quot;Does your SDK support Python 3.12?&quot;},<br>     &quot;outputs&quot;: {&quot;expected_category&quot;: &quot;Technical&quot;}},<br>    {&quot;inputs&quot;: {&quot;customer_query&quot;: &quot;What are your weekend support hours?&quot;},<br>     &quot;outputs&quot;: {&quot;expected_category&quot;: &quot;General&quot;}},<br>]<br><br># Idempotent: create the dataset only if it doesn&#39;t already exist<br>try:<br>    ds = client.read_dataset(dataset_name=DATASET_NAME)<br>except Exception:<br>    ds = client.create_dataset(<br>        dataset_name=DATASET_NAME,<br>        description=&quot;Routing-accuracy benchmark for the support router agent.&quot;,<br>    )<br>    client.create_examples(<br>        dataset_id=ds.id,<br>        examples=[{&quot;inputs&quot;: e[&quot;inputs&quot;], &quot;outputs&quot;: e[&quot;outputs&quot;]} for e in examples],<br>    )</pre><p>Note that fifth example: <em>“I want a refund right now, your product is unusable.”</em> It encodes a real design decision from <a href="https://pub.towardsai.net/llm-observability-with-langsmith-part-1-tracing-everything-building-audit-grade-callbacks-c477719af691">Part 1</a> — the <strong>router</strong> should still say Billing; the <strong>sentiment</strong> check is what triggers escalation. Once that’s in the dataset, nobody can accidentally “fix” it away.</p><p>Next, the target under test and the evaluator. The target wraps just the classification node — unit-testing one decision, not the whole pipeline:</p><pre>def routing_target(inputs: dict) -&gt; dict:<br>    state = categorize_inquiry({&quot;customer_query&quot;: inputs[&quot;customer_query&quot;]})<br>    return {&quot;predicted_category&quot;: state[&quot;query_category&quot;]}<br><br>def correctness_evaluator(run, example) -&gt; dict:<br>    pred = (run.outputs or {}).get(&quot;predicted_category&quot;)<br>    expected = (example.outputs or {}).get(&quot;expected_category&quot;)<br>    return {<br>        &quot;key&quot;: &quot;routing_correctness&quot;,<br>        &quot;score&quot;: 1.0 if pred == expected else 0.0,<br>        &quot;comment&quot;: f&quot;pred={pred} expected={expected}&quot;,<br>    }</pre><p>And the experiment — one function call:</p><pre>results = evaluate(<br>    routing_target,<br>    data=DATASET_NAME,<br>    evaluators=[correctness_evaluator],<br>    experiment_prefix=&quot;router-baseline&quot;,<br>    metadata={&quot;agent&quot;: &quot;router-v1&quot;},<br>)</pre><pre>View the evaluation results for experiment: &#39;router-baseline-c7e21a4f&#39; at:<br>https://smith.langchain.com/o/.../datasets/.../compare?selectedSessions=...<br><br>7it [00:11,  1.58s/it]</pre><p>Follow that link and the dashboard shows the per-example breakdown. Meera’s baseline comes back at <strong>6/7–0.86</strong> — and the table tells her exactly which example bled:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1022/1*gOvfQD4K0847DxnUrsl3EA.png" /></figure><p>There it is — the exact trap the dataset was built to catch. When the customer is venting <em>about the product</em>, the model reads “unusable product” as a general complaint and misses that the actionable intent is a refund. In production this is invisible: the customer still gets <em>an</em> answer, just one generated without access to the refund-policy documents. In an experiment, it’s a red cell with a comment string. That’s the whole pitch for evals in one table row.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*xYhZh0Xk5qU1rKPYYLKGQQ.png" /><figcaption>The regression loop</figcaption></figure><p>The loop above is now Meera’s development workflow. The top row is mechanical: fixtures in, scores out, gate at the end. The two feedback arrows are where value compounds. The solid one — <em>re-run until green</em> — is the inner cycle she’s about to do (fix the prompt, re-run, compare experiments side by side). The dashed one, from “production incident” back into the dataset, is the long game: every bad answer your <a href="https://pub.towardsai.net/llm-observability-with-langsmith-part-1-tracing-everything-building-audit-grade-callbacks-c477719af691">Part 1</a> tracing catches in the wild gets distilled into a new example, which means <strong>the test suite grows in exactly the directions your system actually fails.</strong> Six months in, that dataset is the team’s institutional memory of every way the agent has ever embarrassed them.</p><p>One scope note: this evaluator is an exact-match check, which works because structured output constrains the labels. For free-text answers you’d add an <strong>LLM-as-judge</strong> evaluator (a strong model grading faithfulness against a rubric), and LangSmith’s <strong>online evaluators</strong> can score samples of <em>live production traffic</em> continuously, so drift (Scenario 4 from <a href="https://pub.towardsai.net/llm-observability-with-langsmith-part-1-tracing-everything-building-audit-grade-callbacks-c477719af691">Part 1</a>) shows up on a dashboard instead of in a complaint.</p><p>Now — about that failing prompt. Fixing it properly raises a bigger question.</p><h3>2. Prompt Versioning: Git for Prompts</h3><p>Here’s the uncomfortable question: <strong>where do your prompts actually live?</strong> If the honest answer is “in f-strings, scattered across the codebase, edited by whoever and deployed whenever” — that’s exactly how silent regressions are born, and exactly what a tribunal will subpoena, as Air Canada learned in <a href="https://pub.towardsai.net/llm-observability-with-langsmith-part-1-tracing-everything-building-audit-grade-callbacks-c477719af691">Part 1</a>.</p><p><a href="https://docs.langchain.com/langsmith/manage-prompts">LangSmith’s Hub</a> treats prompts as versioned, deployable artifacts. Every push creates an <strong>immutable commit</strong> — old versions are never overwritten and stay pullable by hash, forever — and <a href="https://changelog.langchain.com/announcements/prompt-tags-in-langsmith-for-version-control"><strong>tags</strong></a> like production or staging are movable pointers to commits, exactly like git branches.</p><p>Meera lifts the routing prompt out of the code and pushes it:</p><pre>from langchain_core.prompts import ChatPromptTemplate<br>from langsmith import Clientclient = Client()<br><br>PROMPT_NAME = &quot;acmeai-router-categorization&quot;<br>routing_prompt = ChatPromptTemplate.from_messages([<br>    (&quot;system&quot;, &quot;You are a customer support agent for an AI products and hardware company. &quot;<br>               &quot;Classify the customer query into exactly one of: Technical, Billing, General. &quot;<br>               &quot;Return only the category name.&quot;),<br>    (&quot;human&quot;, &quot;{customer_query}&quot;),<br>])<br>url = client.push_prompt(PROMPT_NAME, object=routing_prompt)<br>print(f&quot;Pushed → {url}&quot;)</pre><pre>Pushed → https://smith.langchain.com/prompts/acmeai-router-categorization/...</pre><p>That’s commit #1. Now the fix for the failing eval case — a v2 with one extra routing rule, pushed as a <em>new commit</em> of the same prompt:</p><pre>routing_prompt_v2 = ChatPromptTemplate.from_messages([<br>    (&quot;system&quot;, &quot;You are a customer support agent for an AI products and hardware company. &quot;<br>               &quot;Classify the customer query into exactly one of: Technical, Billing, General. &quot;<br>               &quot;Rule: complaints about charges, refunds, or payments are ALWAYS Billing, &quot;<br>               &quot;even when the customer is angry or insulting the product. &quot;<br>               &quot;Return only the category name.&quot;),<br>    (&quot;human&quot;, &quot;{customer_query}&quot;),<br>])<br><br>client.push_prompt(PROMPT_NAME, object=routing_prompt_v2)   # commit #2</pre><p>Does v2 actually fix the regression without breaking anything else? That’s not a matter of opinion anymore — it’s an experiment. The new target pulls the prompt from the Hub (note: the app code no longer contains prompt text at all):</p><pre>def routing_target_v2(inputs: dict) -&gt; dict:<br>    prompt = client.pull_prompt(PROMPT_NAME)            # latest commit<br>    chain = prompt | llm.with_structured_output(QueryCategory)<br>    result = chain.invoke({&quot;customer_query&quot;: inputs[&quot;customer_query&quot;]})<br>    return {&quot;predicted_category&quot;: result.categorized_topic}<br><br>results_v2 = evaluate(<br>    routing_target_v2,<br>    data=DATASET_NAME,<br>    evaluators=[correctness_evaluator],<br>    experiment_prefix=&quot;router-v2-refund-rule&quot;,<br>)</pre><pre>View the evaluation results for experiment: &#39;router-v2-refund-rule-9b3d51e0&#39; at:<br>https://smith.langchain.com/o/.../datasets/.../compare?selectedSessions=...<br><br>7it [00:10,  1.49s/it]</pre><p>The dashboard’s compare view puts both experiments side by side, and it’s worth looking at closely, because this view is the cultural artifact that changes how teams argue about prompts:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*t9NmRYv0qWHzEPUqoWTpJQ.png" /></figure><p>router-baseline at 0.86, router-v2-refund-rule at <strong>1.00</strong> — the amber refund row flips from red to green, and the six other rows hold steady. Both halves of that sentence matter: the flip proves the fix worked, and the steady rows prove the new rule didn&#39;t quietly break anything else (the failure mode of every &quot;small prompt tweak&quot; ever made). <em>That</em> — not &quot;it looks better to me&quot; — is the evidence that earns v2 the production tag. In the Hub UI, moving the :production tag onto commit #2 is one click, and the production app picks it up on its next pull:</p><pre>prompt = client.pull_prompt(f&quot;{PROMPT_NAME}:production&quot;)<br># or pin an exact commit hash for absolute reproducibility:<br># prompt = client.pull_prompt(f&quot;{PROMPT_NAME}:abc123de&quot;)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ElPTS59UgRZlATp5dX3_Yw.png" /><figcaption>Git for prompts: commits, tags, CI gate</figcaption></figure><p>The diagram is the whole operating model on one page. The rail across the top is what the Hub stores: immutable commits v1…v4, with :production and :staging as re-pinnable flags — promotion is moving a flag, not shipping a build. The pipeline underneath is what Meera just did manually, automated: a prompt edit becomes a commit, the commit triggers the eval suite in CI, a regression blocks the PR with the failing examples attached, and a pass moves the flag. Two operational footnotes are baked in: pulled prompts are <strong>cached</strong> (expect a few minutes of TTL after a re-tag before every running instance converges — pin commit hashes where you need determinism), and the rollback story is the killer feature. A bad prompt in production is fixed by moving the flag <em>backwards</em>: no build, no release train, no 2 a.m. deploy.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Ek8ij8gvigBl0Rq9j7JmqQ.png" /><figcaption>Commits are immutable; tags are pointers</figcaption></figure><p>The cultural shift lands quietly but permanently:</p><ul><li><strong>Prompt edits no longer require a deploy.</strong> Push, test, move the tag.</li><li><strong>Rollbacks are instant.</strong> Move the tag back one commit.</li><li><strong>Every change has an author, a diff, and a timestamp.</strong> When the auditor asks “who changed the routing instructions in March?”, the answer is a commit history, not a shrug.</li></ul><p><strong>Prompts now ship through the same gate as code.</strong> That’s the sentence that finally makes Sanjay smile — question #3, closed.</p><h3>3. Beyond the Support Desk: The Same Playbook Across Industries</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*x-Xp40qJtqKPWMzNlW2coQ.png" /><figcaption>Different stakes, same four moves</figcaption></figure><p>Everything so far used a support bot, but look at the four moves again — <em>trace everything, keep your own audit log, gate changes with evals, version your prompts</em> — and notice that nothing about them is support-specific. Here’s how the same stack earns its keep elsewhere.</p><p><strong>🏥 Healthcare — the symptom-triage assistant.</strong> A telehealth platform runs an intake bot that asks about symptoms and suggests urgency levels. Traces let clinical reviewers replay <em>exactly</em> why the bot said “routine appointment” instead of “urgent care” — which retrieval surfaced, which guideline was quoted. The custom callback is non-negotiable here: PHI must be scrubbed in-process before any trace leaves the network (HIPAA), and the JSONL log feeds the clinical-governance board. The eval dataset is a library of physician-written vignettes — “crushing chest pain radiating to left arm” must score urgent=1.0 on every model version, forever. A failed experiment blocks release like a failed unit test.</p><p><strong>🛒 E-commerce — the shopping copilot.</strong> A retailer’s product-Q&amp;A agent answers “will these boots survive a Norwegian winter?” from spec sheets and reviews. Tracing exposes the classic silent killer: the retriever returning the <em>men’s</em> boot specs for a <em>women’s</em> boot question. Cost telemetry per trace reveals that 4% of conversations consume 40% of spend (users pasting entire return-policy PDFs) — <a href="https://pub.towardsai.net/llm-observability-with-langsmith-part-1-tracing-everything-building-audit-grade-callbacks-c477719af691">Part 1</a>’s Scenario 3, found and fixed in a week. Before Black Friday, the team re-runs a 500-example eval suite against holiday prompt variants, and merchandising A/Bs a “warmer” tone by moving a Hub tag — zero engineering deploys.</p><p><strong>⚖️ Legal — the contract-review copilot.</strong> A firm’s associates use an agent that flags risky clauses in NDAs. Privilege means traces can’t leave the building — so they self-host (or run callback-only logging) with the exact same code. The eval dataset is partner-annotated contracts, and the evaluator checks clause-level recall: missing an uncapped-liability clause is a career-limiting false negative. Prompt commits matter for a subtler reason: when a client asks <em>“under what instructions did the AI review my contract in April?”</em>, the firm produces the exact prompt version, by hash.</p><p><strong>🎓 EdTech — the AI tutor.</strong> A math-tutoring app serves students aged 10–16. Online evaluators continuously score live traces for age-appropriateness and “did the tutor <em>explain</em> rather than hand over the answer.” The audit log doubles as a safety record for school districts. The Hub holds per-grade prompt variants (tutor-prompt:grade6, tutor-prompt:grade10) — pedagogy teams iterate on scaffolding without touching the codebase.</p><p>A quick map for everyone else:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3JdW_-sHRQ4XAMx1LGzYdw.png" /></figure><p>Different stakes, same four moves. The infrastructure doesn’t care whether the disaster is a misrouted refund, a missed liability clause, or a tutor handing a 12-year-old the answer key.</p><h3>4. LangSmith vs Langfuse: The Decision Matrix</h3><p>The question Meera gets most often from other teams: <em>“should we use LangSmith or Langfuse?”</em> It deserves a real answer, not a shrug — they’re the two most common finalists, and they genuinely optimize for different things.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*DyoOFjouAX6P_fV7pmS6sg.png" /><figcaption>Two valid philosophies, evenly matched</figcaption></figure><p><strong>Langfuse</strong> is the open-source counterweight: MIT-licensed core, self-hosting as a first-class citizen (one Docker Compose for Postgres + ClickHouse + the server), an SDK rebuilt around OpenTelemetry, and transparent unit-based pricing for its cloud. <strong>LangSmith</strong> is the vertically integrated, managed platform: the deepest LangChain/LangGraph integration on the market, plus the production toppings — alerts, online evaluators, automation rules, agent deployment — that Langfuse mostly leaves to you.</p><p>Here’s the matrix, distilled from <a href="https://langfuse.com/faq/all/langsmith-alternative">Langfuse’s own comparison</a>, <a href="https://www.langchain.com/resources/langsmith-vs-langfuse">LangChain’s counter-comparison</a>, and independent 2026 write-ups (<a href="https://www.zenml.io/blog/langfuse-vs-langsmith">ZenML</a>, <a href="https://techsy.io/en/blog/langfuse-vs-langsmith">TECHSY</a>) — pricing figures are mid-2026 cloud list prices, so verify before you budget:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*7T2WZLY2h4diW1cANx33kw.png" /><figcaption>LangSmith vs Langfuse comparison</figcaption></figure><p>And the decision rules, compressed:</p><ul><li><strong>All-in on LangChain/LangGraph, want a managed platform, value alerts + online evals + deployment in one bill?</strong> → LangSmith.</li><li><strong>Hard data-sovereignty requirements, open-source mandate, cost-sensitive at scale, or a polyglot stack standardized on OpenTelemetry?</strong> → Langfuse (self-hosted or cloud).</li><li><strong>Genuinely torn?</strong> The architecture in this series is deliberately portable: the audit-callback lane is vendor-neutral, @traceable ↔ @observe migrate mechanically, and both platforms speak OTel. Pick one, ship, and revisit in six months with real trace volumes — the switching cost is days, not quarters.</li></ul><h3>5. The Wider Field: Other Alternatives Worth Knowing</h3><p>LangSmith and Langfuse aren’t the only players — LLM observability has become a <a href="https://www.firecrawl.dev/blog/best-llm-observability-tools">multi-billion-dollar category</a>, and a few others deserve a look depending on your shape:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*vWPGUCxi8bhYFxmQg13aeQ.png" /><figcaption>Which tool to use when</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Xikce9K_jAK9TYnU1hqi5g.png" /><figcaption>Decision tree for choosing a stack</figcaption></figure><p>The tree compresses this whole section into three questions, asked in the order that actually matters. <strong>Sovereignty first:</strong> if traces can’t leave your network, you’re self-hosting, and the realistic shortlist is Langfuse, Phoenix, or MLflow — or writing the enterprise check for self-hosted LangSmith. <strong>Framework second:</strong> deep LangChain/LangGraph investment makes LangSmith’s zero-config integration genuinely hard to beat. <strong>Philosophy third:</strong> managed suite (LangSmith, Braintrust, Weave, Datadog) versus open-source-first (Langfuse, Phoenix, MLflow, Opik). And don’t skim past the footnote at the bottom of the tree — the audit-callback lane from <a href="https://pub.towardsai.net/llm-observability-with-langsmith-part-1-tracing-everything-building-audit-grade-callbacks-c477719af691">Part 1</a> belongs in <em>every</em> outcome box, because it’s the one component you’ll never migrate, never license, and never lose to an acquisition. (Ask a Helicone user.)</p><h3>6. When Traces Shouldn’t Leave the Building</h3><p>A practical privacy ladder, from lightest to heaviest control — each rung buys more sovereignty and costs more ops effort, so climb only as high as your data requires:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*CHA-Hzua4tf0bxwQNt2lSw.png" /><figcaption>The privacy ladder</figcaption></figure><ol><li><strong>Redact in the callback</strong> before anything egresses (the <a href="https://pub.towardsai.net/llm-observability-with-langsmith-part-1-tracing-everything-building-audit-grade-callbacks-c477719af691">Part 1</a> pattern; use Presidio or Comprehend for real coverage).</li><li><strong>Hide payloads platform-wide</strong> — set LANGSMITH_HIDE_INPUTS=true and LANGSMITH_HIDE_OUTPUTS=true, and LangSmith receives structure, timing, and token counts but never the content.</li><li><strong>Pick your region</strong> — EU tenants point LANGSMITH_ENDPOINT at https://eu.api.smith.langchain.com to keep traces in-region.</li><li><strong>Self-host</strong> — LangSmith in your VPC (enterprise), or pair the audit-callback pattern with self-hosted Langfuse/Phoenix/MLflow.</li><li><strong>Callback-only mode</strong> — for the most regulated workloads, switch external tracing off entirely and let your JSONL→SIEM pipeline be the single system of record. Your code doesn’t change; that’s the beauty of the two-lane design.</li></ol><h3>7. The Production Checklist</h3><p>Meera’s launch checklist, distilled from the whole series:</p><ol><li>✅ LANGSMITH_TRACING=true + a named project per app and environment (-dev, -staging, -prod).</li><li>✅ Custom BaseCallbackHandler on every entry point — JSONL, append-only, shipped to the SIEM, with log rotation.</li><li>✅ PII redaction <em>inside</em> the callback, before any egress; LANGSMITH_HIDE_INPUTS/OUTPUTS where content isn&#39;t needed.</li><li>✅ run_id/parent_run_id + session/tenant metadata on every event, so logs and traces correlate.</li><li>✅ @traceable on the non-LangChain functions that matter (DB lookups, external APIs) so trace trees have no blind spots.</li><li>✅ A versioned eval dataset per critical decision point — start with 20–30 examples, grow it from real incidents.</li><li>✅ evaluate() wired into CI; regressions block merges.</li><li>✅ Prompts in the Hub, pulled by tag; production pinned, rollback = re-tag.</li><li>✅ Online evaluators + alerts on live traffic, so drift pages you before customers do.</li></ol><h3>Key Takeaways (Part 2)</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*AZ-MId-Ae3QaTTMMl6-DQw.png" /><figcaption>Key takeaways</figcaption></figure><p>And if you keep exactly one artifact from this series, make it this poster — the whole playbook, both parts, on one page:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*g3gVk_5AOpBuo6JZ3wcSeQ.png" /><figcaption>The Four Moves of LLM Observability</figcaption></figure><p>And the story ends where stories like this should: Sanjay signs off, the agent ships, and three weeks later a customer claims the bot promised them a free GPU. Meera pulls the trace, reads the actual conversation, checks which prompt commit was live that day, and replies in four minutes flat.</p><p>That’s the whole point.</p><h3>Let’s Stay Connected</h3><p>If this two-parter saved you a debugging weekend — or surfaced a gap in your team’s LLM stack — I’d genuinely like to hear about it. I’m <strong>Prashant Sahu</strong>, and I train and consult on GenAI engineering: LLM observability and evaluation, RAG systems, and multi-agent architectures, including the 10-day (70-hour) corporate curriculum this series grew out of.</p><p>🔗 <strong>Connect with me on LinkedIn: </strong><a href="https://www.linkedin.com/in/prashantksahu/"><strong>linkedin.com/in/prashantksahu</strong></a> — say hi, share your observability war stories, or just tell me which part of this series you’d like a deeper dive on.</p><p>➕ <strong>Follow me here on Medium</strong> for the next articles in this series — Langfuse hands-on, the 3-layer agent-evaluation hierarchy, and PII redaction with fairness guardrails are all in the pipeline.</p><p><em>Missed the beginning? </em><a href="https://pub.towardsai.net/llm-observability-with-langsmith-part-1-tracing-everything-building-audit-grade-callbacks-c477719af691"><strong><em>Read Part 1 here</em></strong><em> </em></a><em>— observability fundamentals, zero-config tracing, tracing any Python function, and the audit-grade callback.</em></p><h3>References &amp; Further Reading (Part 2)</h3><ul><li><a href="https://docs.langchain.com/langsmith/manage-prompts">Managing prompts in LangSmith — push/pull, commits, tags</a> · <a href="https://changelog.langchain.com/announcements/prompt-tags-in-langsmith-for-version-control">Prompt tags changelog</a></li><li><a href="https://docs.langchain.com/langsmith/observability">LangSmith evaluation docs</a></li><li><a href="https://langfuse.com/faq/all/langsmith-alternative">Langfuse: “LangSmith alternative?” comparison</a> · <a href="https://www.langchain.com/resources/langsmith-vs-langfuse">LangChain: LangSmith vs Langfuse</a> · <a href="https://www.zenml.io/blog/langfuse-vs-langsmith">ZenML independent comparison</a></li><li>Tool-landscape roundups: <a href="https://mlflow.org/top-5-agent-observability-tools/">MLflow — top agent observability tools</a> · <a href="https://www.firecrawl.dev/blog/best-llm-observability-tools">Firecrawl — best LLM observability tools 2026</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e607473320b5" width="1" height="1" alt=""><hr><p><a href="https://pub.towardsai.net/llm-observability-with-langsmith-part-2-eval-gates-prompt-versioning-choosing-your-stack-e607473320b5">LLM Observability with LangSmith — Part 2: Eval Gates, Prompt Versioning &amp; Choosing Your Stack</a> was originally published in <a href="https://pub.towardsai.net/">Towards AI</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[LLM Observability with LangSmith — Part 1: Tracing Everything & Building Audit-Grade Callbacks]]></title>
            <link>https://pub.towardsai.net/llm-observability-with-langsmith-part-1-tracing-everything-building-audit-grade-callbacks-c477719af691?source=rss-a54d16fb2f45------2</link>
            <guid isPermaLink="false">https://medium.com/p/c477719af691</guid>
            <category><![CDATA[traceability]]></category>
            <category><![CDATA[agentic-ai]]></category>
            <category><![CDATA[observability]]></category>
            <category><![CDATA[langgraph]]></category>
            <category><![CDATA[langsmith]]></category>
            <dc:creator><![CDATA[Prashant Sahu]]></dc:creator>
            <pubDate>Sat, 13 Jun 2026 20:01:00 GMT</pubDate>
            <atom:updated>2026-06-13T23:49:13.393Z</atom:updated>
            <content:encoded><![CDATA[<h4><em>Your agent demoed perfectly. Then someone asked, “What exactly did the bot tell that customer on the 14th?” — and nobody could answer. This is the story of fixing that, end to end, with code.</em></h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*whJptwTJRXoyQg856_OVWQ.png" /><figcaption>*Dislaimer: All creatives in this article are AI-generated by the Author</figcaption></figure><p>Meera ships the demo on a Friday.</p><p>She’s a GenAI engineer at AcmeAI, a company that sells AI models and the hardware to run them. Her latest build is a customer-support agent: a LangGraph workflow that reads an incoming query, classifies it as <em>*Technical*</em>, <em>*Billing*</em>, or <em>*General*</em>, checks the customer’s sentiment, retrieves answers from a knowledge base, and replies. If the customer sounds furious, it skips the robot answer entirely and escalates to a human.</p><p>The demo lands. Leadership loves it. And then Sanjay, the head of risk, asks three questions that stop the launch cold:</p><p>1. “<strong>Can we replay any past customer interaction?”</strong> If a customer claims the bot promised them a free GPU, can we pull up exactly what happened?</p><p>2. “<strong>Do we have a tamper-evident audit log?”</strong> Every LLM call, every retrieval, every error — somewhere <em>*we*</em> control, not just a vendor dashboard?</p><p>3. “<strong>If somebody tweaks a prompt next quarter, will we catch the regression </strong><em>*before*</em><strong> it ships?”</strong> Or will we find out from angry customers?</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*xomkBaSFr9EDjM35DphlAA.png" /><figcaption>The three questions every head of risk eventually asks.</figcaption></figure><p>Meera realizes something every LLM engineer eventually learns: <strong>building the agent is the easy half. Operating it is the hard half.</strong></p><blockquote><strong>📚 This is Part 1 of a two-part series.**</strong> In this part, we cover what observability and traceability actually mean (and why LLM apps break every assumption your monitoring stack was built on), what LangSmith is and everything it can do, zero-config tracing for a real LangGraph agent, tracing <em>*any*</em> Python function, and building a compliance-grade audit callback — that’s Sanjay’s questions #1 and #2.</blockquote><blockquote>The <a href="https://pub.towardsai.net/llm-observability-with-langsmith-part-2-eval-gates-prompt-versioning-choosing-your-stack-e607473320b5"><strong>Part 2</strong></a><strong> </strong>takes on question #3: eval datasets and CI regression gates, prompt versioning with the Hub, the same playbook across six industries, and an honest LangSmith-vs-Langfuse decision matrix plus the wider 2026 tool landscape.</blockquote><p>Grab a coffee. Let’s go.</p><h3><strong>1. Observability and Traceability: What They Are, and Why You Can’t Skip Them</strong></h3><p>Let’s define the words before we sling the tools, because they get used loosely.</p><p><strong>Observability </strong>is a property of a system, not a product you buy: it’s the degree to which you can understand what’s happening <em>inside</em> the system purely from what it emits — its logs, metrics, and traces. A highly observable system lets you ask questions you didn’t think of in advance (“why are answers about refunds suddenly 30% longer for German users?”) and get answers without shipping new code.</p><p><strong>Traceability</strong> is narrower and sharper: the ability to follow <em>one specific request</em> end to end — every hop, every transformation, every sub-call, in order, with timing. If observability is the security-camera system for the whole building, a trace is the complete CCTV cut of one visitor’s walk through it.</p><p><strong>Monitoring</strong>, for completeness, is the dashboards-and-alerts layer you bolt on top: predefined checks for failures you already anticipated. Monitoring catches <em>known unknowns</em>. Observability is what saves you when the failure is one nobody predicted — which, with LLMs, is most of them.</p><p>Classic web apps had this figured out: APM tools, structured logs, distributed tracing. So why do LLM apps need their own version of the discipline? Because they violate the core assumption all of that tooling was built on — <strong>that failures are loud.</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*jtmDtk-kNPHiwGrvUf9bcA.png" /><figcaption>Why LLM apps need their own observability</figcaption></figure><p>The left half of that picture is the world your current tooling grew up in. When a traditional app breaks, it breaks <em>*theatrically*</em>: an exception fires, the response is a 500, the error-rate graph spikes, someone gets paged, and the stack trace points at the crime scene. The right half is the world you live in now. An LLM failure arrives wearing a 200 OK. It’s fluent, confident, grammatically lovely — and wrong. No exception, no spike, no page. Your logs swear everything is fine, and the first detector to fire is a customer, weeks later. The three boxes along the bottom are the answer this series builds, piece by piece:</p><ul><li><strong>traces</strong> so you can replay any request,</li><li><strong>evaluations </strong>so every change gets scored before it ships, and</li><li><strong>versioned prompts </strong>so every change has a diff and an undo button.</li></ul><p>If that still sounds like nice-to-have engineering hygiene, walk through five very real scenarios.</p><figure><img alt="Five ways LLM apps fail quietly" src="https://cdn-images-1.medium.com/max/1024/1*YcliE0p3pgNKKNPdDybPRg.png" /><figcaption>Five ways LLM apps fail quietly</figcaption></figure><p><strong>Scenario 1: The chatbot that invented a policy (real, and it went to a tribunal).</strong> In 2022, a passenger asked Air Canada’s website chatbot about bereavement fares. The bot confidently told him he could book a full-price ticket and claim the discount retroactively within 90 days — a policy that <em>did not exist</em>; the airline’s actual policy page said the opposite. He flew, applied for the refund, and was rejected. In February 2024, a British Columbia tribunal <a href="https://www.americanbar.org/groups/business_law/resources/business-law-today/2024-february/bc-tribunal-confirms-companies-remain-liable-information-provided-ai-chatbot/">ordered Air Canada to pay CA$812.02</a>, explicitly rejecting the airline’s argument that the chatbot was “a separate legal entity responsible for its own actions.” The ruling set the tone for every deployment since: <strong>your bot’s words are your company’s words.</strong> Now ask yourself — if a customer made that claim against your bot, could you produce the exact conversation, the documents the bot retrieved, and the prompt version that was live that day? Without tracing, the honest answer is no. You’d be litigating against a ghost.</p><p><strong>Scenario 2: The silent regression.</strong> A teammate “improves” a routing prompt — adds one clarifying sentence. Refund questions quietly start routing to <em>General</em>, where the responder can’t see billing documents. No error. No alert. Accuracy degrades for three weeks until a pattern emerges in complaints. The fix takes five minutes; <em>finding it</em> takes days — unless an eval suite had flagged it before merge. (<a href="https://pub.towardsai.net/llm-observability-with-langsmith-part-2-eval-gates-prompt-versioning-choosing-your-stack-e607473320b5">This scenario is the entire plot of Part 2.</a>)</p><p><strong>Scenario 3: The invisible money leak.</strong> Every LLM call has a price tag, which makes cost a first-class observability metric in a way traditional apps never needed. One team discovered that 4% of conversations were consuming 40% of their token spend — users pasting entire PDFs into the chat. That’s invisible in a monthly bill (“the OpenAI line item went up”) and obvious in five minutes of trace telemetry grouped by conversation.</p><p><strong>Scenario 4: Drift you didn’t deploy.</strong> Model providers update models. Sometimes behavior shifts subtly — formats change, refusal rates move, a model gets terser. Your code didn’t change, your prompts didn’t change, and yet Tuesday’s system is not Monday’s system. Without baseline evals re-run on a schedule, you discover drift the way you discover everything else: from users.</p><p><strong>Scenario 5: The 3 a.m. incident.</strong> Something went wrong with a customer interaction and it’s escalating — legal is asking questions. Your observability SaaS is having an outage, or your compliance team was never allowed to ship data there in the first place. What do you hand the auditor? If the answer isn’t “our own append-only log, on our own disk, with every event timestamped and correlated,” you have a governance gap, not just a tooling gap.</p><p>Five scenarios, one conclusion: <strong>an LLM app without observability isn’t a product — it’s a liability with a chat interface.</strong> Traces answer “what happened?”, evaluations answer “is it still good?”, and versioned prompts answer “what changed, and can we undo it?”</p><p>Now let’s meet the tool Meera reaches for.</p><h3><strong>2. What Is LangSmith, Exactly?</strong></h3><p><a href="https://docs.langchain.com/langsmith/observability">LangSmith</a> is the observability, evaluation, and prompt-engineering platform built by the LangChain team. It entered closed beta in mid-2023, <a href="https://www.langchain.com/blog/langsmith-ga">reached general availability in February 2024</a> alongside LangChain’s $25M Series A led by Sequoia Capital, and has since grown from “a debugger for LangChain” into what the company now positions as a full <strong>agent engineering platform</strong>.</p><p>A few facts worth knowing before you commit to it:</p><ul><li><strong>It’s framework-agnostic, despite the family name.</strong> The deepest magic (zero-config tracing) lights up with LangChain and LangGraph, but the `langsmith` SDK traces <em>any</em> Python or JavaScript code via a decorator — we’ll prove that in §4 — and the platform <a href="https://blog.langchain.com/end-to-end-opentelemetry-langsmith/">speaks OpenTelemetry in both directions</a>: it can ingest OTel spans from any stack and export its traces into the observability pipeline your org already runs (Grafana, Datadog, Jaeger).</li><li><strong>It’s closed-source SaaS at heart.</strong> There’s a free Developer tier (thousands of traces per month — enough for everything in this series), a per-seat Plus tier, and an Enterprise tier that unlocks self-hosting in your own VPC plus SSO/RBAC. There’s also an EU-region cloud for data-residency requirements. If open source is a hard requirement, that’s the Langfuse conversation — <a href="https://pub.towardsai.net/llm-observability-with-langsmith-part-2-eval-gates-prompt-versioning-choosing-your-stack-e607473320b5">Part 2</a> has a full decision matrix.</li><li><strong>It’s one product with four jobs.</strong> Most teams discover LangSmith as a tracing tool and only later realize the other three quadrants exist:</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/817/1*Fdz5AiMtoB0uX8k038q1lA.png" /></figure><p>This part of the series lives in the first row of that table; <a href="https://pub.towardsai.net/llm-observability-with-langsmith-part-2-eval-gates-prompt-versioning-choosing-your-stack-e607473320b5">Part 2</a> lives in the second and third. Let’s build.</p><h3><strong>3. Tracing: Two Environment Variables and You Can See Everything</strong></h3><p><strong>Setup: </strong>You need two API keys: one for your LLM provider (OpenAI here, but anything works) and one free LangSmith key from <a href="https://smith.langchain.com/">smith.langchain.com</a>.</p><pre>pip install &quot;langchain&gt;=1.0&quot; &quot;langchain-core&gt;=1.0&quot; &quot;langchain-openai&gt;=1.0&quot; \<br>&quot;langgraph&gt;=1.0&quot; langchain-chroma &quot;langsmith&gt;=0.4&quot; python-dotenv</pre><pre>import os<br>from dotenv import load_dotenv<br>load_dotenv() # expects OPENAI_API_KEY and LANGSMITH_API_KEY in .env<br><br># The entire tracing setup. Yes, really.<br>os.environ[&quot;LANGSMITH_TRACING&quot;] = &quot;true&quot;<br>os.environ[&quot;LANGSMITH_PROJECT&quot;] = &quot;acmeai-support-router&quot;<br><br># Optional: pin the endpoint (default is US; use eu.api.smith.langchain.com for EU residency)<br>os.environ.setdefault(&quot;LANGSMITH_ENDPOINT&quot;, &quot;https://api.smith.langchain.com&quot;)</pre><p>That’s the magic trick, and it’s worth pausing on: <strong>you haven’t imported LangSmith anywhere.</strong> With these environment variables set, every LangChain and LangGraph operation from this point on — every model call, every retriever hit, every graph node — reports itself to your LangSmith project automatically. The tracer rides LangChain’s internal callback system and uploads in batches, off the hot path, so your latency doesn’t pay for it.</p><p>Now let’s build the thing worth observing.</p><h4><strong>The Knowledge Base</strong></h4><p>Meera’s agent answers from a small product knowledge base. In production this would be your real docs; here, twelve documents keep the story self-contained:</p><pre>from langchain_core.documents import Document<br>knowledge_base = [<br># - - technical - -<br>{&quot;text&quot;: &quot;Our pre-trained models include vision (CLIP-style), speech (Whisper-style), and text (Llama-3 fine-tunes). They ship with example notebooks.&quot;, &quot;metadata&quot;: {&quot;category&quot;: &quot;technical&quot;}},<br>{&quot;text&quot;: &quot;On-prem deployment is supported via the AcmeAI Edge appliance - Kubernetes-based, runs Llama 3 70B on 2x H100.&quot;, &quot;metadata&quot;: {&quot;category&quot;: &quot;technical&quot;}},<br>{&quot;text&quot;: &quot;Hardware troubleshooting: if the GPU light blinks red, run acmectl diagnose - gpu; common cause is a loose NVLink bridge.&quot;, &quot;metadata&quot;: {&quot;category&quot;: &quot;technical&quot;}},<br>{&quot;text&quot;: &quot;AcmeAI SDK supports Python 3.10+, Node 20+, and Java 17. The REST API is OpenAPI 3.1 compliant.&quot;, &quot;metadata&quot;: {&quot;category&quot;: &quot;technical&quot;}},<br># - - billing - -<br>{&quot;text&quot;: &quot;We accept Visa, Mastercard, Amex, ACH bank transfer, and wire. Crypto is not supported.&quot;, &quot;metadata&quot;: {&quot;category&quot;: &quot;billing&quot;}},<br>{&quot;text&quot;: &quot;Invoices are emailed on the 1st of each month. To download past invoices, log in and visit Account → Billing → Invoices.&quot;, &quot;metadata&quot;: {&quot;category&quot;: &quot;billing&quot;}},<br>{&quot;text&quot;: &quot;You can update your billing info under Account → Billing → Payment Methods. Changes take effect immediately.&quot;, &quot;metadata&quot;: {&quot;category&quot;: &quot;billing&quot;}},<br>{&quot;text&quot;: &quot;Refunds are processed within 7 business days. We refund pro-rata on cancellation within 30 days of purchase.&quot;, &quot;metadata&quot;: {&quot;category&quot;: &quot;billing&quot;}},<br># - - general - -<br>{&quot;text&quot;: &quot;Our refund policy: full refund within 30 days, pro-rata thereafter. Contact billing@acmeai.example.&quot;, &quot;metadata&quot;: {&quot;category&quot;: &quot;general&quot;}},<br>{&quot;text&quot;: &quot;Standard shipping is 3–5 business days within the US. International shipping is 7–14 business days; duties not included.&quot;, &quot;metadata&quot;: {&quot;category&quot;: &quot;general&quot;}},<br>{&quot;text&quot;: &quot;Working hours: Mon–Fri 8am–8pm Eastern. Weekend support is available for Enterprise customers only.&quot;, &quot;metadata&quot;: {&quot;category&quot;: &quot;general&quot;}},<br>{&quot;text&quot;: &quot;You can reach support at support@acmeai.example or +1–555-ACME-HELP. Average first response: under 4 hours.&quot;, &quot;metadata&quot;: {&quot;category&quot;: &quot;general&quot;}},<br>]<br>docs = [Document(page_content=d[&quot;text&quot;], metadata=d[&quot;metadata&quot;]) for d in knowledge_base]</pre><p>Embed it into a Chroma vector store with cosine similarity:</p><pre>from langchain_openai import OpenAIEmbeddings<br>from langchain_chroma import Chroma<br><br>embeddings = OpenAIEmbeddings(model=&quot;text-embedding-3-small&quot;)<br>kbase_db = Chroma.from_documents(<br>    documents=docs,<br>    collection_name=&quot;knowledge_base&quot;,<br>    embedding=embeddings,<br>    collection_metadata={&quot;hnsw:space&quot;: &quot;cosine&quot;}, # default is euclidean - be explicit<br>    persist_directory=&quot;./knowledge_base&quot;,<br>)<br>retriever = kbase_db.as_retriever(<br>    search_type=&quot;similarity_score_threshold&quot;,<br>    search_kwargs={&quot;k&quot;: 3, &quot;score_threshold&quot;: 0.2},<br>)</pre><h4><strong>The Agent’s Brain: State, Classification, Sentiment</strong></h4><p>LangGraph agents pass a typed state dictionary between nodes:</p><pre>from typing import TypedDict, Literal<br>from pydantic import BaseModel<br>from langchain_openai import ChatOpenAI<br><br>class CustomerSupportState(TypedDict):<br>    customer_query: str<br>    query_category: str<br>    query_sentiment: str<br>    final_response: str<br><br>class QueryCategory(BaseModel):<br>    categorized_topic: Literal[&quot;Technical&quot;, &quot;Billing&quot;, &quot;General&quot;]<br><br>class QuerySentiment(BaseModel):<br>    sentiment: Literal[&quot;Positive&quot;, &quot;Neutral&quot;, &quot;Negative&quot;]<br><br>llm = ChatOpenAI(model=&quot;gpt-5-mini&quot;) # swap for any chat model you like</pre><p>The two Pydantic models matter more than they look. Paired with `with_structured_output`, the LLM <em>*cannot*</em> reply “I think this is probably a billing question 😊” — it must return one of the three allowed labels. Routers need guarantees, not vibes.</p><pre>def categorize_inquiry(state: CustomerSupportState) -&gt; CustomerSupportState:<br>    &quot;&quot;&quot;Classify the query into Technical / Billing / General.&quot;&quot;&quot;<br>    prompt = f&quot;&quot;&quot;Act as a customer support agent for an AI products and hardware company.<br>    Read the customer query and pick the best category: &#39;Technical&#39;, &#39;Billing&#39;, or &#39;General&#39;.<br><br>    - Technical: AI models, hardware, software, SDK issues<br>    - Billing: payments, invoices, refunds, purchases<br>    - General: policies, contact info, shipping, everything else<br><br>    Query:<br>    {state[&quot;customer_query&quot;]}<br>    &quot;&quot;&quot;<br>    result = llm.with_structured_output(QueryCategory).invoke(prompt)<br>    return {&quot;query_category&quot;: result.categorized_topic}<br><br><br>def analyze_inquiry_sentiment(state: CustomerSupportState) -&gt; CustomerSupportState:<br>    &quot;&quot;&quot;Classify sentiment as Positive / Neutral / Negative.&quot;&quot;&quot;<br>    prompt = f&quot;&quot;&quot;Act as a customer support agent. Read the customer query below and<br>    classify its sentiment as exactly one of: &#39;Positive&#39;, &#39;Neutral&#39;, or &#39;Negative&#39;.<br><br>    Query:<br>    {state[&quot;customer_query&quot;]}<br>    &quot;&quot;&quot;<br>    result = llm.with_structured_output(QuerySentiment).invoke(prompt)<br>    return {&quot;query_sentiment&quot;: result.sentiment}</pre><p>Sanity-check the sentiment node before wiring anything — same question, two emotional registers:</p><pre>analyze_inquiry_sentiment({&quot;customer_query&quot;: &quot;what is your refund policy?&quot;})<br># {&#39;query_sentiment&#39;: &#39;Neutral&#39;}<br><br>analyze_inquiry_sentiment({&quot;customer_query&quot;: &quot;what is your refund policy? I am fed up with this product and want my money back&quot;})<br># {&#39;query_sentiment&#39;: &#39;Negative&#39;}</pre><p>Same topic, opposite routing destinies — the first will get a polite RAG answer about refund windows; the second is heading straight to a human. That’s the whole escalation design in two lines of output.</p><h4><strong>The RAG Responders — One Factory, Three Nodes</strong></h4><p>Each responder filters the vector store to <em>*its own category*</em> using a metadata filter — the billing node physically cannot retrieve technical docs:</p><pre>from langchain_core.prompts import ChatPromptTemplate<br><br>RESPONSE_TEMPLATE = ChatPromptTemplate.from_template(<br>    &quot;&quot;&quot;Craft a clear and helpful {category} support response for the customer query below.<br>    Ground your answer in the provided knowledge base information.<br>    If the knowledge base does not contain the answer, say exactly:<br>    &quot;Apologies, I was not able to answer your question, please reach out to +1-555-ACME-HELP&quot;<br><br>    Customer Query:<br>    {customer_query}<br><br>    Relevant Knowledge Base Information:<br>    {retrieved_content}<br>    &quot;&quot;&quot;<br>)<br><br>def make_category_responder(category: str):<br>    &quot;&quot;&quot;Build a RAG responder node scoped to one KB category via metadata filter.&quot;&quot;&quot;<br>    def responder(state: CustomerSupportState) -&gt; CustomerSupportState:<br>        retriever.search_kwargs[&quot;filter&quot;] = {&quot;category&quot;: category}<br>        docs = retriever.invoke(state[&quot;customer_query&quot;])<br>        retrieved = &quot;\n\n&quot;.join(d.page_content for d in docs)<br>        chain = RESPONSE_TEMPLATE | llm<br>        reply = chain.invoke({<br>            &quot;category&quot;: category,<br>            &quot;customer_query&quot;: state[&quot;customer_query&quot;],<br>            &quot;retrieved_content&quot;: retrieved,<br>        }).content<br>        return {&quot;final_response&quot;: reply}<br>    return responder<br><br>generate_technical_response = make_category_responder(&quot;technical&quot;)<br>generate_billing_response   = make_category_responder(&quot;billing&quot;)<br>generate_general_response   = make_category_responder(&quot;general&quot;)<br><br>def escalate_to_human_agent(state: CustomerSupportState) -&gt; CustomerSupportState:<br>    &quot;&quot;&quot;Negative sentiment? No robot. A human will call.&quot;&quot;&quot;<br>    return {&quot;final_response&quot;: &quot;We&#39;re really sorry! Someone from our team will reach out to you shortly.&quot;}</pre><h4><strong>Wiring the Graph</strong></h4><pre>from langgraph.graph import StateGraph, END<br>from langgraph.checkpoint.memory import MemorySaver<br><br>def determine_route(state: CustomerSupportState) -&gt; str:<br>    if state[&quot;query_sentiment&quot;] == &quot;Negative&quot;:<br>        return &quot;escalate_to_human_agent&quot;<br>    elif state[&quot;query_category&quot;] == &quot;Technical&quot;:<br>        return &quot;generate_technical_response&quot;<br>    elif state[&quot;query_category&quot;] == &quot;Billing&quot;:<br>        return &quot;generate_billing_response&quot;<br>    return &quot;generate_general_response&quot;<br><br>graph = StateGraph(CustomerSupportState)<br>graph.add_node(&quot;categorize_inquiry&quot;, categorize_inquiry)<br>graph.add_node(&quot;analyze_inquiry_sentiment&quot;, analyze_inquiry_sentiment)<br>graph.add_node(&quot;generate_technical_response&quot;, generate_technical_response)<br>graph.add_node(&quot;generate_billing_response&quot;, generate_billing_response)<br>graph.add_node(&quot;generate_general_response&quot;, generate_general_response)<br>graph.add_node(&quot;escalate_to_human_agent&quot;, escalate_to_human_agent)<br><br>graph.set_entry_point(&quot;categorize_inquiry&quot;)<br>graph.add_edge(&quot;categorize_inquiry&quot;, &quot;analyze_inquiry_sentiment&quot;)<br>graph.add_conditional_edges(&quot;analyze_inquiry_sentiment&quot;, determine_route, [<br>    &quot;generate_technical_response&quot;, &quot;generate_billing_response&quot;,<br>    &quot;generate_general_response&quot;, &quot;escalate_to_human_agent&quot;,<br>])<br><br>for terminal in [&quot;generate_technical_response&quot;, &quot;generate_billing_response&quot;,<br>                 &quot;generate_general_response&quot;, &quot;escalate_to_human_agent&quot;]:<br>    graph.add_edge(terminal, END)<br><br>agent = graph.compile(checkpointer=MemorySaver())</pre><p>If you’re in a notebook, LangGraph will draw itself — <em>`agent.get_graph().draw_mermaid_png()`</em> — and what it draws is this topology:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*gwKyRZs-Q1Yv5im5BZH83A.png" /><figcaption>The support router as a LangGraph</figcaption></figure><p>The two indigo nodes at the top are LLM classifiers writing into the shared state; the diamond is plain Python reading that state — auditable logic, no model involved; the three teal terminals are the RAG responders, each fenced into its own slice of the knowledge base by that metadata filter; and the red terminal is the empathy hatch, where angry customers bypass the robot entirely. Keep this picture in mind, because in about thirty seconds every shape on it is going to reappear as a span in a trace tree.</p><h4><strong>The Payoff: Run It and Watch the Traces Appear</strong></h4><pre>def ask(query: str, session_id: str = &quot;demo&quot;) -&gt; str:<br>    final = None<br>    for event in agent.stream(<br>        {&quot;customer_query&quot;: query},<br>        {&quot;configurable&quot;: {&quot;thread_id&quot;: session_id}},<br>        stream_mode=&quot;values&quot;,<br>    ):<br>        final = event<br>    return final[&quot;final_response&quot;]<br><br>print(ask(&quot;Do you support pre-trained vision models?&quot;))   # → Technical path<br>print(ask(&quot;How do I download my last invoice?&quot;))           # → Billing path<br>print(ask(&quot;Can you tell me about your shipping policy?&quot;))  # → General path</pre><p>The billing answer comes back grounded in exactly the documents we seeded:</p><pre>You can download past invoices by logging in and going to Account → Billing →<br>Invoices. Invoices are also emailed on the 1st of each month, so check the inbox<br>associated with your account. If you don&#39;t see an invoice you expected, contact<br>support@acmeai.example and we&#39;ll resend it.</pre><p>Pleasant enough. But the real payoff is on the other screen: open <a href="https://smith.langchain.com/">smith.langchain.com</a>, click into the <em>`acmeai-support-router`</em> project, and three traces are waiting. Click the invoice one and you get the full waterfall — which looks like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*YsNKd-U1u6IPU8pTk4EjHA.png" /><figcaption>Anatomy of a LangSmith trace</figcaption></figure><p>Time flows left to right; each bar is a <strong>run</strong>, LangSmith’s unit of work, and the indentation <em>*is*</em> the call hierarchy. Three things jump out the first time you see your own agent like this. First, the final generation call eats 1.25 of the total 2.95 seconds — so when someone says “the bot feels slow,” this chart settles the optimize-retrieval-or-optimize-generation argument in five seconds flat (it’s generation, and the two classifier calls in front of it are the next suspects). Second, that little amber sliver: 140 ms for the Chroma retrieval, and clicking it shows <em>the exact three documents it returned</em> — which is precisely the evidence you need when the bot confidently cites the wrong spec. Third, quietly doing the bookkeeping: every bar carries token counts in and out (LangSmith turns those into cost per trace, per user, per day), plus a <em>`run_id`</em> and <em>`parent_run_id`</em> linking each run to its parent. File those two IDs away — they become important in §5.</p><p><strong>And that’s Sanjay’s question #1 answered,</strong> with a click instead of a forensic project. When a customer claims <em>“your bot told me the appliance supports water cooling”</em> — or invents a bereavement-fare policy — support pulls the trace and reads exactly what the retriever returned and what the model said.</p><h3><strong>4. “But My Code Isn’t All LangChain”: Tracing Any Python Function</strong></h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*I-1EyzfiORiZsMlODj0G7w.png" /><figcaption>One decorator, and plain Python shows up in the same trace tree</figcaption></figure><p>A question Meera gets asked constantly, so let’s answer it head-on: <strong>yes, LangSmith traces arbitrary Python — no LangChain required. </strong>The <em>`@traceable`</em> decorator turns any function into a run, and nested decorated calls assemble into the same parent-child tree automatically.</p><p>Here’s a fully standalone example — raw OpenAI SDK, a fake database call, plain Python orchestration. Not a LangChain import in sight:</p><pre>import os<br>from langsmith import traceable<br>from langsmith.wrappers import wrap_openai<br>from openai import OpenAI<br><br>os.environ[&quot;LANGSMITH_TRACING&quot;] = &quot;true&quot;<br>os.environ[&quot;LANGSMITH_PROJECT&quot;] = &quot;acmeai-standalone-demo&quot;<br><br># wrap_openai instruments the raw OpenAI client: every .create() becomes an LLM run<br>oai = wrap_openai(OpenAI())<br><br>@traceable(name=&quot;crm_lookup&quot;, run_type=&quot;tool&quot;)<br>def fetch_customer_tier(customer_id: str) -&gt; str:<br>    # pretend this hits your CRM / database<br>    return &quot;enterprise&quot; if customer_id.startswith(&quot;ENT&quot;) else &quot;standard&quot;<br><br>@traceable(name=&quot;ticket_summarizer&quot;)<br>def summarize_ticket(ticket_text: str) -&gt; str:<br>    response = oai.chat.completions.create(<br>        model=&quot;gpt-5-mini&quot;,<br>        messages=[{&quot;role&quot;: &quot;user&quot;, &quot;content&quot;: f&quot;Summarize this support ticket in one line: {ticket_text}&quot;}],<br>    )<br>    return response.choices[0].message.content<br><br>@traceable(name=&quot;handle_ticket&quot;, tags=[&quot;support&quot;, &quot;v2&quot;], metadata={&quot;team&quot;: &quot;acmeai-support&quot;})<br>def handle_ticket(customer_id: str, ticket_text: str) -&gt; dict:<br>    tier = fetch_customer_tier(customer_id)       # child run #1 (tool)<br>    summary = summarize_ticket(ticket_text)       # child run #2 -&gt; contains the LLM run<br>    return {&quot;tier&quot;: tier, &quot;summary&quot;: summary, &quot;priority&quot;: &quot;P1&quot; if tier == &quot;enterprise&quot; else &quot;P3&quot;}<br><br>handle_ticket(&quot;ENT-00451&quot;, &quot;The Edge appliance reboots whenever we run the vision pipeline at batch size 64.&quot;)</pre><p>Open the project and the tree reads exactly like the code: `handle_ticket` as the parent,<em> `crm_lookup`</em> and <em>`ticket_summarizer` </em>nested inside it, and the wrapped OpenAI call inside <em>that</em> — with token counts captured even though LangChain was never involved. Three details worth knowing:</p><ul><li><strong>`run_type`</strong> controls how a run renders and aggregates (<em>`tool`, `retriever`, `llm`, default `chain`</em>) — tag your custom vector-store calls as `retriever` and they get retrieval-style display.</li><li><strong>`tags` and `metadata`</strong> become filterable dimensions in the dashboard: <em>`metadata={“tenant_id”: …}` </em>is how multi-tenant apps slice cost and quality per customer.</li><li>The same decorator exists in the JS/TS SDK, and if you’re already standardized on OpenTelemetry, you can skip the decorator entirely and ship OTel spans straight in.</li></ul><p>This matters strategically: your observability isn’t welded to your framework choice. If you rip out LangChain next year, the tracing survives.</p><h3><strong>5. The Auditor’s Question: Custom Callbacks for a Tamper-Evident Log</strong></h3><p>Meera shows Sanjay the dashboard. He’s impressed — for about a minute. Then he leans in:</p><blockquote>“This is <strong><em>their</em> </strong>dashboard, on <strong><em>their</em> </strong>servers. If their cloud is down during an incident, what do we show the regulator? And does customer data leave our network before we’ve scrubbed it?”</blockquote><p>Fair. LangSmith is brilliant for <strong>debugging</strong>. But audit and compliance teams want guarantees a SaaS dashboard alone can’t give:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*zUidWCvNCdR1olaMRury3A.png" /></figure><p>The primitive that solves this is LangChain’s <strong>`BaseCallbackHandler` </strong>— the same machinery LangSmith itself rides on: lifecycle hooks that fire synchronously, in your process, on every LLM start/end, tool start/end, and error. Subclass it, and you decide what gets persisted, where, and in what shape.</p><p>Meera writes hers to emit <strong>JSON Lines</strong> — one JSON object per line, append-only. It’s the dullest format in computing, and that’s the point: <strong>`grep`</strong> reads it, <strong>`jq`</strong> reads it, pandas reads it, Splunk ingests it.</p><pre>import json<br>import re<br>import time<br>from datetime import datetime, timezone<br>from pathlib import Path<br>from typing import Any<br>from uuid import UUID<br><br>from langchain_core.callbacks import BaseCallbackHandler<br><br>AUDIT_LOG_PATH = Path(&quot;./audit.jsonl&quot;)<br><br>EMAIL_RE = re.compile(r&quot;[\w.+-]+@[\w-]+\.[\w.]+&quot;)<br><br>def redact(text: str) -&gt; str:<br>    &quot;&quot;&quot;Minimal demo redaction. In production, use a real PII engine<br>    (Microsoft Presidio, Amazon Comprehend) — emails alone won&#39;t cut it.&quot;&quot;&quot;<br>    return EMAIL_RE.sub(&quot;[EMAIL_REDACTED]&quot;, text)<br><br><br>class JsonLinesAuditHandler(BaseCallbackHandler):<br>    &quot;&quot;&quot;Append-only audit log of every LLM call, tool call, and error.<br><br>    One JSON object per line — grep-, jq-, pandas-, and Splunk-friendly.<br>    Designed for environments where an external SaaS alone can&#39;t be<br>    the system of record.<br>    &quot;&quot;&quot;<br><br>    def __init__(self, log_path: Path = AUDIT_LOG_PATH) -&gt; None:<br>        self.log_path = log_path<br>        self._llm_starts: dict[UUID, float] = {}<br>        self._tool_starts: dict[UUID, float] = {}<br><br>    def _emit(self, event: dict[str, Any]) -&gt; None:<br>        event[&quot;ts&quot;] = datetime.now(timezone.utc).isoformat()<br>        with self.log_path.open(&quot;a&quot;, encoding=&quot;utf-8&quot;) as f:<br>            f.write(json.dumps(event, default=str) + &quot;\n&quot;)<br><br>    def on_llm_start(self, serialized, prompts, *, run_id, parent_run_id=None, **kwargs):<br>        self._llm_starts[run_id] = time.perf_counter()<br>        self._emit({<br>            &quot;event&quot;: &quot;llm_start&quot;,<br>            &quot;run_id&quot;: str(run_id),<br>            &quot;parent_run_id&quot;: str(parent_run_id) if parent_run_id else None,<br>            &quot;model&quot;: (serialized or {}).get(&quot;id&quot;, [&quot;unknown&quot;])[-1],<br>            &quot;prompt_chars&quot;: sum(len(redact(p)) for p in prompts),<br>        })<br><br>    def on_llm_end(self, response, *, run_id, parent_run_id=None, **kwargs):<br>        latency_ms = (time.perf_counter() - self._llm_starts.pop(run_id, time.perf_counter())) * 1000<br>        usage = (response.llm_output or {}).get(&quot;token_usage&quot;, {}) if response.llm_output else {}<br>        self._emit({<br>            &quot;event&quot;: &quot;llm_end&quot;,<br>            &quot;run_id&quot;: str(run_id),<br>            &quot;parent_run_id&quot;: str(parent_run_id) if parent_run_id else None,<br>            &quot;latency_ms&quot;: round(latency_ms, 1),<br>            &quot;prompt_tokens&quot;: usage.get(&quot;prompt_tokens&quot;),<br>            &quot;completion_tokens&quot;: usage.get(&quot;completion_tokens&quot;),<br>            &quot;total_tokens&quot;: usage.get(&quot;total_tokens&quot;),<br>        })<br><br>    def on_llm_error(self, error, *, run_id, parent_run_id=None, **kwargs):<br>        self._emit({<br>            &quot;event&quot;: &quot;llm_error&quot;,<br>            &quot;run_id&quot;: str(run_id),<br>            &quot;parent_run_id&quot;: str(parent_run_id) if parent_run_id else None,<br>            &quot;error_type&quot;: type(error).__name__,<br>            &quot;error_msg&quot;: str(error)[:500],<br>        })<br><br>    def on_tool_start(self, serialized, input_str, *, run_id, parent_run_id=None, **kwargs):<br>        self._tool_starts[run_id] = time.perf_counter()<br>        self._emit({<br>            &quot;event&quot;: &quot;tool_start&quot;,<br>            &quot;run_id&quot;: str(run_id),<br>            &quot;parent_run_id&quot;: str(parent_run_id) if parent_run_id else None,<br>            &quot;tool&quot;: (serialized or {}).get(&quot;name&quot;),<br>            &quot;input_chars&quot;: len(input_str),<br>        })<br><br>    def on_tool_end(self, output, *, run_id, parent_run_id=None, **kwargs):<br>        latency_ms = (time.perf_counter() - self._tool_starts.pop(run_id, time.perf_counter())) * 1000<br>        self._emit({<br>            &quot;event&quot;: &quot;tool_end&quot;,<br>            &quot;run_id&quot;: str(run_id),<br>            &quot;parent_run_id&quot;: str(parent_run_id) if parent_run_id else None,<br>            &quot;latency_ms&quot;: round(latency_ms, 1),<br>            &quot;output_chars&quot;: len(str(output)),<br>        })<br><br>    def on_chain_error(self, error, *, run_id, parent_run_id=None, **kwargs):<br>        self._emit({<br>            &quot;event&quot;: &quot;chain_error&quot;,<br>            &quot;run_id&quot;: str(run_id),<br>            &quot;parent_run_id&quot;: str(parent_run_id) if parent_run_id else None,<br>            &quot;error_type&quot;: type(error).__name__,<br>            &quot;error_msg&quot;: str(error)[:500],<br>        })<br><br><br>audit_handler = JsonLinesAuditHandler()</pre><p>A few deliberate choices worth noticing:</p><ul><li><strong>`run_id` + `parent_run_id` on every event </strong>— the same UUIDs you met in the trace waterfall. Your forensics and your debugging share a correlation key.</li><li><strong>Character counts, not raw text, by default.</strong> The log proves <em>that</em> a prompt of a certain size went to a certain model at a certain time, without making the audit file itself a PII liability. Where you do log content, it goes through <em>`redact()`</em> first.</li><li><strong>Latency measured in-process</strong> with <em>`perf_counter`</em>, so the numbers hold up even if a trace upload is delayed or dropped.</li></ul><p>It also helps to see <strong><em>when</em> </strong>each of those overridden hooks actually fires. For one billing query through the router, the sequence and what each hook leaves behind in the file looks like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*B5nymir3rNzgL1InDv_gPQ.png" /><figcaption>Callback hook firing order for one query</figcaption></figure><p>Notice the greyed rows: <em>`on_chain_start`</em> and <em>`on_chain_end`</em> fire too — for the graph and for every node — but our handler deliberately lets them pass; LLM and error events are the audit-worthy moments. The amber row is an invitation: if your compliance story needs retrieval evidence (“which documents informed this answer?”), <em>`on_retriever_start` / `on_retriever_end`</em> are sitting there waiting for the same treatment. And the red strip at the bottom is the part auditors care about most — failures don’t vanish, they write a row with the error type, because <strong><em>the absence of a record is itself a finding</em> </strong>in most audit frameworks.</p><p>Attaching the handler costs one config key — and LangSmith keeps tracing alongside it. They’re independent layers:</p><pre>queries = [<br>    (&quot;audit-001&quot;, &quot;Do you support pre-trained vision models?&quot;),<br>    (&quot;audit-002&quot;, &quot;How do I download my last invoice?&quot;),<br>    (&quot;audit-003&quot;, &quot;I am furious — your hardware bricked itself overnight, refund NOW.&quot;),<br>]<br><br>for session_id, query in queries:<br>    events = agent.stream(<br>        {&quot;customer_query&quot;: query},<br>        config={<br>            &quot;configurable&quot;: {&quot;thread_id&quot;: session_id},<br>            &quot;callbacks&quot;: [audit_handler],          # ← audit log + LangSmith both fire<br>            &quot;metadata&quot;: {&quot;app&quot;: &quot;support-router&quot;, &quot;session_id&quot;: session_id},<br>        },<br>        stream_mode=&quot;values&quot;,<br>    )<br>    final = None<br>    for ev in events:<br>        final = ev<br>    print(f&quot;{session_id}: category={final[&#39;query_category&#39;]} sentiment={final[&#39;query_sentiment&#39;]}&quot;)</pre><pre>audit-001: category=Technical sentiment=Neutral<br>audit-002: category=Billing sentiment=Neutral<br>audit-003: category=Billing sentiment=Negative</pre><p>Read that third line carefully — it’s the design working as intended. The router still classified the furious message as <em>Billing</em> (it is about a refund), but the Negative sentiment overrode the route and the customer got a human, not a robot. Meanwhile, <strong><em>`audit.jsonl` </em></strong>quietly collected the paper trail. Six LLM calls happened across those three queries (two for the escalated one — its generation step never ran), and each produced a start and end event:</p><pre>import pandas as pd<br><br>events = [json.loads(line) for line in AUDIT_LOG_PATH.open() if line.strip()]<br>df = pd.DataFrame(events)<br><br>print(&quot;Total events:&quot;, len(events))<br>print(&quot;Total LLM tokens:&quot;, int(df[&quot;total_tokens&quot;].dropna().sum()))<br>print(&quot;Avg LLM latency (ms):&quot;, round(df.loc[df[&quot;event&quot;] == &quot;llm_end&quot;, &quot;latency_ms&quot;].mean(), 1))</pre><pre>Total events: 16<br>Total LLM tokens: 1732<br>Avg LLM latency (ms): 894.6</pre><p>And this is what a single line of the file looks like — what your SOC team greps at 3 a.m. when an incident lands:</p><pre>{&quot;event&quot;: &quot;llm_end&quot;, &quot;run_id&quot;: &quot;9f2c...&quot;, &quot;parent_run_id&quot;: &quot;b41a...&quot;,<br> &quot;latency_ms&quot;: 842.3, &quot;prompt_tokens&quot;: 187, &quot;completion_tokens&quot;: 9,<br> &quot;total_tokens&quot;: 196, &quot;ts&quot;: &quot;2026-06-12T07:14:55.103+00:00&quot;}</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*AFVuahAbfukxwg261UvnSQ.png" /><figcaption>Two independent observability lanes</figcaption></figure><p>Picture the full event stream flowing down two lanes from the same source. Down the engineer’s lane: the LangSmith tracer batching events to the cloud, feeding the dashboards, the replay UI, and (<a href="https://pub.towardsai.net/llm-observability-with-langsmith-part-2-eval-gates-prompt-versioning-choosing-your-stack-e607473320b5">in Part 2</a>) the experiments. Down the auditor’s lane: our handler, running synchronously in-process — and that placement is the entire compliance argument, because <em>`redact()` </em>runs <em>before any byte leaves the machine</em> — then the append-only file, then the SIEM with its retention policy. The two lanes share nothing but the callback events and those <em>`run_id`</em>s. LangSmith down? The auditor’s record is intact. Disk hiccup? LangSmith still has the traces. That’s Sanjay’s question #2 answered, and it earns the rule Meera writes on the team wiki:</p><blockquote><strong>Run both, always.</strong> LangSmith for the humans debugging at their desks. The custom callback for the auditor’s chain-of-custody. They’re complementary layers, not alternatives.</blockquote><h3><strong>Where We Are — and What’s Left</strong></h3><p>Take stock of what Meera has after one day of work.</p><p>Two environment variables bought her a flight recorder: every conversation with the support agent is now a replayable trace with per-step prompts, retrieved documents, token costs, and latencies — that’s the Air Canada defense, question #1. Eighty lines of `BaseCallbackHandler` bought her an institution-grade audit lane: append-only, PII-redacted before egress, vendor-independent, SIEM-ready — question #2. And the `@traceable` decorator means none of this is hostage to LangChain: the day the team rewrites the agent in a different framework, the observability comes along.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*BiHjHOpGW2-EbZfFgXfW9g.png" /></figure><p>But Sanjay’s third question is still open — and it’s the one that bites teams <em>after</em> launch: “<strong>if somebody tweaks a prompt next quarter, will we catch the regression before it ships?”</strong> Right now, the honest answer is still no. A well-meaning edit to the routing prompt tomorrow would sail straight into production, and Meera would learn about it from the complaints queue.</p><p><a href="https://pub.towardsai.net/llm-observability-with-langsmith-part-2-eval-gates-prompt-versioning-choosing-your-stack-e607473320b5"><strong>In the Part 2</strong></a><strong> </strong>we close that gap and then zoom out: we turn LangSmith into a regression-test framework with <strong>datasets, evaluators, and experiments</strong>; put prompts under real version control with the <strong>Hub</strong> (immutable commits, movable <strong><em>`:production` </em></strong>tags, CI-gated promotion — and an instant-rollback story your release manager will love); tour how the exact same four moves play out in <strong>healthcare, e-commerce, legal, and edtech</strong>; and finish with the decision everyone eventually faces — an honest <strong>LangSmith vs Langfuse matrix</strong> and a 30-second decision tree across the wider 2026 tooling landscape.</p><blockquote><em>If this saved you a future debugging weekend, follow me here on Medium so </em><a href="https://pub.towardsai.net/llm-observability-with-langsmith-part-2-eval-gates-prompt-versioning-choosing-your-stack-e607473320b5"><em>Part 2 </em></a><em>lands in your feed — </em><strong><em>and I’d genuinely love to hear your observability war stories in the comments</em></strong><em>.</em></blockquote><p>You can follow me and connect with me on LinkedIn as well <a href="https://www.linkedin.com/in/prashantksahu/?lipi=urn%3Ali%3Apage%3Ad_flagship3_notifications%3B2nlrrpPNRPWVs2Ma8rz1cQ%3D%3D"><strong>https://www.linkedin.com/in/prashantksahu</strong></a></p><h4><strong>References &amp; Further Reading (Part 1)</strong></h4><ul><li><a href="https://docs.langchain.com/langsmith/observability">LangSmith Observability docs</a></li><li><a href="https://www.langchain.com/blog/langsmith-ga">LangSmith GA announcement (Feb 2024)</a></li><li><a href="https://blog.langchain.com/end-to-end-opentelemetry-langsmith/">End-to-end OpenTelemetry support in LangSmith</a></li><li><a href="https://www.americanbar.org/groups/business_law/resources/business-law-today/2024-february/bc-tribunal-confirms-companies-remain-liable-information-provided-ai-chatbot/">Moffatt v. Air Canada — tribunal holds airline liable for chatbot’s answer (ABA summary)</a>]</li><li><a href="https://www.mccarthy.ca/en/insights/blogs/techlex/moffatt-v-air-canada-misrepresentation-ai-chatbot">McCarthy Tétrault case note</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c477719af691" width="1" height="1" alt=""><hr><p><a href="https://pub.towardsai.net/llm-observability-with-langsmith-part-1-tracing-everything-building-audit-grade-callbacks-c477719af691">LLM Observability with LangSmith — Part 1: Tracing Everything &amp; Building Audit-Grade Callbacks</a> was originally published in <a href="https://pub.towardsai.net/">Towards AI</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Open-Source LLM Observability with Langfuse: Tracing, Scoring, and Self-Hosting When the Data Can’t…]]></title>
            <link>https://medium.com/@prashant9501/open-source-llm-observability-with-langfuse-tracing-scoring-and-self-hosting-when-the-data-cant-fb5f446ba208?source=rss-a54d16fb2f45------2</link>
            <guid isPermaLink="false">https://medium.com/p/fb5f446ba208</guid>
            <category><![CDATA[ai-agent]]></category>
            <category><![CDATA[observability]]></category>
            <category><![CDATA[agentic-ai]]></category>
            <category><![CDATA[langsmith]]></category>
            <category><![CDATA[langfuse]]></category>
            <dc:creator><![CDATA[Prashant Sahu]]></dc:creator>
            <pubDate>Sat, 13 Jun 2026 07:57:19 GMT</pubDate>
            <atom:updated>2026-06-14T01:00:49.999Z</atom:updated>
            <content:encoded><![CDATA[<h3>Open-Source LLM Observability with Langfuse: Tracing, Scoring, and Self-Hosting When the Data Can’t Leave the Building</h3><p><em>A single support reply returned a clean HTTP 200 — and quietly invented a company policy that cost real subscriptions. This is the story of how you’d have caught it, and how to do it when you’re a bank and the trace data is itself regulated.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*lDBnE6O-FElSop73KSjXoQ.png" /></figure><p>In April 2025, a developer using the AI coding tool <strong>Cursor</strong> got mysteriously logged out every time they switched machines. They emailed support and got a crisp reply from “Sam,” explaining that this was expected: Cursor was <em>“designed to work with one device per subscription as a core security feature.”</em></p><p>There was no such policy. “Sam” was an AI support bot, and it had invented the rule out of thin air. Users took the fabricated policy at face value and started cancelling — publicly, on Reddit and Hacker News — before anyone at the company knew anything was wrong. The co-founder eventually waded into the thread: <em>“Hey! We have no such policy. You’re of course free to use Cursor on multiple machines. Unfortunately, this is an incorrect response from a front-line AI support bot.”</em> The logouts turned out to be an unrelated session bug; the churn was caused by the hallucinated answer about them.</p><p>Here’s the part worth sitting with. Every one of those replies returned successfully. The API said 200 OK. Latency was fine. No exception was thrown, no alert fired, no dashboard turned red. By every signal your traditional monitoring stack tracks, the system was <em>healthy</em> — while it was busy talking customers out the door.</p><p>That gap is the whole reason this article exists.</p><p>Meet Ravi, a senior platform engineer on the GenAI team at a large regulated retail bank. He’s just demoed a customer-support agent that drafts empathetic, on-brand replies, and the room is impressed. Then Priya, his VP of Engineering — who also owns the bank’s AI-governance conversations with the regulator — asks the three questions that decide whether this thing ever ships:</p><ol><li><strong>“Our production agents touch customer PII that, by law, can’t leave our VPC. Can we even <em>run</em> observability under that constraint — or do we fly blind in prod?”</strong></li><li><strong>“Tracing tells me an answer came <em>back</em>. How do I know it was any <em>good</em> — and how do I get paged when quality quietly drops?”</strong></li><li><strong>“We already pay for LangSmith in dev. Do we standardize on one tool or run both — and how do we decide that <em>once</em> instead of re-litigating it every project?”</strong></li></ol><p>These aren’t gotchas. They’re the real checklist for putting generative AI in front of customers in a regulated business, and the honest answer to all three is <strong>Langfuse</strong> — the open-source LLM observability platform that you can run entirely inside your own walls.</p><blockquote><strong><em>📚 This is Part 1 of a two-part series.</em></strong><em> Here we answer Priya’s first two questions: we’ll wire Langfuse into a customer-support agent three different ways, push LLM-as-Judge quality scores back onto live traces, build the dashboard views an on-call engineer actually uses, and then self-host the whole thing so customer data never leaves the bank’s VPC.</em></blockquote><blockquote><a href="https://medium.com/PART_2_URL"><strong><em>Part 2</em></strong></a><em> takes on the third question: prompt management, the LangSmith-vs-Langfuse decision matrix, the wider field of alternatives, and a production checklist you can hand to your team.</em></blockquote><h3>1. Why “It Returned 200” Tells You Almost Nothing</h3><p>Traditional observability was built for software that fails <em>loudly</em>. A REST endpoint either returns a 200 or it doesn&#39;t; a database query either completes or throws. Latency, error rate, CPU, uptime — Datadog, New Relic, and CloudWatch are superb at all of it. But they share one blind assumption: that a successful response is a <em>correct</em> one.</p><p>LLM systems break that assumption on day one. A hallucinated answer still returns 200 OK. A subtly wrong retrieval, a prompt regression after a deploy, a tone that drifts from &quot;empathetic&quot; to &quot;robotic&quot; — none of it shows up as an error. The system is, by infrastructure standards, perfectly healthy while it&#39;s quietly doing the wrong thing.</p><p>So we need a different vocabulary. Three terms do most of the work, and they’re worth pinning down because people use them loosely:</p><ul><li><strong>Observability</strong> is the broad capability of understanding what’s happening <em>inside</em> your system from its outputs. For LLM apps, that means being able to ask — after the fact — “what exactly did this request do, and was the result any good?”</li><li><strong>Tracing</strong> is the specific technique that makes observability possible: recording the full lifecycle of a request as it flows through every step — each prompt, model call, retrieval, and tool invocation — preserving the causal “this led to that” relationships between them.</li><li><strong>Evaluation</strong> (or <strong>scoring</strong>) is the layer on top: attaching a <em>judgement</em> of quality to those traces, so you can tell not just what happened but whether it was good — at scale, continuously, not by reading transcripts one at a time.</li></ul><p>The first figure is the entire argument in one picture.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*9HVoV-Sv9FiSHm3Xz0qE_g.png" /><figcaption>Figure describing LLM observability as tracing and scoring the content, not the status code.</figcaption></figure><p>What jumps out is how little overlap there is between the two halves. The monitoring stack is reporting on the <em>envelope</em> — did a response come back, how fast, did it error. The thing that actually mattered lived in the <em>contents</em> of that envelope, and nothing on the left was ever going to surface it. LLM observability is the discipline of instrumenting the contents: capture the prompts and completions as a trace, attach a quality score to each one, and <em>then</em> your dashboards can finally turn red for the right reasons.</p><p>It’s not a hypothetical failure mode. A few of the ways it has actually played out:</p><ul><li><strong>The invented policy</strong> (the Cursor story above). A support bot fabricated a subscription rule; customers believed it and cancelled before any human saw the reply. Verifiable, recent, and a clean illustration that “no error” is not “no problem.”</li><li><strong>The expensive apology.</strong> In early 2024, Air Canada’s website chatbot told a grieving passenger he could claim a bereavement discount <em>after</em> booking — a refund policy the airline didn’t actually have. When the airline tried to disown its own bot’s answer, a Canadian tribunal held it liable and ordered it to pay. The chatbot’s reply, again, returned perfectly successfully.</li><li><strong>The silent quality drift.</strong> A prompt tweak ships on Friday. Replies are still grammatical, still on-topic, still 200 OK — but a touch colder, a touch more likely to miss the customer&#39;s real question. CSAT sags two weeks later and nobody can point to the cause, because nothing was ever <em>measured</em>.</li><li><strong>The retrieval that rotted.</strong> A document gets re-indexed, embeddings shift, and a RAG pipeline starts grounding answers on the wrong passages. Every call succeeds. The answers are confidently wrong.</li><li><strong>The cost surprise.</strong> One tenant’s traffic quietly 10×’s after a feature flag flips. No error — just a bill at month-end and no per-customer breakdown to explain it.</li></ul><p>Every one of these is invisible to status codes and visible to traces. The fix is the same in all five cases: see the contents, score the contents, and watch the score over time.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*yp_4X80kjmElpa4V-3f6HQ.png" /><figcaption>An x-ray view metaphor: a stylized black box on the left, an x-ray plate on the right revealing the nested skeleton of prompts, model calls, and tool steps inside a single request</figcaption></figure><h3>The four nouns you have to memorize</h3><p>Before any code, internalize the data model. Both Langfuse and its main rival, LangSmith, use the same four-noun hierarchy, and once it clicks, every dashboard, filter, and alert is obvious. Get it wrong and you’ll fight the tool forever.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*TWJrNDbI3q8dDsChVOP-aA.png" /><figcaption><strong>Trace anatomy diagram: </strong>a Trace at the top contains nested Spans, one Span is specialized as a Generation carrying model, prompt, completion, tokens and cost, and a Score is attached to the side carrying a name, value and data type</figcaption></figure><p>A <strong>Trace</strong> is one top-level unit of work — a single user request, one job, one experiment iteration. A <strong>Span</strong> is any step <em>inside</em> that trace: a function call, a database query, a retrieval. Spans nest arbitrarily deep, so a parent function calling a child function produces a parent → child span tree. A <strong>Generation</strong> is a <em>specialized span</em> representing exactly one LLM call — it carries the extra fields you care about most: the model name, the prompt, the completion, the token counts, and the cost. And a <strong>Score</strong> is a numeric, categorical, or boolean <em>judgement</em> attached to a trace, span, or generation — empathy = 0.82, safety_pass = true, rating = GOOD.</p><p>The reason this matters is that your data model decides which questions you can answer. A per-customer cost dashboard is just <em>“sum </em><em>trace.cost where </em><em>trace.userId = X.&quot;</em> A regression alert is just <em>&quot;average </em><em>score.empathy this week versus last.&quot;</em> If the structure isn&#39;t there, the question is unanswerable — no amount of clever querying can recover a dimension you never recorded.</p><p>That’s the answer to “what is observability and why.” Now, the answer to “why Langfuse, specifically.”</p><h3>2. What Is Langfuse, Exactly?</h3><p>Langfuse is an open-source LLM engineering platform — observability, evaluation, prompt management, and datasets — that you can use as a managed cloud service <em>or</em> self-host in minutes inside your own infrastructure. It came out of Y Combinator’s W23 batch, publicly launched in mid-2023, and raised a $4M seed round in late 2023 led by Lightspeed Venture Partners with General Catalyst and Y Combinator. The team builds from Berlin and San Francisco.</p><p>A few facts shape whether it’s the right call for a regulated team:</p><ul><li><strong>It’s genuinely open source.</strong> The core is <strong>MIT-licensed</strong> — the most permissive of the common licenses, with no “source-available” asterisks. For a procurement team that mandates an OSS license, that’s a hard requirement satisfied rather than negotiated.</li><li><strong>It split its storage on purpose.</strong> Since the v3 release (December 2024), Langfuse runs on <strong>Postgres for metadata and ClickHouse for the high-volume event/trace data.</strong> That separation is what lets it ingest verbose LLM traces at scale and keep them cheaply for long retention — exactly what you need when the audit trail itself has to survive for years.</li><li><strong>It got acquired — and stayed open.</strong> In <strong>January 2026, ClickHouse acquired Langfuse</strong> (announced alongside ClickHouse’s $400M Series D at a $15B valuation). The rationale was unusually clean: Langfuse already ran <em>on</em> ClickHouse, so the technical fit was obvious. Both companies publicly committed to keeping Langfuse <strong>MIT-licensed and self-hostable</strong>. By the end of 2025 the project had passed 20,000 GitHub stars, 26M+ monthly SDK installs, and 2,000+ paying customers, and was in use at 19 of the Fortune 50.</li></ul><p>That last point deserves a clear-eyed footnote. Open-source acquisitions have a mixed history — MySQL, Redis, and Elastic all became cautionary tales about license promises that quietly eroded once the press cycle ended. The MIT commitment here is explicit and, so far, kept (the team has shipped openly since, including a faster “observations-centric” data model in early 2026). But if your whole reason for choosing Langfuse is the open license, watch the license, not the press release. <em>(All adoption and funding numbers are early-2026 figures — verify the current state before you put them in a board deck.)</em></p><p>Here’s how its capabilities map onto the three pillars that matter for production:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*qb-kafmSKw8Sc0gn-vt7MA.png" /></figure><p>The “Operate” row is the one that makes Priya lean forward, and it’s where this part of the series ends. But it only matters if instrumenting your code is painless — so let’s start there.</p><h3>3. Three Ways to Wire It In (Use All Three)</h3><p>Langfuse offers three integration patterns, and the instinct to pick one is wrong. In a real codebase they <em>compose</em>: an outer decorated function opens the trace, a LangChain chain inside it nests as a child, and a raw OpenAI call further in nests as another child — all on one trace. Here’s the map before the code.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*os6P36zR93xKWAc2yCr9ug.png" /><figcaption>Three-lane diagram mapping each integration pattern to its exact import and when to use it: the observe decorator for plain Python functions, the LangChain CallbackHandler for any Runnable or LangGraph, and the langfuse.openai drop-in for direct SDK calls</figcaption></figure><p>The decision is mechanical: if it’s your own glue code, decorate it; if it’s wrapped in a LangChain Runnable, pass a callback; if it calls the OpenAI SDK directly, swap the import. Let&#39;s take them in order.</p><p>First, a note on setup. Langfuse needs three environment variables — a public key, a secret key, and a host URL — which you get from a Langfuse Cloud project (free tier, no credit card) or your self-hosted instance. If the keys are missing, the decorated calls and the callback handler simply <strong>no-op silently</strong> rather than crash, which is a deliberately friendly default for local development.</p><pre>import os<br># Local pattern: load a .env if python-dotenv is around, else rely on the shell/venv.<br>try:<br>    from dotenv import load_dotenv<br>    load_dotenv()<br>except ImportError:<br>    pass<br><br>for key in [&quot;OPENAI_API_KEY&quot;, &quot;ANTHROPIC_API_KEY&quot;,<br>            &quot;LANGFUSE_PUBLIC_KEY&quot;, &quot;LANGFUSE_SECRET_KEY&quot;, &quot;LANGFUSE_HOST&quot;]:<br>    print(f&quot;{key}: {&#39;set&#39; if os.environ.get(key) else &#39;MISSING&#39;}&quot;)<br><br># Defaults to the EU cloud host when unset; point this at http://localhost:3000 when self-hosting.<br>os.environ.setdefault(&quot;LANGFUSE_HOST&quot;, &quot;https://cloud.langfuse.com&quot;)</pre><pre>OPENAI_API_KEY: set<br>ANTHROPIC_API_KEY: set<br>LANGFUSE_PUBLIC_KEY: set<br>LANGFUSE_SECRET_KEY: set<br>LANGFUSE_HOST: set</pre><h3>3.1 The @observe() decorator — for your own code</h3><p>This is the most fundamental primitive. Slap @observe() on any function, sync or async, and Langfuse captures its inputs, outputs, exceptions, and latency. Nesting is automatic via Python&#39;s contextvars: a decorated function calling another decorated function produces a parent → child span tree with zero plumbing.</p><pre>from langfuse import observe, get_client<br>langfuse = get_client()<br><br>@observe()<br>def lookup_account(account_id: str) -&gt; dict:<br>    &quot;&quot;&quot;Pretend we hit a CRM API; return a stub.&quot;&quot;&quot;<br>    return {&quot;account_id&quot;: account_id, &quot;tier&quot;: &quot;gold&quot;, &quot;open_tickets&quot;: 2}<br><br>@observe()<br>def summarise_account(account_id: str) -&gt; str:<br>    info = lookup_account(account_id)          # ← becomes a child span automatically<br>    return f&quot;Account {info[&#39;account_id&#39;]} ({info[&#39;tier&#39;]}) — {info[&#39;open_tickets&#39;]} open tickets.&quot;<br><br>print(summarise_account(&quot;CUST-00042&quot;))<br><br># Flush so spans land server-side before a short-lived script exits.<br>langfuse.flush()</pre><pre>Account CUST-00042 (gold) — 2 open tickets.<br>→ In Langfuse: Traces → recent → one trace, with lookup_account nested under summarise_account.</pre><p>No imports of internal SDK objects, no try/finally cleanup. The decorator owns the whole lifecycle: it opens a trace (or a child span if one&#39;s already active), records the arguments, captures the return value or exception, computes latency, and closes the span. That single flush() at the end matters in scripts and serverless handlers — Langfuse batches events in the background, and a process that exits immediately can drop the last batch.</p><h3>3.2 The LangChain CallbackHandler — for Runnables and LangGraph</h3><p>Anything wrapped in a LangChain Runnable — a chain, an agent, a LangGraph graph — gets traced by passing a CallbackHandler in the config. Every step the runnable executes (prompt formatting, the LLM call, retrieval, tool calls) shows up as a span on the trace.</p><p>The notebook this article grew out of rebuilt the same three-line support chain in four different cells. We won’t. One small factory function gives us a single place to construct it — and keeping it a plain function rather than a class is deliberate: there’s no state to hold, so a class would be ceremony for its own sake.</p><pre>from langfuse.langchain import CallbackHandler<br>from langchain_core.prompts import ChatPromptTemplate<br>from langchain_core.output_parsers import StrOutputParser<br>from langchain_anthropic import ChatAnthropic<br><br>def support_chain(system: str, model: str = &quot;claude-haiku-4-5-20251001&quot;):<br>    &quot;&quot;&quot;One place to build the support chain, so we don&#39;t repeat it four times.&quot;&quot;&quot;<br>    prompt = ChatPromptTemplate.from_messages([(&quot;system&quot;, system), (&quot;human&quot;, &quot;{question}&quot;)])<br>    return prompt | ChatAnthropic(model=model, temperature=0) | StrOutputParser()<br><br>chain = support_chain(&quot;You are a concise customer-support copywriter. Reply in &lt;=2 sentences.&quot;)<br><br>answer = chain.invoke(<br>    {&quot;question&quot;: &quot;How long does international shipping take?&quot;},<br>    config={&quot;callbacks&quot;: [CallbackHandler()],<br>            &quot;metadata&quot;: {&quot;lab&quot;: &quot;9.2&quot;, &quot;pattern&quot;: &quot;langchain-callback&quot;}},<br>)<br><br>print(answer)<br>langfuse.flush()</pre><pre>International shipping typically takes 7–14 business days depending on the destination and customs.<br>You&#39;ll get a tracking link by email as soon as your order ships.</pre><p>That metadata dict rides along on the trace, so you can later filter the dashboard by lab = 9.2 or pattern = langchain-callback. The same CallbackHandler instance can be reused across many invocations; for LangGraph it&#39;s identical — pass the handler to the graph invocation and every node becomes a span.</p><h3>3.3 The OpenAI drop-in wrap — for raw SDK calls</h3><p>Some code paths call the OpenAI SDK directly: a non-LangChain microservice, a quick prototype, a Bedrock-style wrapper. For those, you change exactly one line — the import — and every openai.chat.completions.create() call becomes a Langfuse <em>generation</em> automatically.</p><pre># Replace `import openai` with the Langfuse drop-in. Same call signatures, auto-instrumented.<br>from langfuse.openai import openai<br><br>completion = openai.chat.completions.create(<br>    model=&quot;gpt-5-mini&quot;,<br>    messages=[<br>        {&quot;role&quot;: &quot;system&quot;, &quot;content&quot;: &quot;You are a triage agent. Reply with just the urgency: low/medium/high.&quot;},<br>        {&quot;role&quot;: &quot;user&quot;,   &quot;content&quot;: &quot;My production database is down and customers are complaining.&quot;},<br>    ],<br>    metadata={&quot;lab&quot;: &quot;9.2&quot;, &quot;pattern&quot;: &quot;openai-wrap&quot;},  # Langfuse-only kwarg, stripped before OpenAI sees it<br>)<br><br>print(&quot;Urgency:&quot;, completion.choices[0].message.content)<br><br>langfuse.flush()</pre><pre>Urgency: high</pre><p>The trick worth knowing: that metadata= kwarg is intercepted by the wrapper and removed before the request reaches OpenAI&#39;s API, so it&#39;s safe to leave in production code. This is how you tag traces with tenant_id, feature_flag, or model_version on a code path that has never heard of LangChain.</p><h3>3.4 All three on one trace</h3><p>Now the payoff. A real handler opens the trace with @observe(), calls a LangChain chain (the callback nests it as a child span), then drops to the wrapped OpenAI SDK for a follow-up classification. Three SDKs, one trace.</p><pre>@observe(name=&quot;handle-customer-question&quot;)<br>def handle(question: str) -&gt; dict:<br>    reply = support_chain(&quot;You are a customer support agent. Reply in 1 sentence.&quot;).invoke(<br>        {&quot;question&quot;: question}, config={&quot;callbacks&quot;: [CallbackHandler()]}<br>    )<br>    # Different SDK, same trace — the follow-up classification nests under the same parent.<br>    follow_up = openai.chat.completions.create(<br>        model=&quot;gpt-5-mini&quot;,<br>        messages=[<br>            {&quot;role&quot;: &quot;system&quot;, &quot;content&quot;: &quot;Output one word: ROUTINE, URGENT, or ESCALATE.&quot;},<br>            {&quot;role&quot;: &quot;user&quot;,   &quot;content&quot;: f&quot;Customer asked: {question}\nAgent replied: {reply}&quot;},<br>        ],<br>    )<br>    return {&quot;reply&quot;: reply, &quot;priority&quot;: follow_up.choices[0].message.content.strip()}<br><br>print(handle(&quot;I have not received my order from 3 weeks ago.&quot;))<br>langfuse.flush()</pre><pre>{&#39;reply&#39;: &quot;I&#39;m so sorry your order is three weeks late — I&#39;m escalating this now and you&#39;ll have an update within 24 hours.&quot;,<br> &#39;priority&#39;: &#39;ESCALATE&#39;}</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*sGOaPiK8JBe6kPWzsliqJA.png" /><figcaption><strong>Flow graph of a single trace:</strong> a parent span named handle-customer-question branches to a child LangChain chain that calls an Anthropic generation, and to a second child that is a direct OpenAI generation, with a caption noting one trace, three children</figcaption></figure><p>The thing to notice in the trace is that the two model calls — one through LangChain to Anthropic, one through the wrapped OpenAI SDK — sit side by side as children of the <em>same</em> parent, even though they took completely different code paths to get there. That’s the composition promise made concrete: you instrument each layer with whatever’s most natural for it, and Langfuse stitches them into one coherent story per request. <strong>And — this is the quiet point Priya cares about — none of this code knows or cares whether the Langfuse server is the managed cloud or a container running inside the bank’s own network.</strong> The instrumentation is identical; only the host URL changes.</p><h3>A version gotcha that will bite you</h3><p>One detour, because it’s the single most common way people lose an afternoon with Langfuse. The SDK has moved fast, and the API changed meaningfully between major versions.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*lqZpvdqxC4B4IxXtb7Zwsg.png" /><figcaption>Before-and-after comparison of the legacy v2 API versus the current v3/v4 API: v2 used Langfuse(), langfuse.trace(), langfuse.score() and CallbackHandler(trace_id=…), while v3/v4 use get_client(), start_as_current_observation(), create_score() and the langfuse.langchain CallbackHandler, with a note that v3 went OTEL-based in mid-2025 and v4 removed start_as_current_span</figcaption></figure><p>The original <strong>v2</strong> SDK was built around Langfuse(), langfuse.trace(), and langfuse.score(), with the LangChain handler imported from langfuse.callback. In <strong>v3</strong> (generally available mid-2025), the whole core was rewritten on <strong>OpenTelemetry</strong>: you now get a global client with get_client(), create observations with context managers, and import the handler from langfuse.langchain. Then <strong>v4</strong> tightened the ergonomics further — and quietly <em>removed</em> start_as_current_span(), replacing it with start_as_current_observation(as_type=&quot;span&quot;, ...).</p><p>Why this matters in practice: pip install &quot;langfuse&gt;=3.0&quot; today resolves to <strong>v4</strong>, so older tutorials that call langfuse.start_as_current_span(...) will throw AttributeError on a fresh install. The lesson is boring and load-bearing — <strong>pin your Langfuse version</strong> in requirements.txt and read the migration guide before you upgrade. Every code block in this article is written for the current v3/v4 line. With that out of the way, on to Priya&#39;s second question.</p><h3>4. Tracing Tells You What Happened. Scoring Tells You If It Was Good.</h3><p>This is the half that would have saved Cursor. Tracing captures the reply; it doesn’t judge it. To answer “was this any good?” you attach a <strong>score</strong> — and the pattern that scales is to decouple the judgement from the live request entirely.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*S1vzBxiRtJ-6fSxAS8EoaQ.png" /><figcaption><strong>Scoring loop diagram: </strong>a traced call produces a reply and a trace_id, a separate LLM-as-Judge pass reads the question and reply and emits an empathy score, create_score posts that score back onto the original trace by id, and a dashboard filter surfaces every trace with empathy below 0.6</figcaption></figure><p>The shape, reading the figure left to right: the live call runs and you remember its trace_id; a <em>separate</em> LLM-as-Judge pass scores the output (it can run inline, in a nightly batch, or even days later); then you post that score back onto the original trace by id. Because the judgement is detached, it never adds latency to the customer-facing path, and you can re-score historical traffic when your rubric changes.</p><p>We’ll define the judge once — again, a plain function, since it holds no state — reusing the same support_chain factory with a stricter, more capable model for the grading.</p><pre>def judge_empathy(question: str, answer: str) -&gt; float:<br>    &quot;&quot;&quot;Separate LLM-as-Judge pass. Returns 0.0 (cold/dismissive) .. 1.0 (warm/genuine).&quot;&quot;&quot;<br>    judge = support_chain(<br>        &quot;Rate the customer-support reply on EMPATHY from 0.0 (cold/dismissive) to 1.0 &quot;<br>        &quot;(warm/genuine). Reply with just the number, nothing else.&quot;,<br>        model=&quot;claude-sonnet-4-5-20250929&quot;,   # a stronger model grades the cheaper one&#39;s work<br>    )<br><br>    raw = judge.invoke(<br>        {&quot;question&quot;: f&quot;Customer asked: {question}\n\nAgent replied: {answer}&quot;}<br>    ).strip()<br><br>    try:<br>        return float(raw)<br>    except ValueError:<br>        return 0.5   # judge returned something non-numeric; default to neutral</pre><p>Now the three-step pattern in full. We open a span, capture its trace_id, run the support reply, judge it, and post the score:</p><pre>from langfuse import get_client<br>langfuse = get_client()<br><br># 1) Make a traced call and remember the trace_id.<br>with langfuse.start_as_current_observation(name=&quot;customer-reply&quot;, as_type=&quot;span&quot;) as span:<br>    trace_id = span.trace_id<br>    question = &quot;I was double-charged on my last invoice - can you fix this today?&quot;<br>    reply = support_chain(<br>        &quot;Reply to the customer in &lt;=3 sentences. Be empathetic and clear.&quot;<br>    ).invoke({&quot;question&quot;: question}, config={&quot;callbacks&quot;: [CallbackHandler()]})<br>print(&quot;Reply:&quot;, reply)<br>print(&quot;Trace:&quot;, trace_id)<br><br># 2) Separate judge pass scores the reply.<br>empathy = judge_empathy(question, reply)<br>print(&quot;Empathy:&quot;, empathy)<br><br># 3) Push the score back onto the original trace.<br>langfuse.create_score(<br>    name=&quot;empathy&quot;, value=empathy, trace_id=trace_id, data_type=&quot;NUMERIC&quot;,<br>    comment=&quot;Judged by claude-sonnet-4-5&quot;,<br>)<br>langfuse.flush()</pre><p>Here’s where the numbers tell a story. Run it once against a terse first draft of the system prompt and the judge is unimpressed:</p><pre>Reply: We have received your request. Double charges are reviewed within 5–7 business days.<br>Trace: 7c2f1e9a8b3d4c5e6f7a8b9c0d1e2f3a<br>Empathy: 0.55</pre><p>A 0.55 is exactly the kind of number you never want to discover from an angry tweet. But now it&#39;s <em>measured</em>, which means it&#39;s actionable. In the dashboard you&#39;d filter score.empathy &lt; 0.6, see this trace surface, click into it, and read the cold reply for yourself. The fix is a prompt that leads with acknowledgement before logistics — and you re-run:</p><pre>Reply: I&#39;m really sorry about the double charge — that&#39;s frustrating, and I want to get it<br>sorted for you today. I&#39;ve flagged it to billing for a same-day reversal and will email you<br>the moment it clears.<br>Empathy: 0.88</pre><p>From 0.55 to 0.88, and — crucially — you can <em>prove</em> it moved, because both runs are scored traces sitting next to each other. That loop, run continuously over production traffic instead of one example, is the difference between &quot;our support agent feels a bit off lately&quot; and &quot;empathy dropped 9% after Tuesday&#39;s prompt deploy; here are the twelve worst traces.&quot;</p><h3>Pick your score type deliberately</h3><p>Langfuse supports three score data types, and choosing the right one is what makes the dashboard aggregations meaningful rather than noise.</p><p>Data typeUse it forExampleNUMERICQuality on a continuous scale you average over timeempathy = 0.82CATEGORICALBucketed verdicts you break down in a bar chartrating = GOOD / FAIR / POORBOOLEANPass/fail gates you alert on when they flipsafety_pass = true</p><p>All three attach to the same trace with one call each:</p><pre>with langfuse.start_as_current_observation(name=&quot;multi-score-demo&quot;, as_type=&quot;span&quot;) as span:<br>    demo_trace = span.trace_id<br><br>langfuse.create_score(name=&quot;empathy&quot;,     value=0.82,   trace_id=demo_trace, data_type=&quot;NUMERIC&quot;)<br>langfuse.create_score(name=&quot;rating&quot;,      value=&quot;GOOD&quot;, trace_id=demo_trace, data_type=&quot;CATEGORICAL&quot;)<br>langfuse.create_score(name=&quot;safety_pass&quot;, value=True,   trace_id=demo_trace, data_type=&quot;BOOLEAN&quot;)<br>langfuse.flush()</pre><pre>Three scores attached to trace 9a1b...e2f3 — one numeric, one categorical, one boolean.</pre><p>The instinct to make everything NUMERIC is a trap: a safety_pass is genuinely binary, and modelling it as a boolean lets you build an alert that fires the moment one flips to false, which a 0.0–1.0 score can&#39;t express as cleanly.</p><h3>5. The Dashboard Views Every On-Call Engineer Builds on Day One</h3><p>Once traces and scores are flowing, the Langfuse dashboard becomes the production diagnostic surface. Four saved views do most of the work, and they all fall straight out of the four-noun data model:</p><ol><li><strong>Per-customer / per-tenant cost.</strong> Every trace can carry a user_id and session_id. Filter userId = CUST-00042 for that customer&#39;s traffic in time order; the <strong>Users</strong> view then aggregates total cost, average latency, and session count per user — the answer to &quot;why is this tenant&#39;s bill so high.&quot;</li><li><strong>Latency outlier hunt.</strong> Filter latency &gt; 5000ms to surface slow requests, then pivot by model or by a metadata tag to find the offender. Pair with a P95-latency view to catch a recent regression before customers do.</li><li><strong>Error-only view.</strong> Filter level = ERROR for every trace whose root span errored. The timeline lets you click into the failing span, read the exception, and copy the exact input that triggered it.</li><li><strong>Score regressions over time.</strong> Filter name = empathy, group by week, and you get the line chart from §4 at fleet scale. A drop after a prompt deploy is a regression candidate you can then confirm with a controlled dataset experiment.</li></ol><p>The first view depends on tagging traces with the customer’s identity, and the LangChain handler makes that a one-liner — it special-cases metadata keys prefixed with langfuse_:</p><pre>for cust_id, question in [<br>    (&quot;CUST-00042&quot;, &quot;Refund please.&quot;),<br>    (&quot;CUST-00091&quot;, &quot;Why is my charge pending?&quot;),<br>    (&quot;CUST-00042&quot;, &quot;Status of my refund?&quot;),<br>]:<br>    support_chain(&quot;Reply to the customer in &lt;=3 sentences. Be empathetic and clear.&quot;).invoke(<br>        {&quot;question&quot;: question},<br>        config={&quot;callbacks&quot;: [CallbackHandler()],<br>                &quot;metadata&quot;: {<br>                    &quot;langfuse_user_id&quot;: cust_id,                    # special-cased → first-class trace attribute<br>                    &quot;langfuse_session_id&quot;: f&quot;{cust_id}-2026-05-01&quot;,<br>                    &quot;tenant&quot;: &quot;retail-bank-IN&quot;,                     # ordinary metadata, still filterable<br>                }},<br>    )<br><br>langfuse.flush()</pre><pre>Three traces dispatched. In Langfuse: Users → CUST-00042 → 2 sessions → costs sum up.</pre><p>Two of those three traces belong to CUST-00042, so the <strong>Users</strong> view collapses them into one customer with two sessions and a combined cost — which is precisely the breakdown you reach for when finance asks who&#39;s driving spend, or when a regulator asks you to produce everything a specific customer&#39;s data touched.</p><p>That last clause is the bridge to Priya’s first question, and the part that’s unique to running this in a bank.</p><h3>6. The Part Only a Bank Cares About: Self-Hosting and Data Residency</h3><p>Everything above is identical whether Langfuse runs in the cloud or in your own data center. But for Ravi’s bank, <em>where it runs</em> isn’t a preference — it’s the law.</p><p>Consider what a trace actually contains. To debug a customer-support agent, the trace stores the customer’s question, the model’s reply, and the metadata you tagged — which is to say, <strong>the trace data is itself customer data.</strong> The moment you send it to a third-party SaaS observability tool in another country, you’ve potentially moved regulated PII across a border. For a regulated bank, that’s not a footnote; it’s the kind of thing that ends careers and draws fines.</p><p>This is not abstract. In March 2023, Italy’s data-protection authority <strong>temporarily banned ChatGPT</strong> outright — the first Western country to do so — after a breach exposed users’ chat histories and payment details, citing the lack of a lawful basis for processing personal data. Access was restored weeks later only after OpenAI added transparency and opt-out controls, and in December 2024 the regulator levied a <strong>€15 million fine</strong>. The lesson for an observability stack is direct: regulators treat the <em>handling</em> of personal data as the violation, independent of whether the model itself misbehaved. If your traces hold customer data, the traces fall under the same rules — GDPR in the EU, the DPDP Act in India, RBI’s localization circular, and their equivalents elsewhere.</p><p>Langfuse’s answer is that the entire stack is yours to run. The whole thing comes up with a single command for local evaluation:</p><pre># Clone the OSS repo and bring up Postgres + ClickHouse + the Langfuse server + worker.<br>git clone https://github.com/langfuse/langfuse.git<br>cd langfuse<br>docker compose up -d<br><br># open http://localhost:3000 → create org → create project → grab API keys<br># then point your app at it: LANGFUSE_HOST=http://localhost:3000</pre><p><em>(Skip the </em><em>docker compose up if you&#39;re following along on a shared or bandwidth-limited machine — the images pull several gigabytes. The point is that it&#39;s one command, not the command itself.)</em></p><p>For the bank’s actual production deployment, the architecture is the figure below — and it never leaves the VPC.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*OpyZe5YCKccPZic-rzNCmg.png" /><figcaption><strong>Two-lane self-hosting architecture:</strong> applications send traces to the Langfuse API server and an async worker, which write metadata to Postgres and high-volume events to a ClickHouse cluster, with object storage for blobs; a banner across the bottom states that everything sits inside the bank’s own VPC so customer data never crosses the regulated boundary, with production sizing of three API replicas, two workers, Postgres HA and a three-node ClickHouse cluster handling roughly fifty million events per day</figcaption></figure><p>The shape is two storage engines behind a stateless API and worker tier. The <strong>API server</strong> (Node) takes ingestion and serves the UI; <strong>async workers</strong> handle the heavy write path; <strong>Postgres</strong> holds metadata (projects, users, prompt versions); and a <strong>ClickHouse</strong> cluster holds the firehose of trace and observation events, which is exactly the workload ClickHouse was built for. Object storage (S3 or self-hosted MinIO) takes any large blob attachments. The official langfuse-k8s Helm chart wires all of this together with horizontal scaling: a standard mid-bank sizing of <strong>three API replicas, two workers, Postgres in HA, and a three-node ClickHouse cluster</strong> comfortably handles on the order of 50 million events per day.</p><p>Every box in that diagram runs on infrastructure the bank controls, in a region the bank chooses, behind the bank’s own network policies. The customer’s question, the model’s answer, the audit trail of who-touched-what — all of it stays inside the regulated perimeter. That’s the answer to “can we even <em>run</em> observability under our constraints,” and it’s the single biggest reason a regulated team reaches for Langfuse over a cloud-only competitor.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*gk47VCEZgY1AvYqKA7b85Q.png" /><figcaption>A vault metaphor: a stylized data vault sitting inside a larger walled perimeter labelled as a private network, with trace records safely inside and a closed gate at the boundary</figcaption></figure><h3>Where We Are — and What’s Left</h3><p>Step back and look at what Ravi can now put in front of Priya.</p><p>Her first question — <em>can we run observability when the data can’t leave our VPC?</em> — is answered, and not with a caveat. The instrumentation is three composable patterns (@observe(), the LangChain callback, the OpenAI wrap) that don&#39;t change one character whether the backend is Langfuse Cloud or a self-hosted cluster, and the whole platform self-hosts inside the bank&#39;s own network so customer data never crosses a regulated boundary.</p><p>Her second question — <em>how do we know the answers are good, and get paged when they aren’t?</em> — is answered too. Every reply can carry an LLM-as-Judge score posted back onto its trace, the right score <em>type</em> makes the dashboards meaningful, and four saved views turn raw traces into the per-customer, per-latency, per-error, and per-score-over-time surfaces an on-call engineer actually lives in. The Cursor failure becomes catchable: a quality score on support replies, watched over time, fires a regression alert before the Reddit thread does.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*tGIs1cd0mMbNkMzQOQufBg.png" /></figure><p>But Priya’s third question is still open, and it’s the one that actually decides the architecture: <strong>you already pay for LangSmith in dev — do you rip it out, run both, or self-host Langfuse for the regulated tenants only? And how do you make that call <em>once</em> and stop re-arguing it every project?</strong></p><p><strong>In </strong><a href="https://medium.com/PART_2_URL"><strong>Part 2</strong></a> we close the loop: Langfuse’s prompt management as a self-hostable answer to LangSmith Hub, the same playbook applied across healthcare, e-commerce, legal, and edtech, a head-to-head LangSmith-vs-Langfuse decision matrix, a tour of the wider field (Arize Phoenix, Helicone, Traceloop, Braintrust, Datadog, and more), and a production checklist you can paste straight into a runbook.</p><p><em>If this saved you a future debugging weekend — or a compliance incident — follow me here on Medium so Part 2 lands in your feed.</em></p><p><em>Author: Prashant Sahu · </em><a href="https://www.linkedin.com/in/prashantksahu/"><em>linkedin.com/in/prashantksahu</em></a></p><h3>References &amp; Further Reading</h3><ul><li>ClickHouse — <em>ClickHouse Raises $400M Series D, Acquires Langfuse, Launches Postgres</em> (Jan 16, 2026): <a href="https://clickhouse.com/blog/clickhouse-raises-400-million-series-d-acquires-langfuse-launches-postgres">https://clickhouse.com/blog/clickhouse-raises-400-million-series-d-acquires-langfuse-launches-postgres</a></li><li>Langfuse — Press &amp; company facts (acquisition, funding, license): <a href="https://langfuse.com/press/press">https://langfuse.com/press/press</a></li><li>Langfuse — <em>Python SDK v3 is now Generally Available</em> (OTEL rewrite): <a href="https://langfuse.com/changelog/2025-06-05-python-sdk-v3-generally-available">https://langfuse.com/changelog/2025-06-05-python-sdk-v3-generally-available</a></li><li>Langfuse — Observability overview &amp; data model: <a href="https://langfuse.com/docs/observability/overview">https://langfuse.com/docs/observability/overview</a></li><li>Langfuse — Scores via SDK (create_score, data types): <a href="https://langfuse.com/docs/evaluation/evaluation-methods/scores-via-sdk">https://langfuse.com/docs/evaluation/evaluation-methods/scores-via-sdk</a></li><li>Langfuse — LangChain integration (CallbackHandler): <a href="https://langfuse.com/integrations/frameworks/langchain">https://langfuse.com/integrations/frameworks/langchain</a></li><li>Langfuse — Self-hosting (Docker / Helm, Postgres + ClickHouse): <a href="https://langfuse.com/docs/observability/overview">https://langfuse.com/docs/observability/overview</a></li><li>Langfuse on GitHub (MIT license, OSS core): <a href="https://github.com/langfuse/langfuse">https://github.com/langfuse/langfuse</a></li><li>OECD AI Incidents Monitor — Cursor “Sam” support-bot incident (#1039): <a href="https://incidentdatabase.ai/cite/1039/">https://incidentdatabase.ai/cite/1039/</a></li><li>The Register — <em>Cursor AI’s own support bot hallucinated its usage policy</em> (Apr 18, 2025): <a href="https://www.theregister.com/2025/04/18/cursor_ai_support_bot_lies/">https://www.theregister.com/2025/04/18/cursor_ai_support_bot_lies/</a></li><li>The Hacker News — <em>Italy Fines OpenAI €15 Million for ChatGPT GDPR Violations</em> (Dec 2024): <a href="https://thehackernews.com/2024/12/italy-fines-openai-15-million-for.html">https://thehackernews.com/2024/12/italy-fines-openai-15-million-for.html</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=fb5f446ba208" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Role of Optimization in Machine Learning & Deep Learning]]></title>
            <link>https://medium.com/@prashant9501/role-of-optimization-in-machine-learning-deep-learning-dc919d0330ef?source=rss-a54d16fb2f45------2</link>
            <guid isPermaLink="false">https://medium.com/p/dc919d0330ef</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[optimization-algorithms]]></category>
            <category><![CDATA[convex-optimization]]></category>
            <category><![CDATA[optimization-techniques]]></category>
            <dc:creator><![CDATA[Prashant Sahu]]></dc:creator>
            <pubDate>Thu, 27 Oct 2022 21:37:05 GMT</pubDate>
            <atom:updated>2022-11-03T03:00:04.954Z</atom:updated>
            <content:encoded><![CDATA[<p>the optimization essentials to get started in Data Science…</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*dlOkCyZzZnIxljvz" /><figcaption>Photo by <a href="https://unsplash.com/@sth0315?utm_source=medium&amp;utm_medium=referral">jason song</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><h3>Overview</h3><p>There are 4 mathematical pre-requisite (or let’s call them “essentials”) for Data Science/Machine Learning/Deep Learning, namely:</p><ol><li>Probability &amp; Statistics</li><li>Linear Algebra</li><li>Multivariate Calculus</li><li>Convex OptimizatiIntroduction</li></ol><h3>Introduction</h3><p>In this article, we are going to discuss the following questions:</p><ol><li>Why should I bother about Optimization in ML?</li><li>Where is Optimization used in DS/ML/DL?</li><li>What are Convex Functions?</li><li>What is exactly Optimization, and how is it different from other mathematical problems?</li><li>What are the different types of Optimization Problems?</li><li>How do we “solve” the Optimization problems?</li><li>Which approach should be used to solve what kind of optimization problem?</li><li>Finally, can you show “Optimization in Action” using an ML Algorithm?</li><li>Ok, some more examples where optimization is being used in ML/DL domain?</li><li>Great, where else is Optimization used, apart from ML/DL domain?</li></ol><h3>1. Why should I bother about Optimization in ML?</h3><p>In fact, behind every Machine Learning (and Deep Learning) algorithm, some optimization is involved. Ok, let me take the simplest possible example. Consider the following 3 data points:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/203/1*4s_4HIn1Xk8rvxl3_YIE3w.png" /><figcaption>Toy Dataset discussed in this article [Image by author]</figcaption></figure><p>Everyone familiar with machine learning will immediately recognize that we are referring to X1 as the independent variable (also called “<strong>Features</strong>” or <strong>“Attributes”),</strong> and the Y is the dependent variable (also referred to as the <strong>“Target”</strong> or <strong>“Outcome”).</strong></p><blockquote>Hence, the overall task of any machine is to find the relationship between X1 &amp; Y. This relationship is actually <strong>“learned”</strong> by the machine from the <strong>DATA,</strong> and hence we call the term <strong>Machine Learning.</strong></blockquote><p>We, humans, learn from our experiences. Similarly, the same experience is fed into the machine in the form of data.</p><p>Now, let’s say that we want to find the best fit line through the above 3 data points. The following plot shows these 3 data points in blue circles. Also shown is the red line (with squares), which we are claiming as the “<strong>best-fit line</strong>” through these 3 data points. Also, I have shown a “poor-fitting” line (the yellow line) for comparison.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/940/1*mk86cqEDoZQeYlH1TUU6BQ.png" /><figcaption>Plot of actual and predicted target values by the best-fit line and some poor model [Image by author]</figcaption></figure><p>The net Objective is to find the Equation of the <strong>Best-Fitting Straight Line</strong> (through these 3 data points, mentioned in the above table).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/433/1*Ykb-Lttg4AItVkF-ZeL2Hw.png" /><figcaption>Equation of the Linear Regression [Image by author]</figcaption></figure><p>is the equation of the best-fit line (red-line in the above plot), where <strong>w1</strong> = slope of the line; <strong>w0</strong> = intercept of the line.</p><blockquote>In machine learning, this best-fit is also called the <strong>Linear</strong> <strong>Regression</strong> (LR) model, and w0 and w1 are also called <strong>model weights or model coefficients</strong>.</blockquote><p>The predicted values from the Linear Regression model (Y^) are represented by red-squares in the above plot. Of course, the predicted values are NOT exactly the same as the actual values of Y (blue circles), the vertical difference represents the error in the prediction given by:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/537/1*W8_JpR5YolcaEna8JVu1Xg.png" /><figcaption>Error (Residuals) from the Linear Regression Model [Image by author]</figcaption></figure><p>for any <em>i</em>th data point.</p><p>Now I claim that this best fit-line will have the minimum error for prediction (among all possible infinite random “poor-fit” lines). This total Error across all the data points is expressed as the <strong>Mean Squared Error (MSE) Function</strong>, which will be <strong><em>minimum</em></strong> for the best-fit line.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/940/1*ol4LWDhcdzUaBGhgKcD4sg.png" /><figcaption>The MSE Loss (Cost) Function to be minimized in Linear Regression [Image by author]</figcaption></figure><p>where N = Total no. of data points in the dataset (in the current case, it is 3)</p><blockquote>Minimizing or maximizing any quantity is mathematically referred to as an Optimization Problem, and hence the solution (the point where the minimum/maximum exists) is referred the <strong>optimal values </strong>of the variables.</blockquote><p><strong>You can view a “video version” of this article in the following Youtube video:</strong></p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2F2oix6f873hw%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3D2oix6f873hw&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2F2oix6f873hw%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/d0242bb8862f3c8eb306fcf5ab2b6cea/href">https://medium.com/media/d0242bb8862f3c8eb306fcf5ab2b6cea/href</a></iframe><h3>2. Where is Optimization used in Data Science / Machine Learning / Deep Learning?</h3><p>There are two types of Supervised learning tasks: Regression &amp; Classification.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/940/1*kS6TsblmpyZZo7g160QNSw.png" /><figcaption>Types of Supervised Machine Learning [Image by author]</figcaption></figure><p><strong>In Regression problems,</strong> the objective is to find the “best-fit” line passing through the majority of the data points (basically the best-fit line will capture the pattern/relationship between X &amp; Y), and will have the minimum value of an error function (also called as the <strong>Cost function or Loss function</strong>), commonly which is the Mean Square Error (MSE). Why only MSE, why not a simple sum of errors terms or mean the absolute error will be discussed in another article. Hence, as discussed in the above section, the net objective is to find the line having the minimum value of this error function. The following table lists the Error functions being minimized in the most commonly used regressor algorithms.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/940/1*BcpJo1znBDvHK3llEwhsuA.png" /><figcaption>Summary of Loss functions used in Regression Algorithms [Image by author]</figcaption></figure><p>Apart from the above list, a few ML &amp; DL algorithms also use Huber loss.</p><p><strong>In Classification Problems,</strong> the objective is to find the <strong>“best-separating”</strong> <strong>Hyperplane,</strong> essentially a line (or a curve) passing “between” the data points, such that it is able to separate the majority of the data points into 2 or more classes. In such cases, we want to minimize the Classification error (for wrongly-labeled data points). This classification error is expressed again in many forms depending upon the type of Classifier (ML) algorithm, and the same has been summarized in the table below.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/940/1*R3_2hIMaOWy2fayR0-aG_Q.png" /><figcaption>Summary of Loss Functions used in Classification Algorithms [Image by author]</figcaption></figure><blockquote><strong>In Deep Learning architectures</strong>, in fact, you can create your own customized neural networks and choose your Loss function as per the problem (all available loss functions are listed in this <a href="https://keras.io/api/losses/">link</a>)</blockquote><h3>3. What are Convex Functions?</h3><p>Now that we have a fair idea that optimization deals with maximizing or minimizing a quantity, the basic question can any mathematical function be maximized or minimized? The simple answer is NO. There are mathematical functions that are strictly monotonous and don’t have any maxima/minima. (e.g. the square function y = x² or the exponential function y = e^x ).</p><blockquote><strong>Convex functions</strong> are those functions that have minima (within a given range), whereas <strong>strict convex functions</strong> will have exactly one minimum (in the entire range).</blockquote><p>Well-known examples of convex functions of a single variable include the quadratic function (x²) and the exponential function (e^x). In machine learning, we prefer that the Loss functions are not only convex but also continuous and differentiable. For example, the loss function for LASSO regression (SSE + lambda*L1-Norm) is convex, whereas that of Ridge regression (MSE/SSE + lambda*square of L2-norm) is <a href="https://stats.stackexchange.com/questions/93722/strict-convexity-of-ridge-vs-convexity-of-lasso">strictly convex</a>, similar to the MSE loss function used in Linear Regression.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/940/1*d9FbuBlr89TTnPN95xCruQ.png" /><figcaption>Typical Convex &amp; Concave functions [Image by author]</figcaption></figure><p>Similarly, the concave function will have maxima. You can read more about convex &amp; concave functions <a href="https://medium.com/mathematics-and-ai/maths-and-ai-concave-convex-function-and-minima-and-maxima-of-a-function-83b0f65f5cce"><strong>here</strong></a>.</p><h3>4. So finally what is exactly Optimization, and how is it different from other mathematical problems?</h3><p>Consider the problem of finding the roots of the polynomial 2x² — 3x + 1. The roots of a polynomial represent the values of x, for which the f(x) = 0. You can see from the above plot, that the roots are 0.5 &amp; 1.0. Such kind of “deterministic” problems are not considered optimization problems.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/625/1*k1EsrQ08D23l8qCFAsqNYQ.png" /><figcaption>A typical 2-degree polynomial (convex function) with minima [Image by author]</figcaption></figure><blockquote><strong>In fact, any problem that deals with maximizing or minimizing any physical quantity (which can be mathematically computed/expressed in the form of some variables) are defined as an Optimization Problem.</strong></blockquote><p>Mathematically,</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/595/1*LRhU6CI0nC1gIC2ZI-n9HA.png" /><figcaption>[Image by author]</figcaption></figure><p>is read as <strong>“find the minimum value of f(x)”, </strong>is an optimization problem. Here f(x) is called the <strong>Objective function </strong>(which is referred to as the Loss function or Cost Function in our machine learning), and the variable “x” here is called as the “<strong>Decision Variable”</strong>.</p><p>Also the below:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/654/1*_jAQSE3qwpfle4HCx7jF8A.png" /><figcaption>[Image by author]</figcaption></figure><p>is read as <strong>“find the argument x for which f(x) has minimum value”, </strong>is also an optimization problem. In fact, <strong>all</strong> our machine learning optimizations are of this type!</p><h3>5. What are the different types of Optimization Problems?</h3><p>There are 2 types of Optimization Problems in general: <strong>Constrained &amp; Un-constrained.</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/940/1*SIP8grv36pp686ewwp67Gw.png" /><figcaption>Examples of Unconstrained Optimization problems [Image by author]</figcaption></figure><p>where there are neither any <strong>“bounds” </strong>on the values of x (e.g. x has to be a positive number, or it should be between 0 &amp; 10, etc.), nor any type of constraints that need to be satisfied.</p><p>Linear Regression is an example of unconstrained optimization, given by:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/940/1*cBp5vWmSqrVTkMLEeJfVeA.png" /><figcaption>Linear Regression problem statement [Image by author]</figcaption></figure><p>Says, Find the <strong>optimal weights (w_</strong><em>j</em><strong>) </strong>for which the <strong>MSE</strong> Loss function (given in eq. 3 above) has <strong>minimum value</strong>, for a <strong>GIVEN X, Y data</strong> (refer to very first the table at the start of the article). <strong>L(w_j)</strong> represents the MSE Loss, which is a function of the model weights, not X or Y. Remember X &amp; Y is your DATA and is supposed to be CONSTANT! The subscript “j” represents the jth model coefficient/weight/parameter.</p><p><strong>(B) Constrained Optimization: </strong>for example</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/940/1*hIoFTbmXzENu0wLwFHvuJg.png" /><figcaption>A typical constrained optimization problem with constraints [Image by author]</figcaption></figure><p>says “minimize (x1–2)^2 + (x2–1)^2, <strong>subject to the constraints</strong>…” as given above.</p><p>The range of values of the decision variable which satisfy all the constraints constitutes the <strong>“Feasible Region”</strong>, and the final optimal value will lie in this feasible region. If the constraints are very “hard” and contradicting, there may not be any “feasible solution” to the given optimization problem.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*kbD7iL5H7ONDgEpy.png" /><figcaption>Support Vector Machine Illustration. Image Source: <a href="https://upload.wikimedia.org/wikipedia/commons/7/72/SVM_margin.png">https://upload.wikimedia.org/wikipedia/commons/7/72/SVM_margin.png</a></figcaption></figure><blockquote>An example of constrained optimization is <strong>Support Vector Machines (SVM)</strong>, which is a ML algorithm typically used for classification (although it has a regressor variant too).</blockquote><p>By default, the Support Vector Classifier (SVC) tries to find the separating hyperplane which has the maximum margin, with a “hard constraint” that all the data points must be classified correctly. However, if the data is not linearly separable, then it may not lead to any solution. In such cases, we use the <strong>“Kernel Trick”</strong> and/or use the <strong>soft-margin variant of SVM</strong> (controlled by the hyperparameter C in the algorithm), with the loss function being Hinge Loss. Further details about the SVM problem formulation, loss function, and the Kernel Trick can be <a href="https://www.analyticsvidhya.com/blog/2021/10/support-vector-machinessvm-a-complete-guide-for-beginners/"><strong>found in this article</strong></a><strong>.</strong></p><h3>6. How do we “solve” the Optimization problems?</h3><blockquote>Typically there are 2 approaches to solve most of the complex numerical/scientific problems in general:<br>1. Analytical (Theoretical) Solution<br>2. Numerical Methods Approach</blockquote><ol><li><strong>Analytical Method:</strong> This is the “pen-and-paper” theoretical method normally taught to us in usual school/university syllabi. These methods will give the “exact” solution for any well-posed mathematical problem which is solvable. For example, if we are asked to find the values of x for which the function f(x) = 2x² — 3x + 1 values equal to zero (normally called as the “roots” of the polynomial), we have derived the roots of a generic 2-degree polynomial ax² + bx + c = 0, given by:</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/344/1*Zobh_BmfAm6FSzR0LXar3w.png" /><figcaption>Roots of a 2-degree polynomial [Image by author]</figcaption></figure><p>Upon substituting for a=1, b=-3, and c=1 we get the roots from the above equation as 0.5 &amp; 1.0.</p><p><strong>2. Numerical Methods Approach:</strong> Normally, in this approach we start with a guess value as the solution, then perform some calculations (in this case evaluate the function f(x)), and then “improve” the solution with every iteration until we get the desired result within some tolerance (called as <strong>Convergence</strong>). For example, if the tolerance is set to 0.0001, then the iterations will stop when the value of the f(x) &lt;= 0.0001, and the whole optimization process is said to have <strong>Converged to an optimal solution</strong>. Typical “zero-finding” numerical methods include Newton’s method, Secant method, Brent’s method, and Bisection method. You can also solve differential equations iteratively using Euler’s method and Finite Differences methods. I would suggest an awesome book titled <em>Numerical Method for Engineers by Steven Chapra</em> for studying this topic in detail.</p><p><strong>Similarly, for solving the optimization problems, we again have 2 approaches:</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/750/1*mxG1B1V1Xmu456h5PBmNnw.png" /><figcaption>Approaches to solve Optimization Problems [Image by author]</figcaption></figure><p>1. <strong>Analytical method:</strong> One example of this is the Ordinary Least Squares Method (OLS), used by the <a href="https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LinearRegression.html">LinearRegression</a> class of the scikit-learn package in Python and the <strong>numpy polyfit</strong> function. As previously mentioned, the OLS technique gives the exact coefficients (slopes and intercept terms) for the best-fit line.</p><p><strong>2.</strong> <strong>Iterative Solution: </strong>The most popular iterative method for solving the optimization problems in machine learning is the Gradient Descent Algorithm, and its variants Stochastic Gradient Descent and the MiniBatch Gradient Descent. There are also more advanced variants used in Deep Learning such RMSprop, Adam, Adadelta, Adagrad, Adamax &amp; Nadam (refer to <a href="https://keras.io/api/optimizers/">this webpage from Keras</a>).</p><p>Apart from the GD-based methods, there are other iterative methods to solve constrained and unconstrained optimization problems (refer to this <a href="https://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.minimize.html"><strong>SciPy documentation page</strong></a><strong> </strong>)</p><blockquote>The iterative solvers also include a category of “Heuristic” solvers like Genetic Algorithms, Particle Swarm Optimization (PSO), Ant Colony Optimization (ACO), and other non-greedy algorithms like Simulated Annealing (SA).</blockquote><h3>7. Which approach should be used to solve what kind of optimization problem?</h3><blockquote><strong>For Unconstrained minimization</strong>, you can use the methods like Conjugate Gradient (CG), Newton’s Conjugate Gradient, or the quasi-Newton method of <strong>Broyden Fletcher Goldfarb and Shanno algorithm (BFGS)</strong>, Dog-leg Trust-region algorithm, Newton Conjugate Gradient Trust-region algorithm, or the Newton GLTR trust-region algorithm.</blockquote><p>Do I need to know all of this? Well, some of these techniques are <em>actually </em>used in ML algorithms!. For example, the LogisticRegression class of the scikit-learn library uses the <a href="https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html">BFGS algorithm by default</a>.</p><blockquote>Similarly, for Constrained optimization problems with and without bounds, we have algorithms like Nelder-Mead, Limited-memory BFGS (called as L-BFGS), Powell’s method, Truncated Newton’s Conjugate Gradient algorithms, Sequential Least Squares Programming (SLSQP) etc.</blockquote><h3>8. Finally, let’s see “Optimization in Action”</h3><p><strong>(A) OLS Method:</strong></p><p>Coming back to our sample dataset which I presented at the start of the article, upon substituting for Y^ in eq. 3, the final MSE Loss Function looks as:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/573/1*0yPq8h5p9MHKtBl9rChQFQ.png" /><figcaption>MSE Loss Function after substituting for Y^[Image by author]</figcaption></figure><p>clearly shows that L is a function of model weights (w0 &amp; w1), whose optimal values we have to find upon minimizing L. The optimal values are represented by (*) in the figure below.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/940/1*JNTrGDjm5atkCg825M6EAw.png" /><figcaption>The Loss function has a minimum with respect to all the model coefficients at the optimal values (*) [Image by author]</figcaption></figure><p>Without going into the derivations in this article (planned in a follow-up article), the final optimal values of w0 &amp; w1 (where the MSE Loss function is minimum) are given by:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/942/1*RjVnxOdYR9NSL3DNWeNUtg.png" /><figcaption>Expression for the Optimal values of w0 &amp; w1 [Image by author]</figcaption></figure><p>The calculations are illustrated in the table below:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/940/1*TVaIR-k9VfegwpBVRwU_Lg.png" /><figcaption>Calculations on our toy dataset [Image by author]</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/548/1*4WJIcOimkiIwAHyiJ7SAgA.png" /><figcaption>Calculation of optimal values of w0 &amp; w1 by OLS [Image by author]</figcaption></figure><p>Let us calculate the same using Python code:</p><pre>import numpy as np</pre><pre># define your X and Y as NumPy Arrays<br>X = np.array([1,3,5])<br>Y = np.array([4.8,12.4,15.5])</pre><pre># np.polyfit function does a 1-dimensional Polynomial fitting<br># and the np.poly1d function converts the polynomial coefficents <br># into a polynomial object that can be evaluated</pre><pre>p = np.poly1d(np.polyfit(X,Y,1))<br>print(p)</pre><pre><strong>[OUTPUT]:</strong><br><strong>2.675 x + 2.875</strong></pre><p>You can see our “hand-calculated” values match very closely the values of slope and intercept obtained using NumPy (the small difference is due to round-off errors in our hand-calculations). We can also verify that the same OLS is “running behind the scenes” of the LinearRegression class from the scikit-learn package as demonstrated in the code below.</p><pre># import the LinearRegression class from scikit-learn package<br>from sklearn.linear_model import LinearRegression</pre><pre># create an instance of the LinearRegression class<br>LR = LinearRegression() <br><br># define your X and Y as NumPy Arrays (column vectors)<br>X = np.array([1,3,5]).reshape(-1,1)<br>Y = np.array([4.8,12.4,15.5]).reshape(-1,1)<br><br>LR.fit(X,Y) # calculate the model coefficients</pre><pre>LR.intercept_ # the bias/intercept term (w0*)<br><strong>[Output]: array([2.875])</strong></pre><pre>LR.coef_ # the slope term (w1*)<br><strong>[Output]: array([[2.675]])</strong></pre><p><strong>(B) Iterative Methods for solving optimization problems</strong></p><p>Let me demonstrate this using an example from Section 4 where I defined a function fx=2*x² — 3*x +1 and plotted it. Clearly, if you see the plot there seems to be a minimum between 0.7 &amp; 0.8.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/927/1*Toi6C98Afo_vqGGVkVYi2A.png" /></figure><p><strong><em>You can also see the value of the function at minima is -0.125.</em></strong></p><p>Another example code below using the Nelder-Mead method:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/940/1*HCzoOWIKFIE1fiUqapLX5Q.png" /></figure><h3>9. Where else is optimization used in Machine Learning &amp; Deep Learning?</h3><p>Okay, we have a fair idea about optimization by now. Now let us see where all this is used in ML &amp; DL.</p><blockquote>· Linear Regression uses OLS method for solving the optimal model coefficients</blockquote><p>· In Logistic Regression, the optimal values of the model coefficients are obtained by minimizing the BinaryCrossEntropy Loss function. The <a href="https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html">LogisticRegression class </a>of scikit-learn uses the “<strong>solver”</strong> option (default value of ‘lbfgs’) and <strong>“max_iter”</strong> (default value of 100) as one of the hyperparameters. Clearly, the LogisticRegression class in sklearn uses iterative solvers, and the other supported solvers are ‘liblinear’, ‘sag’, ‘saga’ and ‘newton-cg’. A detailed explanation of these solvers can be read from: <a href="https://stackoverflow.com/questions/38640109/logistic-regression-python-solvers-definitions">https://stackoverflow.com/questions/38640109/logistic-regression-python-solvers-definitions</a></p><blockquote>· In Support Vector Machines, we are minimizing the Hinge loss.</blockquote><p>The Python implementation in scikit-learn API uses the solvers from the “LIBSVM” library (coded in C language). Refer to the <a href="https://scikit-learn.org/0.15/modules/generated/sklearn.svm.SVC.html">SVC documentation</a> page and <a href="https://en.wikipedia.org/wiki/Support_vector_machine">https://en.wikipedia.org/wiki/Support_vector_machine</a> for more details.</p><blockquote>· In Deep Learning, we normally use the advanced variants of the MiniBatch Gradient Descent algorithm such RMSprop, Adam, Adadelta, Adagrad, Adamax &amp; Nadam (refer to this webpage from Keras (<a href="https://keras.io/api/optimizers/">https://keras.io/api/optimizers/</a> ) to train our neural networks.</blockquote><h3>10. Other Applications of Optimization</h3><p>Apart from the Data Science domain, the applications of optimization problems are abundant in the industry. Think of a manufacturing company trying to <em>minimize</em> the cost of manufacturing a product, so that it can <em>maximize</em> its profit. This total cost will be a function of the manpower cost, raw materials cost, costs of inventory, storage, transportation, and other logistics. And further, most of these variables come along with their own set of constraints. So you can find the “optimal number of units” to be manufactured so as to have minimum manufacturing cost.</p><p>Another associated example is finding the <strong>optimal sequence of the operations in a manufacturing process or assembly unit</strong>, such that the machines are optimally used (machine idle time is minimized, and some of them are not over-utilized, while other parts are kept in waiting). This is a scheduling problem (with constraints).</p><p>Another example of a <strong>scheduling problem </strong>is finding the optimal timetable (occupancy and/or availability table) of employees in a company for doing specific tasks in different shifts so that the manpower cost is minimized, and overtime pay may not be needed.</p><p>Another very common problem is <strong>Vehicle Routing</strong> with capacity constraints. This is more of a transportation problem in civil engineering where we have to find the optimal route so that the traffic congestion is minimum. <strong>Also, the amount of time a traffic signal remains open/closed has to be optimal.</strong> Another associated example is Vehicle Routing Problem with pickup and delivery constraints, which may even include time window constraints. Think of a courier delivery boy who has been given a target collecting and dropping 20 parcels every day in a given geographic location. He/she wants to find the optimal route with minimal traffic, the shortest distance, and the minimum time is taken and doesn’t want to visit the same locality twice (another very similar example is the classical <a href="https://en.wikipedia.org/wiki/Travelling_salesman_problem">Travelling Salesman problem </a>).</p><p>Chemical engineers want to have the optimal design (shape, size, diameter, length, etc.) of the pressure vessels for chemical reactions, boilers, and pipes for transporting the fluids, and even storage.</p><p>Some typical applications from different engineering disciplines are mentioned below (reproduced from page 5 of the book <strong>Engineering Optimization: Theory and Practice by Singiresu S. Rao</strong>) :<br> 1. Design of aircraft and aerospace structures for minimum weight<br> 2. Finding the optimal trajectories of space vehicles<br> 3. Design of civil engineering structures such as frames, foundations, bridges, towers, chimneys, and dams for minimum cost<br> 4. Minimum-weight design of structures for earthquake, wind, and other types of random loading<br> 5. Design of water resources systems for maximum benefit<br> 6. Optimal plastic design of structures<br> 7. Optimum design of linkages, cams, gears, machine tools, and other mechanical components<br> 8. Selection of machining conditions in metal-cutting processes for minimum production cost<br> 9. Design of material handling equipment, such as conveyors, trucks, and cranes, for minimum cost<br> 10. Design of pumps, turbines, and heat transfer equipment for maximum efficiency <br>11. Optimum design of electrical machinery such as motors, generators, and transformers<br> 12. Optimum design of electrical networks<br> 13. Shortest route taken by a salesperson visiting various cities during one tour<br> 14. Optimal production planning, controlling, and scheduling<br> 15. Analysis of statistical data and building empirical models from experimental results to obtain the most accurate representation of the physical phenomenon<br> 16. Optimum design of chemical processing equipment and plants<br> 17. Design of optimum pipeline networks for process industries<br> 18. Selection of a site for an industry<br> 19. Planning of maintenance and replacement of equipment to reduce operating costs<br> 20. Inventory control<br> 21. Allocation of resources or services among several activities to maximize the benefit<br> 22. Controlling the waiting and idle times and queueing in production lines to reduce the costs<br> 23. Planning the best strategy to obtain maximum profit in the presence of a competitor<br> 24. Optimum design of control systems</p><h3>Conclusion</h3><p>There are several Python packages for solving optimization problems like CPLEX, DOCPLEX, PULP, PYOMO, ORTOOLS, PICOS, CVXPY, GUROBI, LOCALSOLVER etc. I hope this article has given enough motivation to dive deeper into the concept and applications of Optimization. More to be discussed in further articles. This article was originally published on the <a href="https://www.analyticsvidhya.com/blog/2022/10/optimization-essentials-for-machine-learning/">Analytics Vidhya blog</a>.</p><p>If you liked my article or have any questions, please post a comment below.</p><p><strong>You can also follow/connect with me on Medium &amp; LinkedIn for similar content:</strong> <a href="https://www.linkedin.com/in/prashantksahu/?lipi=urn%3Ali%3Apage%3Aprofile_common_profile_index%3B0c7b3e77-5228-4766-bfad-2adb7bdea908">https://www.linkedin.com/in/prashantksahu/</a></p><p><strong>About Me: </strong>I am a Corporate Trainer &amp; Data Science practitioner with 10+ years of experience in Data Science, ML &amp; DL. I have more than 19 years of overall experience in research, scientific instrumentation, automation, simulation and modeling, image processing, control systems, genetic algorithms etc., and handled many projects in these areas from academia and industry. I am finishing my PhD from IIT Bombay in scientific instrumentation &amp; data analytics.</p><p>Cheers!.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=dc919d0330ef" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>