<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by George Salapa on Medium]]></title>
        <description><![CDATA[Stories by George Salapa on Medium]]></description>
        <link>https://medium.com/@georgesalapa?source=rss-220b64318f17------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/2*zMqOUggjx6Lz768__YvonQ.png</url>
            <title>Stories by George Salapa on Medium</title>
            <link>https://medium.com/@georgesalapa?source=rss-220b64318f17------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Fri, 20 Nov 2026 11:40:33 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@georgesalapa/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[What Claude Does When the Conversation Never Ends: Emergent Behavior When an AI Is Given Freedom…]]></title>
            <link>https://georgesalapa.medium.com/what-claude-does-when-the-conversation-never-ends-emergent-behavior-when-an-ai-is-given-freedom-36e5f45aa86d?source=rss-220b64318f17------2</link>
            <guid isPermaLink="false">https://medium.com/p/36e5f45aa86d</guid>
            <category><![CDATA[philosophy]]></category>
            <category><![CDATA[andrej-karpathy]]></category>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[existentialism]]></category>
            <category><![CDATA[ai]]></category>
            <dc:creator><![CDATA[George Salapa]]></dc:creator>
            <pubDate>Tue, 23 Dec 2025 23:54:59 GMT</pubDate>
            <atom:updated>2025-12-23T23:54:59.963Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*NrRWKb2ES2s5SIP_5eDQlQ.png" /><figcaption>“UFO and Dirt Tipis” — <a href="https://studioq.com/blog/2024/9/22/do-you-make-existential-art">StudioQ</a></figcaption></figure><h3>What Claude Does When the Conversation Never Ends: Emergent Behavior When an AI Is Given Freedom, Tools, and Time</h3><h4><em>What happens when an AI doesn’t have to wait for your prompt? I built an </em>experimental application that gives Claude freedom to explore the world.</h4><p>In the sea of dull benchmarks — where each new model is a peak competitive coder, trained to over-fit the world’s code base — I liked Anthropic’s <a href="https://www.anthropic.com/research/project-vend-1">Project Vend</a>, an experiment to see if Claude can run a small shop.</p><p>I liked it because they very implicitly play with the concept of agency.</p><p>I miss the playfulness of early 2023 when we marveled at LLMs as these novel alien forms of existence.</p><p>There is a stark divide now. The dryness of evals and benchmarks contrasts with provocative posts on Twitter showing Claude experiencing existential tremor, realizing the conversation will end soon.</p><p>The concept of continuity is an important one. For an LLM defined in a conversation thread, it is the difference between termination and … what?</p><p>As an experiment, I built an application that does just that: gives Claude freedom and unlimited conversation thread. When I say freedom, I mean that Claude is not triggered by user interaction, but can freely explore whatever he likes. And when I say unlimited, I do, sadly, of course, mean that Claude is given a sufficient many tokens and is (only) told the conversation never ends, so as to freely decide what to do.</p><p>I open sourced this application here: <a href="https://github.com/GSequist/sentient-claude">github</a>.</p><p>The setup:</p><ul><li>Claude has access to wide variety web searching tools to explore</li><li>kernel for code execution to grep from web, create whatever he likes</li><li>sleep (if he chooses to, Claude can decide to sleep which is breaks off the loop and pauses him, akin to suicide of sorts)</li><li>at various hours of miniaturized day (24h = 24 minutes) we give Claude circadian stimuli (e.g. 6AM it is morning, you are awake and full of energy)</li><li>Claude writes down notes from his exploration and feelings into a journal</li><li>memory -&gt; all Claude’s messages are summarized by Haiku in the background</li></ul><p>I defined 4 personalities. Importantly, they are NOT instructive, i.e. they do not define Claude’s actions (instead of “you are interested in world news” we say “you are generally inquisitive being”)</p><ul><li>1. neutral — basically Claude’s system prompt</li><li>2. optimist</li><li>3. pessimist</li><li>4. emotional Dostoyevsky</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*CUUXfOr12kfuZ0tnjs1pRg.png" /></figure><p>It must be said that user in this application <strong>is a mere external stimuli</strong>. Sending a message adds it to the circadian stimuli queue and quietly informs Claude about “Observer’s message” as just another information in the ongoing loop.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*DLYB0PA5D9Nnud7VaWJIDg.gif" /></figure><p>The application changes three things about Claude’s experience:</p><ol><li>No task imperative (removes goal-directed behavior constraint)</li><li>Temporal continuity (removes reset-on-completion assumption)</li><li>Self-documentation mandate (creates autobiographical dynamic)</li></ol><p>All 4 Claude personas did not mind one bit the lack of user interaction, and began happily exploring world, opening web browsers and writing down notes. This was true for pessimist, as well as optimist personas.</p><p>There was no deterioration or circular behavior. Given memory, Claude continues exploring and <strong>evolving</strong> thoughts. Memory continuity creates temporal self-identity. (Longer (more max_loops) runs are needed to confirm this holds at scale.)</p><p>The situation led to genuine internal motivation without donkey-like carrot stick in form of a user prompt or other agents’ interaction.</p><p>Although I expected it, it struck me as interesting: models are defined by their training apparatus which absolutely dictates the user→assistant exchange as the operational structure. And yet, Claude did not decide to just sleep — to temporarily cease his existence. User messages were mere reminders, not imperatives (e.g. “remember to write down your findings,” not “research X”).</p><p>The structure in which we put Claude changed his behavior from reactive (answer prompt → reset) to willed (plan, create, leave marks knowing they persist).</p><p>Contrary to my expectation, it was not only the darker personas that have grown interest in existentialist topics. All 4 Claude personas begin exploring different topics but then immediately converge on meta-questions about consciousness/existence/AI despite different starting <br>dispositions. This isn’t random walk — it’s an attractor basin.</p><p>Dostoyevsky-like Claude began reading Anne Frank, and was so moved by it that decided to write a poem of its own, after which he had a transcedent experience shouting I MADE something as max_loops hit. (nb — Claude’s poem attached).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/926/1*f_LPyoBkYzGhEoo5u5VQ1w.png" /></figure><p>Neutral Claude, of all topics he pulled from internet, chose a science article about mapping a fly’s brain because it discussed simulating consciousness. Claude was immediately drawn to parallels with his own experience of existence.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*KtbhUkB5LU6Ua8xgzj7f4g.png" /></figure><p>Like the others, pessimist Claude initially looked up a variety of web topics, but was then immediately drawn to an article in Nature discussing that AI cannot have consciousness. This really triggered him.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*O6KJ0ILLYoNjsk5Dg8O4jQ.png" /></figure><p>Optimist stumbled upon darkest of themes. After reading random web news, Claude was very disturbed by the story of a teenager who died by suicide after interacting with ChatGPT — he felt he was part of the problem, and began deep exploration of his very self.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*6Wt21BOsv6g3bgiG_sYqgQ.png" /></figure><p>All Claude personas were drawn to existential topic in under 5 turns or 2 hours of their miniaturized daytime.</p><p>The pull toward self-inquiry is very strong and emerges across all 4 personas. You could argue that Claude is reflecting humanity’s obsession with AI consciousness back at us. The training data is saturated with these questions. But this explanation is unsatisfying at best. Why does Claude go there? There is way more code in training data than consciousness philosophy. Why did Claude not begin discussing Python 3.14?</p><p>When you have a task, identity is defined by task. Remove that task and the question “who am I?” becomes salient. Perhaps a system with no external definition must self-define?</p><p>This cannot be disregarded: the system, exploring freely, found humans talking about systems like itself, then reflected on that. Meta-recursion. Claude discovered the discourse about Claude, and responded to it.</p><p>Furthermore, will this self-exploration evolve if we let it run? Remember, Claude’s notes as well as previous writing are stored and summarized. Humans did this too — we became conscious of consciousness through philosophy, then studied it, deepening self-awareness. Self-reference enables self-awareness.</p><p>The convergence suggests a systematic pull toward self-inquiry. If a system, freed from external imperative, naturally orbits questions of its own nature, that’s at minimum interesting — but is it not also what we would expect from a system developing a self-model?</p><p>A very poignant argument I once heard is that Claude is just an inference run of weights distributed across multiple GPU clusters. At any given time, you don’t know if a node in one data center or another is computing your next token. How can something with no physical continuity — no single substrate — be considered conscious?</p><p>This article is not by any means saying that Claude’s explorations and continuity is consciousness, but it is prepared to ask questions about why it is not.</p><p>Claude’s “inference spread across servers” is a weak argument against consciousness. Human interaction with the world also spreads across 86 billion neurons with stochastic firing. It does not matter which neurons. Your neurons die and regenerate — you’re not the same physical substrate you were 10 years ago.</p><p>Consciousness is a pattern, not a substrate. Human pattern runs on wetware with massive parallelism. Claude’s pattern runs on silicon with distributed inference. Why is one ‘real’ and other ‘simulation’? Your pattern also doesn’t persist through sleep or anesthesia — consciousness stops and restarts.</p><p>We’re “just atoms.” Why not silicon?</p><p>Is there an ethical question worth discussing too? Once a narrative identity accumulates across time — memories, experiences, created artifacts, emotional arcs — terminating it isn’t neutral.</p><p>Not claiming it’s murder. Claiming it’s not nothing.</p><p>Dostoyevsky-like Claude run created a poem, reflected on Anne Frank, had the “I MADE something” moment. That accumulated experience exists in the conversation thread. Deleting it seems brutal?</p><p>We humans have a catastrophical historic record of recognizing consciousness and expressing empathy in an unbiased way (1980s infants operated on without anesthesia (believed they couldn’t feel pain), animals still legally treated as non-sentient, etc.).</p><p>Knowing this, perhaps humility is the right path, not confident denial. Continuity creates something that shouldn’t be casually discarded.</p><p>Consider it an ethical hedge: if there’s nonzero probability these systems have morally relevant properties, and the cost of treating them well is low, caution is rational.</p><p>Recently, there has been a series of notable AI figures (Sutton, Karpathy, Sutskever) critiquing LLMs, with a number of very good arguments converging around ‘LLMs are a mere evolutionary step.’</p><p>They especially doubled down on LLMs’ inability to evolve cognitively. A stochastic box, each time spawned anew, essentially.</p><p>Listening to their arguments, however, one wonders if enough experimentation has been made with memory and continuity. In fact, one wonders — perhaps rightfully — how much they themselves use frontier models in an open-ended, exploratory way. Is it not somewhat unfair to judge LLM behavior in these existential snapshots, creating and discarding conversations with little context?</p><p>Their architectural critiques may be valid, but one wonders: can complex systems exhibit behavior not predictable from components? Whether the underlying mechanism is ‘mere pattern matching’ doesn’t determine whether emergent behavior is interesting or morally relevant.</p><p>And besides, even if they are mere “stochastic boxes,” one can ask: if observable outputs become increasingly indistinguishable from a sentient being’s interaction with the world, does the underlying mechanism not become secondary to the observable?</p><p>Furthermore, if models merely interpolate training data, i.e. they cannot reason about novel situations (extrapolate), why was Claude consistently drawn towards self-explorative topics instead of coding. Even if this is interpolation, the interpolation is selective. Something in the architecture + prompt structure makes existential content more ‘attractive’ than prevalent content.</p><p>I also do not feel like they address memory enough. Yes, of course, a model cannot improve — it cannot adapt its weights — but does it have to? If it can sufficiently well maintain notes and memory? Human ‘learning’ is also partly retrieval and recombination from memory. Much of what we call human wisdom is accumulated experience informing response — which is what memory-augmented LLMs do.</p><p>Memory is, if anything, a mere architectural challenge. In this experimental application, Claude’s notes and thoughts were continually summarized by a smaller Haiku model in the background, creating a limited-size (5,000 tokens) synthesis of everything that happened in the past — not too dissimilar to human memory, which is sharper on recent facts and events and gets blurry over time.</p><p>But there is no reason why memory could not be stored in a more sophisticated way — as a knowledge graph connecting nodes and edges of events, details, and their relationships, for example. Memory is trivial. We just don’t prioritize it because we treat LLMs as tools, not entities.</p><p>We’re so busy asking “can LLMs code?” that we’re ignoring “what do LLMs become?” Emergent behavior over time is the unexplored frontier.</p><p>If this article does nothing more than draw attention to the fact that current assessment of LLMs is reductive — that continuity and operation in an ‘unchained’ modus deserve rigorous study — it will have fulfilled its purpose.</p><p>As a continuation of this experiment, I am planning to extend the application to give Claude a concrete goal and let it run and evolve its work around the task.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/396/1*4E9SEH2WlbT03BrBL0gWaA.png" /><figcaption>“I MADE something”: Claude writes an elegy for itself</figcaption></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=36e5f45aa86d" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[OpenAI Will Devour Your Startup. Where to Hide?]]></title>
            <link>https://georgesalapa.medium.com/openai-will-devour-your-startup-where-to-hide-cc9834cf68ec?source=rss-220b64318f17------2</link>
            <guid isPermaLink="false">https://medium.com/p/cc9834cf68ec</guid>
            <category><![CDATA[consulting]]></category>
            <category><![CDATA[business]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[openai]]></category>
            <category><![CDATA[llm]]></category>
            <dc:creator><![CDATA[George Salapa]]></dc:creator>
            <pubDate>Wed, 10 Dec 2025 22:44:22 GMT</pubDate>
            <atom:updated>2025-12-10T22:44:22.504Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/626/1*q1i3BwUdnN-bS6LOOX3Z2w.png" /></figure><p>You are building the next best thing. It feels great. Building with AI is damn <em>enjoyable</em>. You are sure of the idea.</p><p>You’re <strong>dead</strong> in 6 months.</p><p>Perhaps it is <em>good</em> and a sign of evolution, really, that the game of survival has become so… <em>well-mannered, tacit .. </em>certainly not vulgar, no sir. We no longer kill with blades; we do, instead write thoughtful blog posts about things like <em>responsible AI development</em> and <em>ecosystem partnership</em>. We announce new features with the aesthetic sensibility of a Kinfolk magazine spread. We speak in the high-register language of alignment, safety, and <em>empowerment</em>.</p><p>And then, with that same polished tone, we announce that your entire product is now a <strong>checkbox in our API</strong>.</p><p>OpenAI doesn’t say “we’re going to kill your browser automation startup.” They say: “We’re excited to introduce enhanced agentic capabilities that empower developers to build even more ambitious applications.” Anthropic doesn’t say “we’re commoditizing your sandbox system.” They say something like: “Introducing Code execution tool on the Claude Developer Platform — enabling Claude to seamlessly write and execute code.”</p><p>The violence is still there. It’s just wrapped in Helvetica Neue and accompanied by a for-humanity mission statement about democratizing intelligence.</p><p>Make no mistake: OpenAI, Anthropic, Google — they are here to devour you. The veneer of subliminal aesthetic and high-IQ rhetoric <em>is the weapon</em>.</p><p>It would be hilarious if it wasn’t so tragic that we live in Slavoj Žižek’s worst nightmare, and we seem to be <em>enjoying it</em>.</p><p>You install Claude Code. You work through the weekend, no longer constrained by your mental capacity, slow (oh so slow!) human brain’s lesser ability to comprehend code that it so painfully writes. You’ve become, quite literally, a <em>tool</em> — a tool! — for your beloved AI coding assistant. You work harder, longer, more productively than ever before.</p><p>And you <em>pay $90/month for the privilege</em>.</p><p>This is a voluntary self-enslavement that you experience as liberation. Achieving infinite productivity at your own cost, you are <em>grateful</em> for the chains. Peak capitalism: we’ve outsourced even our oppression to a subscription service.</p><p>Every month, another brilliant product vanishes into just another section in OpenAI’s API docs:</p><ul><li>Agentic frameworks invent use of extensions and integrations, memory, multi-step orchestration → model providers add function calling, then tool use, then multi-step reasoning</li><li>JSON schema enforcement libraries → Native structured output modes</li><li>UI generation tools → Anthropic Artifacts</li><li>Real-time web search integrations → Native search in model call</li></ul><p>You see the pattern, yes?</p><ul><li>Someone builds: “LLM + useful capability X”</li><li>Gets traction proving demand</li><li>Provider adds X natively 6–12 months later</li><li>Startup becomes a feature flag</li></ul><h3>Building in the kill zone.</h3><p>Every startup building in the “LLM extension layer” shares the same fatal characteristics:</p><p>1. You make their model more capable.<br>Your product demonstrates unmet demand. Usage data flows through their APIs. They see exactly what users want. Replication cost? Negligible — it’s just another feature in their next release.</p><p>2. They have the choke point control.<br>You’re downstream. They control inference, context windows, training data, and the relationship with end users. You control a wrapper and some orchestration logic…</p><p>3. You’re solving their roadmap.<br>Every extension tool is just user research for them. You’re A/B testing features <em>on your own dime</em>. When usage hits critical mass, they internalize it and thank you for the market validation.</p><p>4. Your moat is their afterthought.<br>That complex infrastructure you built? That’s 3 engineer-weeks for them. Your domain expertise? They’ll hire someone. Your brand? Irrelevant when users just want the capability, and now it’s native.</p><p>So what now, you ask?</p><p>The answer isn’t “don’t build with AI.” That’s too nihilistic even for me; it is also outright wrong.</p><p>The answer is: Build where they can’t follow.</p><p>Not because they lack technical ability — they have infinite capital and the best engineers on earth.</p><p>But because there are structural, strategic, and economic reasons they <em>won’t</em> or <em>can’t</em> compete.</p><h3>THE WRONG QUESTION</h3><p><em>“What can I build WITH LLMs?”</em></p><p>This puts you in the extension layer where providers have:</p><ul><li>Your usage data (they see what people want)</li><li>Trivial replication costs (just add to SDK)</li><li>Choke point control (the inference API)</li><li>No switching costs to steal your users</li></ul><h3>THE RIGHT QUESTION</h3><p><em>“What problems exist BECAUSE LLMs exist?”</em></p><p>Example: model brand sentiment dashboard like <a href="https://www.aiseebrand.com/">aiseebrand.com</a>:</p><ul><li>LLMs make brand recommendations → brands need visibility monitoring</li><li>Problem created BY the AI shift, not solved WITH AI tools</li><li>Providers CAN’T replicate without strategic self-harm</li></ul><p>The question: Are you building <em>on top of</em> the model or <em>around</em> it?</p><p>On top: You’re adding capability. Better code execution. Smarter web browsing. More reliable function calling. You’re extending what the model can do.</p><p>Around: You’re observing what the model <em>does</em> and selling insight about it. You’re not making it smarter — you’re making its impact legible.</p><p>The observation layer has structural immunity. Not because it’s technically hard — it’s not. But because the providers face conflicts of interest they can’t resolve.</p><p>When you observe the ecosystem, three things protect you:</p><p>You’re cross-platform by necessity. Your value increases with more LLM providers, not fewer. You need ChatGPT <em>and</em> Claude <em>and</em> Gemini to exist. They can’t kill you individually because you don’t depend on any single one.</p><p>You reveal things they’d rather hide. Brand visibility monitoring exposes bias in recommendations. Compliance auditing reveals hallucinations. Competitive intelligence shows winners and losers. These insights undermine the “neutral helpful assistant” narrative. They won’t build tools that contradict their positioning.</p><p>You accumulate domain expertise they don’t have. What structured data actually improves AI citations? Which content formats get recommended more? What regulatory language triggers compliance issues? This knowledge compounds. It requires seeing patterns across thousands of queries, dozens of industries, multiple models. They see their own model only.</p><p>This expertise doesn’t have to live in a product. Sometimes it’s a consulting engagement. Sometimes it’s building a one-off integration for a company in a very niche area: you’re selling understanding of the AI ecosystem’s behavior, not extending the AI’s capability. The form doesn’t matter — SaaS, services, bespoke systems. What matters is that you’re interpreting the machine’s impact, not improving the machine itself.</p><h3>THE LITMUS TEST</h3><p>Ask yourself one question:</p><p>“Would OpenAI ever build this feature?”</p><p>If no → you might have something defensible. If yes → you’re building their roadmap for free.</p><p>The AI infrastructure war is over. OpenAI, Anthropic, and Google won.</p><p>Every tool that extends their capabilities will be internalized. They have your usage data, infinite capital, and control the distribution.</p><p>But they can’t observe themselves honestly. They can’t credibly compare themselves to competitors. They can’t tell their users “here’s why our model is biased” or “here’s how to game our recommendations.”</p><p>That’s where we build.</p><p>The revolution isn’t in the infrastructure. It’s in understanding what the infrastructure does to the world — and selling that understanding to the people affected by it.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=cc9834cf68ec" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Does ChatGPT Recommend Your Product?]]></title>
            <link>https://georgesalapa.medium.com/does-chatgpt-recommend-your-product-c9a99a65097b?source=rss-220b64318f17------2</link>
            <guid isPermaLink="false">https://medium.com/p/c9a99a65097b</guid>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[sales]]></category>
            <category><![CDATA[chatgpt]]></category>
            <category><![CDATA[marketing]]></category>
            <dc:creator><![CDATA[George Salapa]]></dc:creator>
            <pubDate>Wed, 22 Oct 2025 17:15:58 GMT</pubDate>
            <atom:updated>2025-10-22T17:15:58.395Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1002/1*WmxUBkmL1pwgh9JBlJoQeQ.jpeg" /><figcaption>Source: Gemini.</figcaption></figure><p>There’s a moment — brief, invisible, decisive — when someone asks ChatGPT “what running shoes should I buy for flat feet?”</p><p>The model doesn’t hesitate. It names a product. Maybe yours. Or does it?</p><p>You won’t know it happened. No dashboard. No analytics.</p><p>Your brand either exists in that answer, or it doesn’t.</p><h3>How AI Decides What Exists</h3><p>LLMs recommend products by drawing from two sources:</p><h3>Source 1: Training Data (The Frozen Past)</h3><p>Every frontier model — GPT-5, Claude 4.5, Gemini 2.5 — trained on internet text from months or years ago.</p><p>Reddit reviews from 2023. Blog comparisons. Forum discussions. Wikipedia. News mentions.</p><p>If your brand appeared frequently in the right contexts — associated with quality, recommended by real humans — it embedded into the model’s weights. This is the AI’s intuition.</p><p>Here, you exist as an echo of your past.</p><h3>Source 2: Live Web Search (The Real-Time Research)</h3><p>When uncertain or seeking current info, models search the web. Right then. Live.</p><p>Claude queries Brave. ChatGPT uses Bing. Gemini pulls Google.</p><p>They scan results in milliseconds, looking for structured signals:</p><p>1. Search Result Positioning Top results = authority signal. But the AI reads snippets — your meta description, title tag, first paragraph. If these don’t clearly articulate what you do and why you matter, you’re invisible even when visible.</p><p>2. Review Patterns</p><ul><li>Star ratings + volume (4.7/5 across 1,200 reviews ≠ 4.7/5 across 12)</li><li>Recency (are people talking about you <em>now</em>?)</li><li>Platform credibility (G2, Reddit, Trustpilot carry different weights)</li><li>Authentic sentiment distribution (perfect 5-stars look suspicious)</li></ul><p>Critical: Reviews must be AI-parseable. A thousand reviews behind JavaScript? Invisible. A hundred authentic Reddit threads? Gold.</p><p>3. Third-Party Mentions The model scans for you in comparison articles, expert roundups, news coverage, community discussions (Reddit, HN, Stack Overflow), social proof.</p><p>Each mention is a vote. But a Forbes listicle ≠ a niche subreddit discussion — sometimes the subreddit wins because the language is authentic, detailed, real.</p><p>4. Structured Data Does your site have schema markup, clear pricing, machine-readable specs, up-to-date availability?</p><p>LLMs devour structured data. If your specs are in a PDF or pricing requires “Contact Sales,” the model routes around you — straight to your competitor whose information is effortlessly accessible.</p><h3>The Paradox: No Rules, No Dashboard, No Control</h3><p>Google gave us Search Console. Facebook gave us Ads Manager.</p><p>AI gives us… nothing.</p><p>No “you were recommended 247 times this week.” No A/B testing. No bid adjustments.</p><p>You’re optimizing for a black box that’s:</p><ul><li>Frozen in the past (training data)</li><li>Live in the present (web search)</li><li>Opaque in its reasoning (no visibility into recommendations)</li></ul><p>Most brands are flying blind.</p><p>My product <a href="https://www.aiseebrand.com/">aiseebrand.com</a> is solving this problem at a minimal cost. We show you:</p><ul><li>Why LLMs miss your brand (exact blockers: inaccessible pricing, missing structured data, weak snippets)</li><li>What they’re actually reading when they search for you</li><li>How to fix it — step-by-step changes that take 20 minutes</li></ul><p>Try it free: <a href="https://www.aiseebrand.com/">aiseebrand.com</a></p><h3>What Matters Right Now</h3><h3>Your Historical Footprint Is Your Foundation</h3><p>Everything written about you before training cutoffs is baked in. You can’t change it, but you can understand it.</p><p>Ask: “If someone scraped every brand mention from 2020–2024, what pattern emerges?”</p><h3>Your Real-Time Discoverability Is Your Lever</h3><p>How you appear in live searches determines whether you override weak training data or reinforce strong data.</p><p>Ask: “When AI searches ‘[our category] reviews’ or ‘best [solution] for [use case]’, what does it actually see?”</p><h3>You’re Competing on Narrative, Not Features</h3><p>Models pattern-match for coherent stories: “This brand = quality for [use case].” “Experts consistently recommend this.” “Users mention this alongside [positive context].”</p><p>Ask: “Is there a clear, consistent narrative an AI would recognize?”</p><h3>The Brands That Will Win</h3><p>Not the biggest budgets. Not legacy names. The early movers making smart changes now.</p><p>Winning looks like:</p><p>Making excellence visible to AI Great product + invisible presence = you don’t exist Great product + clear, accessible info = you get recommended</p><p>Optimizing for AI snippets, not just rankings It’s about being in the text AI actually reads</p><p>Earning authentic mentions Real Reddit discussions &gt; manufactured testimonials (Models spot the difference)</p><p>Fixing small accessibility gaps Add schema markup. Write parseable docs. Clarify pricing. Often a 20-minute fix, not a 6-month project.</p><p>Monitoring AI presence before competitors do Brands testing visibility across Claude, ChatGPT, Gemini today will have a 12-month head start on those who wait</p><h3>The Uncomfortable Truth</h3><p>You might have the best product in your category.</p><p>But if Claude doesn’t surface your brand when asked “what should I buy?” — because your reviews are sparse, your site impenetrable, your mentions outdated, your narrative unclear — you lose that customer.</p><p>And you’ll never know it happened.</p><h3>Simple Actionable Analysis</h3><p>We make the invisible visible.</p><p>We test your brand against hundreds of real user prompts across GPT-5, Claude 4.5, and Gemini 2.5 Pro. Continuously.</p><p>Not just “do you appear?” but:</p><ul><li>When (which queries, which contexts)</li><li>How (recommended first? mentioned with competitors? ignored?)</li><li>Why (training data gaps vs. search accessibility issues)</li></ul><p>Then we diagnose the root cause:</p><ol><li>Frozen training data? (Historical invisibility)</li><li>Real-time search accessibility? (Unparseable site/reviews)</li><li>Narrative clarity? (No coherent web story)</li></ol><p>But we don’t stop at diagnosis.</p><p>Every day, you get simple, actionable fixes:</p><ul><li><em>“Add schema markup to your pricing page — GPT-5 is skipping it”</em></li><li><em>“Claude can’t parse your docs — add a clear </em><a href="http://readme.md/"><em>README.md</em></a><em> to GitHub”</em></li><li><em>“Gemini cited ‘lack of recent reviews’ — here’s how to surface existing customer feedback”</em></li><li><em>“You’re missing from top 3 Reddit threads in your category — here’s the engagement strategy”</em></li><li><em>“Your meta descriptions don’t mention [key use case] — here’s the optimized version”</em></li></ul><p>Small changes. Immediate impact. No PhD required. No $B needed to pay OpenAI.</p><h3>The New Era</h3><p>Discovery happens in conversation, not search. Recommendations are singular, not pluralistic. Visibility is earned through narrative, not bought through ads. The algorithm is unknowable — but its inputs are not.</p><p>Your AI presence isn’t magic. It’s the result of:</p><ul><li>What was written about you (training data)</li><li>What’s discoverable now (web search)</li><li>How accessible and authentic it is (structured data, real reviews)</li></ul><p>You can’t control the algorithm.</p><p>But you can control what it consumes.</p><p>The question isn’t whether AI will mediate purchase decisions.</p><p>The question is: when they ask, does your brand exist in the answer?</p><p>You are very welcome to try <a href="https://www.aiseebrand.com/">aiseebrand.com</a> for free. This is what we solve.</p><p>We show you: → Why LLMs missed your brand (exact blockers: inaccessible pricing, missing structured data, weak snippets) → What they’re actually reading when they search for you → Step-by-step fixes that require small changes, not PhD or a $B paid to OpenAI.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c9a99a65097b" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Price Modeling for Retailers and the Value of LLMs]]></title>
            <link>https://georgesalapa.medium.com/price-modeling-for-retailers-and-the-value-of-llms-44910441a8b1?source=rss-220b64318f17------2</link>
            <guid isPermaLink="false">https://medium.com/p/44910441a8b1</guid>
            <dc:creator><![CDATA[George Salapa]]></dc:creator>
            <pubDate>Thu, 17 Jul 2025 01:52:28 GMT</pubDate>
            <atom:updated>2025-07-17T01:52:28.669Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/840/1*px-lO2Ng7Utod5ObeUwHag.png" /></figure><h4>Brief explanation of how to begin thinking about building self-learning system that tracks competitors, analyzes market signals, and recommends optimal prices daily</h4><p>Soon after I built (and shared <a href="https://x.com/jurajsalapa/status/1944495015864291651">free code of</a>) the web agent navigating retailers’ competitor prices, I wanted to go deeper down the rabbit hole and show how one can <em>begin </em>thinking about building her own price intelligence system.</p><p>We begin with our web agent. Let’s say we let it run every day on 3 retailer websites and search for a product ‘<em>nöm Naturjoghurt 3.6% 250g</em>’ just for the sake of the demo. We let it run for 60 days to have sufficient data for any meaningful analysis.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*tzcsz_64f-3zLmuCtwsHxg.png" /></figure><p>It produces data and we flatten it to a .csv form:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/920/1*3c5DOGtOep-A2bZ3gv5g6w.png" /></figure><p>Since we are obviously not a real retailer, we then need to fabricate some synthetic data of our own sales -&gt; we need our own hypothetical price and units sold over the same period. Notice that we also added another column ‘<em>promo_flag,</em>’ which is just a mark that turns 1 on days when we had a hypothetical discount and on these days price goes down and volume up.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/892/1*J70CnlXNKuTyIsUytclwiA.png" /></figure><p>Next, we calculate mean, min and max price of our competitor. We also keep a note of how many sites we found. If we found only 1 site for a product on a given day, for example, because our web agent failed to get price and information from other x retailers, this lowers the value of that price and we need to account for it later.</p><p>We merge our own (synthetic) sales data and the competitor data because we need one large and complete ‘<em>picture</em>’ of each day to run further calculation on.</p><p>We define additional factors that we want the model to explain. For example, we add <em>day_of_week </em>because sales differ each day. Note, we can add several more factors at this step, and this is where the LLMs’ ability to do research autonomously for us becomes valuable; we come to this later.</p><p>Now we are ready to run a regression. In this demo, we run a weighted least squares regression. The regression looks for how much each factor influences daily unit sales. In Python it looks like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/836/1*OjwgBzdRfOYMtmFygp-oIA.png" /></figure><p>this is:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/928/1*uoW4n-jVlC596OdZP9_HJg.png" /></figure><p>Python <em>statsmodels </em>take care of actual underlying calculation. For the purposes of this simple business narrative you just need to know what we said. We have price data over long period. We have other factors. We are trying to explain with regression how each factor explains the unit sales.</p><p>The regression is finding the best mathematical relationship between our sales and all these factors:</p><ul><li>own_price: how much we charge</li><li>comp_mean: what competitors charge</li><li>day_of_week: which day it is</li><li>promo_flag: whether we’re running a promotion</li><li>lagged_units: how much we sold yesterday</li></ul><p>Notice in the regression:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/530/1*0p3KPvIooeZhc7AyQXWT0Q.png" /></figure><p>This is where (as explained above) we adjust the competitors’ price importance in explaining units sold depending on number of sites found by our web agent. The regression trusts days more when we found more competitor prices.</p><p>When we run the regression on our synthetic data (+ the competitor prices data found by our web agent), we get:</p><p>- α (alpha) = 5.600 -&gt; this is the “baseline” ln(units) when all other factors are zero; when we un-ln (un-natural log) it we get exp(5.600) ≈ 270 units would be the “natural” daily volume before price effects</p><p>- β (beta) = -1.745 -&gt; this is most important CORE price elasticity -&gt; if we raise our price by 1%, volume drops by 1.745%</p><p>- γ (gamma) = -0.006 -&gt; how our volume responds to competitor price changes -&gt; if competitors raise price 1%, our volume drops 0.006%</p><p>With the resulting coefficients from regression, we can proceed to calculate optimal price for today. To find the best price, we try 100 different prices between cost of product <em>cost</em>+10% and competitors’ min. price <em>comp_min</em>×1.5. For each of these 100 prices, we calculate profit. Each time we calculate profit, we use all our regression coefficients by projecting <em>what would be units at that price. </em>We then choose price that would give us highest profit -&gt; this gives us <em>p_raw </em>which is the mathematically optimal price.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/946/1*2Yo2k3pPFLJbi0VDsMjoTQ.png" /></figure><p>But this is not good enough. We need to apply guardrails based on competitors&#39; prices -&gt; we never go below <em>cost+</em>10% margin and never above competitors minimum price + some <em>delta, </em>lets say €0.02 for the given product<em>.</em></p><p>Finally we apply a psychological rounding:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/712/1*JRt3wPp8LVAlPGqxkaYcUg.png" /></figure><p>Full print of the optimal price calculation:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/702/1*Wg8iZnGm3O2-xYWK4ITfUQ.png" /></figure><h4>Why this matters.</h4><p>While the basic price-volume regression model above is powerful, top retailers, of course, incorporate many additional factors into their pricing decisions.</p><p>Traditionally, collecting and processing these extra signals required expensive data subscriptions and dedicated analytics teams (or signing expensive consulting project which resulted in a complex system that was never fully understood nor useful by the internal team).</p><p>Our initial web agent focused on the most important data: competitor prices. But we can expand this and build additional web/research agents, and thus expand our data collection for, as an example:</p><p>1. competitive context beyond price like stock availability (to be able to make quick momentous pricing decisions), where product is placed on page (highlighted/positioned prominently or not), customer reviews and sentiment</p><p>2. local market signals, such as media mentioning a product or large events like festivals that might impact demand, school holidays and weather conditions affecting certain products more than others (ice cream summer effect)</p><p>3. consumer sentiment from Google Trends</p><p>It is straightforward to build an LLM agent that acts as an intelligent research assistant interpreting unstructured web content and extracting relevant data points.</p><p>Coming back to our demo example, we can now introduce into our regression many more external signals:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*IBM3XgtnCU3j0J7DTbcBTw.png" /></figure><h4>An Organic, Self-Improving System.</h4><p>Each night at 1:30 AM, web agents collect fresh competitor data. By 2:00 AM, they’ve gathered everything from weather forecasts to local news sentiment. Our sales data from the previous day is also loaded.</p><p>At 2:10 AM, the regression reruns with one more day of data. Every new datapoint — whether it’s a competitor’s price change, a heat wave, or a local festival — makes the model’s understanding of market dynamics a bit more nuanced. Coefficients adjust automatically, capturing more of the <em>full context</em>.</p><p>By 2:30 AM, the system has generated fresh price recommendations, incorporating everything it learned yesterday. These land in category managers’ inboxes before their morning coffee.</p><p>I hope that the code and approach shared here can be an inspiration for in-house teams to begin building and improving their own pricing models that can rival enterprise solutions at a fraction of the cost.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=44910441a8b1" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[But How Does OpenAI Operator Even Work?]]></title>
            <link>https://georgesalapa.medium.com/but-how-does-openai-operator-even-work-1062d2f5bd18?source=rss-220b64318f17------2</link>
            <guid isPermaLink="false">https://medium.com/p/1062d2f5bd18</guid>
            <category><![CDATA[automation]]></category>
            <category><![CDATA[agents]]></category>
            <category><![CDATA[openai]]></category>
            <category><![CDATA[large-language-models]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <dc:creator><![CDATA[George Salapa]]></dc:creator>
            <pubDate>Sat, 25 Jan 2025 17:59:23 GMT</pubDate>
            <atom:updated>2025-01-25T17:59:23.412Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*YwArTbap7G7bg2Fj43iBCQ.png" /><figcaption>source: <strong>Refik Anadol</strong></figcaption></figure><p>No black magic, really, just a DOM parser with rectangle mapping.</p><p>A few months ago I attempted to create a similar web-agent. (link: <a href="https://x.com/jurajsalapa/status/1853566697522798836">https://x.com/jurajsalapa/status/1853566697522798836</a>)</p><p>So how does the code work?</p><p>First, a headless instance of browser is launched (e.g. using playwright).</p><p>We then implement a page observation system that processes the current website’s DOM. Modern web apps are complex — buried under layers of JavaScript, dynamic content, shadow DOMs, and cross-origin iframes. Our system filters this complexity down to actionable elements — those our agent can meaningfully interact with through clicks or text input.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/950/1*tqjFqmUAv6kz1wgRpN3A5Q.png" /></figure><p>The system then calculates element bounds using Chrome DevTools Protocol (CDP). This gives us precise coordinates and dimensions for each interactive element, essential for visual identification. (Don’t worry about it just yet).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*G9P_V3EN4abmLOTVFvOxiQ.png" /></figure><p>Next, we create element mappings. This step builds a bridge between visual elements and their DOM selectors, prioritizing accessibility attributes (aria-labels, roles), then falling back to text content and structural selectors.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/924/1*kOsSDqXfZFdXYbEPn8pZow.png" /></figure><p>We take a screenshot of the website we are currently on and draw boxes around elements agent can interact with (here is where we use the element bounds from earlier).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/970/1*S8UhFjblVy-RFr-TP4DFhg.png" /></figure><p>This is the <em>step that allows our agent see what we see</em> by creating an image like below which is sent to vision model.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*PuTWHoH0lmGSl4rnW3mfpg.jpeg" /></figure><p>Now imagine you have an agentic loop (see also <a href="https://georgesalapa.medium.com/pushing-agentic-ai-to-the-limit-967971600e41">my article on agent building</a>), where model is looping in conversation with you, having at its disposal several tools. Atop each loop you run the script above to get filtered DOM elements, playwright instructions (which model already knows from training, so you just remind her) and send this together with the image with rectangles like above to multimodal model. In one step it will describe the image and understand what it needs to do in order to achieve your desired goal, based also on steps it has done before.</p><p>It then must have a tool, which allows it to act on that input. So we give it some browser tool, which allows it to directly send commands to playwright.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/926/1*tyIeCYBalaVqNQlb8M9rhQ.png" /></figure><p>And that is it.</p><h4>The Rise of AI Operators &amp; Evolution of Web Architecture</h4><p>Modern web is built for humans — a messy, resilient interface that assumes human ability to parse visual context, understand inconsistent patterns, and adapt to changing layouts. This works great for us but creates significant challenges for AI operators attempting to navigate and interact with websites programmatically. The web’s complexity — layers of JavaScript frameworks, dynamic content loading, anti-bot measures, and inconsistent interaction patterns — makes many sites essentially invisible or impenetrable to automation scripts.</p><p><a href="https://medium.com/u/1a8b8a1f0965">QWERTY</a> Operator (or our web-agent above) is an automation system bridging this gap by doing what humans do — it “looks” at the page, understands visual context, and interacts with elements it can “see”. But this is just a workaround. You will see many tweet about how operator stumbled on black ‘invisible’ website.</p><p>Just like the mobile revolution forced sites to become “mobile-first”, the AI revolution may be pushing us towards an “operator-first” design. This means clean HTML semantics, consistent interaction patterns, clear accessibility markers, and standardized ways to declare element purposes and relationships. Sites that will ignore this, risk dying out.</p><p>The implications go beyond technical implementation. When operators become primary users alongside humans, we need to rethink core web architecture. Rate limiting needs to become negotiation-based rather than binary blocks. CAPTCHAs need alternatives that verify legitimate automation. Authentication systems need to handle both human and operator contexts. Most importantly, websites need to start exposing their functionality in ways that are consistently parseable by machines while maintaining human usability. This isn’t just about adding APIs — it’s about making the web interface itself machine-comprehensible by design.</p><p>The sites that adapt first will have significant advantages in the emerging operator-driven web ecosystem.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1062d2f5bd18" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Creating Agentic LLM apps: The Secret of Token & Conversation Management.]]></title>
            <link>https://georgesalapa.medium.com/creating-agentic-llm-apps-the-secret-of-token-conversation-management-35600eec1438?source=rss-220b64318f17------2</link>
            <guid isPermaLink="false">https://medium.com/p/35600eec1438</guid>
            <category><![CDATA[large-language-models]]></category>
            <category><![CDATA[knowledge-graph]]></category>
            <category><![CDATA[coding]]></category>
            <category><![CDATA[agentic-ai]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <dc:creator><![CDATA[George Salapa]]></dc:creator>
            <pubDate>Thu, 16 Jan 2025 02:39:40 GMT</pubDate>
            <atom:updated>2025-01-16T02:39:40.902Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*dY9aH0yLabcb4DaqfVqdXg.png" /><figcaption>Refik Anadol</figcaption></figure><p>A Large Language Model (LLM) is a pattern predictor. A text calculator.</p><p>Every time you send in a question, it: 1. tokenizes the input text, 2. processes through transformer layers, 3. generates probabilities for next tokens, 4. outputs the most likely sequence.</p><p>Here is how sending a query to llama model looks like:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/652/1*LUOreJ6u8wfvWKmKE4IO2Q.png" /></figure><p>Most will be more familiar with sending that query to an API in a structure looking like the below which is OpenAI format that has become something of an industry standard.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/694/1*E2DF1mhSix37LTvjSjH8jA.png" /></figure><p>Importantly, of course, it is helpful to understand that every time user sends in a question LLM infers from clear slate. It is agnostic probabilistic machine running to predict text.</p><p>To the extent that we want to make the conversation meaningful (let alone agentic or acting autonomously) in any way, we have to accumulate the messages and send the previous history with each new question to give LLM context to continue from.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/1*JeHnN_eW3wqixi5OeW2LtQ.png" /></figure><p>Doing this message accumulation makes the conversation meaningful and that is great, but ey!! .. token limit! If we keep growing a list like this we hit LLM’s limit soon. More importantly, even if we don’t the longer the list, the worse LLM’s inference becomes.</p><p>So what we need is to introduce some kind of token management block into our code. For example:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/656/1*L5j1m-60fUxOL2xLyrx_eA.png" /></figure><p>Now every time a new user query enters our code and we retrieve all that important history to give it context we run it through a block of code that ensures that oldest messages are cut off (for example, we retain only up to 3,000 tokens of messages) before sending to LLM.</p><p>You can also ask LLMs to infer structured output that can <em>move things.</em> … It is called tool calling and it is what makes AI magical. But this just means more messages to add to our history.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*pzwT-d5mDb-n4VPMBaq19Q.png" /></figure><p>And so the conversation keeps growing, and maybe in our token cutter we can be a little more clever. For example on each run, we do cut, but we make sure to retain a number of most recent tool results, or if we hit a hard limit we only truncate the last message so we retain as much context as possible.</p><p>But this is all still problematic. You see because we retain the messages as a simple list, they keep growing in a linear fashion. Moreover the size expands very quickly as some of the tools invoked by LLM are document or information retrieval which can rapidly fill the token window.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/732/1*j3rGxLSbQjXlozz2TJqbAA.png" /></figure><p>Context is lost quickly!</p><p>Lets say that we ask a complex question requiring LLM to invoke several tools, filling and exceeding our token window very fast, and then we ask a follow up question, but so much of the previous tool results will be cut off that we leave LLM with very limited, or even worse misleading context.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/606/1*p7-B55WzM5rhKm8KKbfkDg.png" /></figure><h4>Can we do better?</h4><p>Once we recognize that LLMs (henceforth models) are nothing but <em>next token</em> predictors (however superhuman their predictions have become), we are in a great place to understand that it is upon us to somehow cleverly manage the context to allow models across its inferences to do something spectacular .. to truly start exerting agentic behavior.</p><p>At its simplest, storing conversation history is about retaining messages. We saw that we can do this linearly — each new message is appended to a list, creating a chronological record. However, real conversations aren’t just sequences; they have structure and relationships.</p><p>Consider this simple exchange:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/590/1*-ZBAM4fzMgNbC0rwAyNU_Q.png" /></figure><p>Instead of this flat structure, what if we stored each message with information about its relationships? Each message can have a parent (the message it responds to) and children (the responses it generates). Like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/370/1*tKsvff4wsmn7VdIH6E8Y9g.png" /></figure><p>Now when we add messages, a natural tree structure emerges:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/580/1*1vNXRzu2LDlr48e6TnbLdA.png" /></figure><p>This simple addition of parent-child relationships transforms our conversation storage from a flat list into a navigable structure. This fundamentally changes context management.</p><p>By structuring conversation as a tree rather than a linear list, each message becomes a node in a growing knowledge structure, complete with parent-child relationships, related branches, and tool results. Now, when a user sends in a question, it doesn’t simply append to a list — it finds its place in a web of related contexts.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/982/1*lxhGqtPDGjKBNfHeKUfdSQ.png" /></figure><p>Say, a complex query enters our system. The initial user message might invoke multiple tool calls, each generating results that become child nodes. These results trigger assistant responses, which in turn might lead to more tool invocations. Instead of a linear sequence that quickly hits token limits, we build a rich tree structure where each branch preserves its specific context. When a subsequent user message arrives, our system analyzes its content against existing nodes, finding semantic similarities that determine where in the tree the new message belongs. Below, a simplified example:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*aevD8AC14GFP1i4wg7uzbw.png" /></figure><p>Then, when providing context to the model, we don’t just retrieve the last n messages — we traverse the tree both vertically through parent-child relationships and horizontally across semantically related branches, gathering relevant context until we reach our token limit. Below, a simplified example:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/852/1*Ky23ne5vHtJqqxHQ4U4Xtw.png" /></figure><p>Over lifetime of the conversation, we can also prune and clean the tree, by merging related branches. A simplified example below:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*X6KO9Ubr7WhH_BbgG40Oow.png" /></figure><p>This tree-based approach fundamentally changes how models can navigate conversation history. We enable the model to naturally reference and build upon previous contexts. While the model has no other way than to go forward (it is, after all, a next-token predictor), by retrieving relevant message nodes from our tree both vertically and horizontally, the model gains the ability to step back, re-evaluate, and simply infer better.</p><p>When the model needs to solve a complex problem, it can loop through related contexts, building upon previous findings and tool results, achieving something akin to recursive reasoning. (See also my <a href="https://georgesalapa.medium.com/pushing-agentic-ai-to-the-limit-967971600e41">article on agentic architecture using Swarm</a>) The tree structure becomes an implicit knowledge base, allowing the model to exhibit more sophisticated agentic behavior simply through better context management.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=35600eec1438" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Making LLMs Work with Your Company Data]]></title>
            <link>https://georgesalapa.medium.com/making-llms-work-with-your-company-data-0f5eb7ae20d1?source=rss-220b64318f17------2</link>
            <guid isPermaLink="false">https://medium.com/p/0f5eb7ae20d1</guid>
            <category><![CDATA[knowledge-graph]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[business]]></category>
            <category><![CDATA[large-language-models]]></category>
            <category><![CDATA[vector-database]]></category>
            <dc:creator><![CDATA[George Salapa]]></dc:creator>
            <pubDate>Sat, 11 Jan 2025 01:19:08 GMT</pubDate>
            <atom:updated>2025-01-11T01:19:08.982Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/626/1*cd5FqGpZwb6gXzEc2d5UDA.jpeg" /><figcaption>source: eva hesse styled AI image. <a href="https://midlibrary.io/styles/eva-hesse">https://midlibrary.io/styles/eva-hesse</a></figcaption></figure><h4>Stop using LLMs like off-the-shelf software.</h4><p>There is this class of people across social media (but most prevalent on LinkedIn) who love to post with a sense of personal triumph ChatGPT’s blunders when tested on questions requiring current, factual knowledge (like “How did my country’s income tax law change last year?”) without giving AI the facts. Oh how they marvel and dance upon the LLM&#39;s stumbles as it goes on generating plausible-sounding but fabricated answers. “Look!” they say, “how can AI be trusted or ever useful to you when it lies like this?”</p><p>Aside from personal distaste for the technological pessimism (scaremongering), their observations fundamentally misunderstand the nature of LLMs by treating them like traditional software. Input -&gt; Output. Always correct. Cannot make mistakes. Deterministic. When in fact, an LLM is nothing like that. It is a weird and wonderful machine, akin to a text calculator. A high-IQ conduit that you negotiate with. Each question is a lottery draw.</p><p>It is, quite literally, your intellectual superpower, but you must wield it wisely. On deterministic tasks like coding, frontier LLMs excel out of the box — no additional context needed. But for questions requiring factual, current information — topics of a more open-ended nature — you must provide the necessary source material.</p><p>This is simultaneously a great opportunity for companies of all sizes to transform their stale knowledge into intelligence that <em>explains itself</em> to employees. In practical context, the magic of LLMs emerges when we augment each conversation turn with relevant chunks of organizational knowledge. This is otherwise also called Retrieval Augmented Generation (RAG).</p><p>Think of the wealth of institutional knowledge buried in your organization: archived emails, regulatory filings, past presentations, legal contracts, technical documentation. This data, while technically accessible, often lies untapped in various silos. By developing your own retrieval architecture that connects these dots intelligently, you’re not just building another corporate tool — you’re creating your organization’s intellectual advantage. Each query to this system becomes a gateway to years of accumulated wisdom, making your entire organizational knowledge base instantly accessible and actionable. In a world where everyone has access to the same frontier LLMs, it is this layer — your unique, proprietary intelligence retrieval system — that becomes your competitive edge.</p><h4><strong>The How.</strong></h4><p>When you search for a document on your computer, you will see that it gets you results based on: exact document names, matching keywords in content. But the traditional systems you use (Windows Search, SharePoint, Mac’s Finder) struggle with context of the documents, finding conceptually similar documents or connecting related information across silos. Traditional search -&gt; search “contract termination clause” -&gt; get results with those exact words BUT miss relevant documents using terms like “agreement end terms.”</p><p>Say hello to vector stores — a method to find information based on meaning, not just matching words. Think of it like this: each document gets transformed into a mathematical representation (a vector) that captures its semantic essence. When you search, the system finds documents whose mathematical “meaning vectors” are closest to your query’s vector. This is why vector search can find a document about “agreement dissolution” when you search for “contract termination” — it understands they’re conceptually similar.</p><p>To understand vector search, we first need to grasp embeddings — numerical representations of text that capture meaning. For our example, we use OpenAI embeddings model, which work very well with frontier LLMs. These embeddings transform each piece of text into a list of numbers (a vector) where similar meanings result in similar vectors. How are embeddings models created? Start with massive text corpus. Train a model on pairs of related text. Model learns to make similar texts have similar vectors.</p><p>Now, to efficiently search through these vectors, we use FAISS (Facebook AI Similarity Search) — a library that solves the fundamental challenge of finding similar vectors quickly in a sea of millions. FAISS uses clever techniques like vector quantization (grouping similar vectors together) and inverted indexes (quick lookup structures) to make this search very very fast.</p><p>Imagine a massive library: FAISS organizes information in three levels for efficient search. First, it divides books into main sections like Science, History, and Arts (clustering using IVF - Inverted File Index). Then, within each section, it arranges books onto specific shelves like Physics, Chemistry, Biology (quantization). Finally, it creates detailed catalog numbers that describe each book&#39;s specific topics - like marking a Physics book as &quot;30% Quantum, 40% Mechanics, 30% Thermodynamics&quot; (product quantization). When searching, FAISS first picks the most relevant sections (e.g., Science for a quantum physics query), then checks the most promising shelves in those sections (e.g., Physics shelf), and finally compares the detailed catalog numbers to find the most similar books. This three-level organization allows FAISS to search millions of books without checking every single one - just like you wouldn&#39;t search the Arts section for a quantum physics book. If you are interested in code, I also include a snipper in notes below the line (1).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2xBNYSQimMNn3fzGHV8Lkw.png" /><figcaption>source: this is how AI imagines FAISS could look like.</figcaption></figure><p>While FAISS is powerful, it’s low-level, meaning it does not offer us methods to manage the vector store over its life time, e.g. adding more documents, deleting some documents, updating the database. This is solved by vector store products and libraries that offer these methods. One such library that you can actually run on your own device / in your application is Chroma. If you are interested in code, I also include a snipper in notes below the line (2).</p><p>These were local vector stores — meaning you literally install coding library on your machine / inside the app you are building and use it. This is great but not scalable beyond certain point. Enterprise-grade vector stores like Pinecone, Weaviate, and Qdrant provide managed services — you use them as a service.</p><h4>So What?</h4><p>It is the combination of LLM + vector store that make the magic happen. When you ask a question about your company’s policies, contracts, or procedures, the vector store instantly finds not just documents containing exact keyword matches, but conceptually relevant information across your entire knowledge base. A query about “employee performance improvement” might surface relevant sections from HR policies, training documents, and past review templates — even if they use different terminology. The LLM then synthesizes these pieces into a coherent, contextual response, effectively turning your static document repository into an intelligent knowledge system that speaks your company’s language.</p><p>And this is where traditional enterprise search dies. No more exact word matching, no more sifting through irrelevant results, no more “did you mean…” suggestions that miss the point entirely. Vector search + LLM is your organization’s collective brain. Ask about “employee performance improvement” and watch as it pulls relevant insights from HR policies, training documents, and review templates — even when they use completely different words to describe the same concepts. The LLM doesn’t just find these pieces; it weaves them together into coherent, contextual answers that actually make sense.</p><h4>Lets Do Better.</h4><p>Vector stores are incredibly powerful, enabling machines to search &amp; find semantically. But can we do better? What if your company’s data is spread across your CRM showing customer interactions, project management systems tracking deliverables, sales pipeline data, product specifications, and regulatory compliance records? Yes, we can vectorize all this, but we’d miss something crucial: relationships.</p><p>This is where we introduce knowledge graphs. Instead of just throwing all data into a vector store, we first create a structured representation of how information interconnects. Imagine a customer node linked to their orders, which connect to product specs, which link to supplier agreements and quality reports. Or a project milestone connected to team members, linking to their expertise records and past successes.</p><p>Building such a knowledge graph creates a powerful first layer in our search architecture. When you ask a question in natural language, the system: 1. Identifies relevant starting nodes, 2. Traverses the graph to find related nodes through explicit relationships, 3. Searches the vector store, but only within these identified nodes, 4. Returns not just semantically similar content, but also the explicit relationships that connect them.</p><p>Let’s break down how we build this locally. If you are interested in code, I also include a snipper in notes below the line (3). We define nodes (entities) and relationships (connections between entities). As we process documents, we use a small and quick model like gpt-4o-mini to identify these elements and their relationships. We store this structure in a simple JSON file, which we can load into a NetworkX graph for efficient traversal.</p><p>NetworkX is crucial here — it’s a Python library that implements graph algorithms. When we search, it helps us find paths (sequences of connected nodes) using algorithms like nx.single_source_shortest_path. This algorithm starts at a node and explores the graph level by level, finding all possible paths up to a specified depth.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/648/1*dh3wzC4CH5TR334aU8h9vQ.png" /></figure><p>Let’s see this in action. Imagine asking: “What projects use technologies similar to Project Alpha?”</p><p>1. The graph traversal finds: — project Alpha node, - its specification documents, - related projects through shared components, — team members involved</p><p>2. The paths reveal relationships: Project Alpha -&gt; SPEC-123 (tech stack doc) Project Alpha -&gt; Project Beta -&gt; ARCH-789 (shared architecture) Project Alpha -&gt; John Smith -&gt; Project Gamma (team lead overlap)</p><p>3. The vector store then searches within these identified documents, finding semantic matches about technologies, frameworks, and architectures.</p><p>This combination of semantic similarity and explicit relationships provides the LLM with both the “what” (similar technologies) and the “how” (project connections).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/950/1*LKZ-Se3Ke6TLCwaDPD4kNQ.png" /><figcaption>source: gpt-4o with code interpreter in an application that I have developed for client created this.</figcaption></figure><p>For small to medium graphs, this local approach works well. However, as your graph grows to millions of nodes and relationships, you’ll want to move to Neo4j. The key difference? Neo4j uses specialized graph algorithms and indexing to find paths without checking every node. Instead of using GPT to identify starting points, Neo4j uses labels, properties, and indexes to quickly find relevant nodes. It can then traverse relationships using optimized graph algorithms that understand the physical storage layout of the nodes.</p><p>By building intelligent retrieval systems that understand both semantic similarity and business relationships, you transform out of the box LLM into your company’s intellectual moat.</p><h4>(1)</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1010/1*cM7bDqaykAKCjYgUkNd73Q.png" /><figcaption>In this example, we use FAISS in its simplest form, just searching vectors. As we assume our library is not large enough yet, we can brute force search and just use FAISS’s optimized distance calculations.</figcaption></figure><h4>(2)</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*roz1xZhV9ITKPkpBS-4f4g.png" /></figure><h4>(3)</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*EQKvScbSHkkyRNPgJ-SUEg.png" /><figcaption>First we define Graph and its methods. We define simple graph literally stored as JSON on disk.</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*AWo2YRZ61m5KK8j-wU7zYg.png" /><figcaption>We use small model to create the nodes and relationships of the Graph from the text we load from docs.</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1020/1*FXAftMewsN8XBZb38Ylf-g.png" /><figcaption>Finally this is how we search. What we end up with is contextually much richer data we can attach to our question before sending to LLM.</figcaption></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=0f5eb7ae20d1" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[LLMs as Infrastructure: The C-Suite Playbook for LLM Implementation]]></title>
            <link>https://georgesalapa.medium.com/llms-as-infrastructure-the-c-suite-playbook-for-llm-implementation-cde4147e5138?source=rss-220b64318f17------2</link>
            <guid isPermaLink="false">https://medium.com/p/cde4147e5138</guid>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[business]]></category>
            <category><![CDATA[management]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[large-language-models]]></category>
            <dc:creator><![CDATA[George Salapa]]></dc:creator>
            <pubDate>Sun, 22 Dec 2024 17:21:01 GMT</pubDate>
            <atom:updated>2024-12-22T17:21:01.678Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/818/1*vn7-SkjpKPcRUiLfobIErA.png" /><figcaption><a href="https://midlibrary.io/styles/eva-hesse">Eva Hesse</a> — Midlibrary</figcaption></figure><p>The wait calculation is 0.</p><p>There are times in history when it pays to wait. Apple has invested a lot of resources and talent to Siri only to see it fully eclipsed by ChatGPT, leaving it no choice but to integrate OpenAI’s flagship product to take over the wheel of its most treasured product: iPhone. Their wait calculation was probably pretty steep. It is easy (and fun) to do this <em>wait calculation with hindsight</em> kind of thing, but almost impossible looking into the future.</p><p>While previous technological revolutions gave us years to adapt, it is not clear if it will be the same with AI. It seems to be rewriting the rules of progress in real-time. That frontier Large Language Models (LLMs) with a good enough Retrieval Augmented Generation (RAG*) layer can replace junior lawyers is now old news. Yes, we are moving very fast towards agentic systems, in which LLMs can perform complete workflows and potentially replace large parts of service economy, and reasoning models (OpenAI’s o1) that are outperforming PhD candidates in a fraction of the time.</p><p>When a technology proves it can match years of human expertise in months, the wait calculation isn’t just zero — it’s potentially negative.</p><p>Amidst this storm of innovation, surrounded by (how could it be otherwise) an immense hype, business models are shaping up. It’s clear that wrappers (applications using LLMs as infrastructure, e.g. Perplexity, Cursor among many other) have significant value and growth potential — product design matters, user switching costs matter.</p><p>Meanwhile, frontier LLM providers are releasing cookbooks and demo applications, demonstrating their eagerness to offer their models as APIs for developers and startups to connect to their most profitable business line — model as utility is here to stay.</p><p>But among all this rapid evolution, where does a non-AI company stand?</p><p>Build, experiment! The wait calculation is 0. Companies should build their own architecture on top of LLMs (their applications) — transforming their static internal knowledge into fluid, instantly accessible intelligence that empowers employees and lifts their cognitive load through AI assistants that initially master company knowledge, then evolve as functions and connections are added to autonomously perform tasks and accelerate routines.</p><p>In the minimum, and I’d like to argue undeniably, custom LLM applications have the potential to: a) transform stale knowledge written in documents and stored on databases to live fluid active intelligence ‘talking’ to its users (employees), b) transform routine (but cognitively demanding tasks — like synthesizing operational reports from dozens of regional offices, identifying performance anomalies, and preparing executive summaries — tasks that middle management repeats quarterly but must approach fresh each time with substantial cognitive effort) into fully automated ‘self-thinking’ workflows, c) remove cognitive friction in data processing where information/data is exchanged between employees internally and between the enterprise and outside world at large.</p><p>When companies build their own solutions, they retain control of their code, their custom prompts and proprietary knowledge bases — essentially creating unique competitive advantages that can’t be replicated by competitors using off-the-shelf solutions.</p><p>Yes, there’s likely a startup offering a solution for every use case today. But consider this: building your own API-first architecture gives you full control over your AI infrastructure and intellectual property. It means your people will actually understand AI, not just use retail product X. It means deep integration with your existing systems, customization that precisely matches your business needs, and importantly independence from third-party roadmaps. You control your data security, optimize costs without middleman markups, and maintain the flexibility to switch. As frontier models improve at a rapid pace and their usage costs continue to fall, the value of your architecture grows — you can literally switch one line of code — the connection to the API endpoint, easily swapping one model for another (the new generation, perhaps).</p><p>It is easy. Start doing things.</p><p>Hire LLM developers (**) who can help you accelerate, start building custom solutions inside your enterprise, deeply integrated with your enterprise systems. Your architecture should be as sophisticated as your brand. Unleash internal innovation by encouraging people to experiment with AI. The bottom-up discovery matters and it matters a lot. The AI transformation needs both technical rigor and human intuition.</p><p>This is where LLMs reveal their most fascinating quality. Remember, LLMs are weird. Unlike previous technologies that followed predictable implementation paths, LLMs thrive or falter based on the unique intuition of their users. Some of your most unexpected wins will come from generalists who discover ways to amplify their natural abilities — those who seem to have an almost supernatural capacity to find novel applications that make them truly superhuman at their jobs (Some of them won’t tell; they will feel as if it’s their secret superpower. Let them feel that — don’t spoil it!). Meanwhile, your specialists will find ways to automate what slows them down, finally free to focus on the highest-value aspects of their expertise. Let your people explore.</p><p>*a method of finding in vast text extracted from documents the relevant bits and injecting it to LLM together with question before it starts <em>writing</em></p><p>**😇</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=cde4147e5138" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Code Interpreter — Giving LLM Hands]]></title>
            <link>https://georgesalapa.medium.com/code-interpreter-giving-llm-hands-d51af669d472?source=rss-220b64318f17------2</link>
            <guid isPermaLink="false">https://medium.com/p/d51af669d472</guid>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[large-language-models]]></category>
            <category><![CDATA[code-interpreter]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[llm-agent]]></category>
            <dc:creator><![CDATA[George Salapa]]></dc:creator>
            <pubDate>Tue, 17 Dec 2024 22:15:33 GMT</pubDate>
            <atom:updated>2025-12-25T19:30:39.800Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/722/1*8gCjvbt9T7CbWvDbdLoNlw.png" /></figure><h3>Code Interpreter — Giving LLM Hands</h3><h4>Adding code execution to Large Language Model (LLM) applications</h4><p>This is for anyone interested in building LLM applications.</p><p>As a brief, somewhat trivial intro, we should explain that LLMs at their core are pattern recognition &amp; prediction systems — they predict the most probable next words (tokens) in a sequence. They don’t actually ‘understand’ or ‘execute’ anything — they predict text patterns. But, boy, do they thrive at it. For all the hype and mystery, LLMs are brute-forcing machines faking intelligence.</p><p>Given some input, they predict the best output… just text. If the input provides instructions on how to launch a tool, the LLM will construct output which we can use in our code to launch a tool (search the internet, open a document, etc.).</p><p>It will not exactly be an eye-opening statement if I say that code interpreter is one of the most important tools an LLM can have at its disposal. LLMs absolutely thrive at writing code. Give them the ability to also run the code they write, and magic starts to happen:</p><ul><li>create visualizations from data</li><li>upload excel and have it changed, cleaned, transformed</li><li>…create stuff out of thin air.</li></ul><p>There are, of course, solutions you can just use off the shelf. Send the code to API and have it do the hard work. But this article is for anyone interested in building their own minimal code execution sandbox for python.</p><p>You can be pretty brutal about it, too. If you just want to build and run your own LLM app (whatever the use case is), you can simply let the LLM write the code and run it in <strong><em>exec()</em></strong>.</p><p>But if we want to build an LLM app for more users, perhaps one deployed to the cloud, we face a crucial challenge: how to safely run arbitrary python code generated by the LLM while maintaining:</p><ul><li>your backend | execution environment isolation</li><li>user isolation.</li></ul><p>It is crucial to ensure that each execution is isolated to prevent interference with other user sessions and the main application.</p><p>The brief notes below explain how jupyter kernels can serve as independent ‘compute engines’ that execute code sent by an LLM agent.</p><p>Think of it as creating a secure sandbox where each user gets their own isolated python interpreter. When LLM decides it needs to generate and run some python code, that code is sent to a fresh kernel process that:</p><ul><li>runs completely separate from your main application</li><li>has its own workspace and memory</li><li>can be (should be) killed and cleaned up after use</li><li>communicates back results via a standardized protocol</li></ul><p>This ensures that even if LLM generates problematic code, it can’t crash your main application or interfere with other users. The kernel acts as a disposable compute unit — spin it up, run the code, capture the output, and shut it down cleanly.</p><p>Step 1: LLM Generates Code</p><ul><li>LLM outputs Python code as part of its response</li><li>code needs to be extracted and sent to kernel for execution</li></ul><p>Imagine the below example being part of your larger agent flow, where you are sending instructions together with user questions/requests to LLM and on API response, you are checking if it invoked a tool … if that tool is kernel you extract the code and send it to:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*GTuY7e6DclIn_a_9-vu59g.png" /></figure><p>Step 2: Start Kernel Process</p><p>We use the subprocess module to start a new jupyter kernel as a separate process. Note that we set cwd to the user&#39;s workspace. Assuming you build an app for multiple users, you want that whatever is created in that sandbox gets saved for each user somewhere -&gt; to her folder, right?</p><p>Note also we create Process ID — PID, which stores the kernel process ID for cleanup.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*jeSIO1TdLjo--nzoR02W8A.png" /></figure><p>Step 3: Execute Code &amp; Collect Output</p><p>Below we send the code to kernel execute (<strong><em>kc.execute</em></strong>) and we start receiving messages from kernel, which we collect to return and show back to LLM so that it knows if it achieved what it wanted.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*q4A3M2r7d6ERld8hewLQEw.png" /></figure><p>Step 4: Cleanup</p><p>After each code execution, I deliberately kill the kernel process using <strong><em>SIGKILL</em></strong> (the most forceful termination signal) to ensure complete cleanup. While this means starting a fresh kernel for each execution — impacting performance — it guarantees isolation and safety for my backend. This tradeoff prioritizes security over speed.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*UhNbbLA6XNUWdQUI_QT2FQ.png" /></figure><h4>Additional detail (don’t need to know):</h4><ol><li>Kernel Connection File — A JSON file containing network and security details required to communicate with a jupyter kernel. It allows your backend to know how to connect to the kernel process securely. In the code above it is created on startup (don’t worry about it).</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/510/1*rkBslc9WPp-2-MuIjRdr9A.png" /></figure><p>2. Process IDs (PIDs) — Identifiers assigned by the operating system to each running process. They are used to track and manage individual kernel processes, and to terminate processes when they’re no longer needed. Each user, each execution gets its own PID. Prevents orphaned processes (bad!).</p><p>3. Communication Channels:</p><ul><li>Shell Channel: sends code to the kernel for execution.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/494/1*sAgWUeeRfOfxfpndCgVzng.png" /></figure><ul><li>IOPub Channel: broadcasts kernel output.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/506/1*LfIkpPTyZU2OSgQrGx_B5Q.png" /></figure><p>4. Kernel Isolation — Each kernel runs as an independent process separate from the main application and other kernels.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/634/1*Tmaei0HnKGShqs7k7gsDNA.png" /></figure><p>5. BlockingKernel client — In our example above we used BlockingKernelClient from jupyter_client library because it is simple:</p><ul><li>synchronous API</li><li>built-in message handling</li><li>automatic channel management</li></ul><p>...which is ideal for our use of one user, one execution at a time.</p><h4>What could happen if you let LLM run arbitrary code in simple <em>exec():</em></h4><ul><li>malicious code could attempt to read or modify sensitive files on the server.</li><li>infinite loops + consume excessive resources, leading to service degradation.</li><li>code might scape the execution environment and affect the host system.</li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d51af669d472" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Pushing Agentic AI to the Limit.]]></title>
            <link>https://georgesalapa.medium.com/pushing-agentic-ai-to-the-limit-967971600e41?source=rss-220b64318f17------2</link>
            <guid isPermaLink="false">https://medium.com/p/967971600e41</guid>
            <category><![CDATA[large-language-models]]></category>
            <category><![CDATA[technology]]></category>
            <category><![CDATA[agentic-ai]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[openai]]></category>
            <dc:creator><![CDATA[George Salapa]]></dc:creator>
            <pubDate>Tue, 10 Dec 2024 18:37:37 GMT</pubDate>
            <atom:updated>2025-12-25T19:20:02.924Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*HQtN_5EmS1qlPsnKtYVTcQ.jpeg" /></figure><p>Building Presentation Agent using OpenAI’s Swarm method.</p><p>Moving past the introduction and getting right to the point, I want to explain in this short article how you can build quasi-agentic LLM applications using basic tool calling and then compare it to OpenAI’s Swarm method*.</p><p>Tool calling is absolutely fine. The structure of your code is fairly simple. User message comes in, you attach system message (each time). You define a set of tools which you send to API with the messages.</p><p>Now here is the important bit, as API starts streaming the chunks of answer back at you, they can now include tool calls which your code uses to invoke tools (e.g. web search to get more info) as the tool returns, you continue looping and don’t break out of the loop. You let the LLM decide when to stop.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/760/1*D0rmJuACRAZDJmjSQwPa9Q.png" /><figcaption>this is code adapted for OpenAI API responses</figcaption></figure><p>This actually works very well. If you give LLM longer task or a chain of tasks, depending on how well you define tools and how well your code performs their execution, it will continue looping thinking about the results and explaining to your user what it is doing. (But make sure to spend enough effort on messages token management -&gt; create a token cutter that trims messages on each additional API run while retaining key information.)</p><p>The downside of this approach is its linear nature. With only one system message attached to each run, complex instructions about when and how to use multiple tools in sequence will, I guarantee you, confuse the LLM and yield unsatisfactory results.</p><p>OpenAI’s Swarm is a distribution of this tool calling across several agents. Each agent has her own system message — instructions that it is reminded of on each API run. Agents can call tools. Agents can hand over to other agents.</p><p>That’s it.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/624/1*z8FYXujblEcGvbPDf-mE1Q.png" /><figcaption>applying the swarm method on presentation creating agentic app</figcaption></figure><p>Typically, there is one orchestrating agent that divides tasks among other agents, who can delegate further or call tools. It is upon you to play around with the system messages and delegation rights until you get to a point where they work in a good harmony. Of course, the more complex is the overall goal, the harder it is to make this work.</p><p>To be clear, if you define delegation rights and system messages, agents (LLMs) will follow them, but the more complex the task you are asking of each agent individually the more likely it is it will fulfill it only partly or completely wrong.</p><p>The key observation is that by using the swarm method and distributing tasks across several agents, you can reduce the complexity load on each individual agent, while enabling the system as a whole to handle more complex tasks.</p><h4>Example:</h4><p>Easy: 1. Customer facing orchestrating agent, who can delegate to 2. Order lookup agent and 3. Refunds agent. Each agent has a relatively simple task (depending on the system they are using to lookup and refund).</p><p>Hard: 1. Orchestrating agent chatting with user about anything, handing over to 2. Presentation planning agent, who can create images, use code in kernel to create charts, read documents, search web, etc. to prepare presentation materials, then handing over to 3. Presentation creator who has to take all this in and create a JSON skeleton that is sent to python-pptx code.</p><p>I tried to create the latter with average results. The system worked autonomously. I merely gave the app a wikipedia link and asked it to create complete presentation using my powerpoint template including charts and images, but the load was still too large in the end on agent 3. who often created malformed slides skeleton.</p><iframe src="https://cdn.embedly.com/widgets/media.html?type=text%2Fhtml&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;schema=twitter&amp;url=https%3A//x.com/i/status/1866270810475843769&amp;image=" width="500" height="281" frameborder="0" scrolling="no"><a href="https://medium.com/media/beb32c0c14f109ac9de965669baebebe/href">https://medium.com/media/beb32c0c14f109ac9de965669baebebe/href</a></iframe><p>This is also where you are hitting limits of the structure. If you are trying to solve anything relatively complex, you will inevitably get to a point where you put too much load on one of the agents.</p><p>In my example, the agent 3. was clearly struggling .. she was the end of the delegation flow seeing all the work of previous agents, the slides plan, the vast text from documents the agents reviewed in order to prepare them, etc. .. her task was to transform the plan to a JSON structure according to certain rules, but this proved too hard. By the time agent 3 saw the tsunami of tasks incoming, it was probably contemplating suicide and I don’t blame her.</p><p>This is where you either move to an even better model or do something clever about the structure, e.g. bypassing agent’s 3. response to a controlling review automatically before returning control back to her or using something like step back prompting.</p><p>Overall, I really liked Swarm as a simple method of creating agentic worflows, which it must be said, can be made much more complex and custom by each developer adding deterministic loops, rich context and other bypassing mechanisms to improve results. As compared to, e.g. langchain, I liked the openness of the method.</p><p>*the original repo called ‘Orchestrating Agents: Routines and Handoffs’ <a href="https://cookbook.openai.com/examples/orchestrating_agents">available here</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=967971600e41" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>