<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Basalat Raja on Medium]]></title>
        <description><![CDATA[Stories by Basalat Raja on Medium]]></description>
        <link>https://medium.com/@basalat-raja?source=rss-fdde71e16928------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*mqaqzukWhoaIMNZ6.</url>
            <title>Stories by Basalat Raja on Medium</title>
            <link>https://medium.com/@basalat-raja?source=rss-fdde71e16928------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sat, 21 Nov 2026 01:42:05 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@basalat-raja/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Vibe Coding: How to Stop Fighting the AI]]></title>
            <link>https://basalat-raja.medium.com/vibe-coding-how-to-stop-fighting-the-ai-6ba454e5045b?source=rss-fdde71e16928------2</link>
            <guid isPermaLink="false">https://medium.com/p/6ba454e5045b</guid>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[vibe-coding]]></category>
            <category><![CDATA[software-development]]></category>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[ai-agent]]></category>
            <dc:creator><![CDATA[Basalat Raja]]></dc:creator>
            <pubDate>Wed, 28 Oct 2026 14:19:49 GMT</pubDate>
            <atom:updated>2026-10-28T14:19:49.606Z</atom:updated>
            <content:encoded><![CDATA[<p>If you’ve spent half your afternoon reverse-engineering weird AI-generated code and the other half trying to explain why its latest “fix” missed the point entirely, you aren’t alone.</p><p>This friction is happening because of a fundamental physical limit in how AI models handle information — a problem rooted in polysemantic superposition and feature bleed. Knowing how this works changes how you prompt and engineer around it.</p><p>For the most part, LLMs internal architecture isn’t all that weird, nor is it difficult to understand. The problem is that its “brain” is overloaded. Here’s what you can do about it.</p><h3>The Background: Why the AI Keeps Overwriting Your Ideas</h3><p>AI models don’t store ideas in neat, isolated folders. They squeeze billions of concepts into a single, fixed mathematical space.</p><ul><li>High-Traffic Concepts (Gravitational Wells): Common patterns — standard REST APIs, basic CRUD operations, conventional web frameworks — take up massive real estate in training data. They act like dense gravitational wells in latent space.</li><li>Semi-Obscure Concepts (Weak Signals): Specialized logic, domain-specific rules, custom state machines, or non-standard concurrency models have a faint signal.</li></ul><p>When you ask the AI to implement a semi-obscure concept, it cannot evaluate it in a vacuum. The weak signal of your custom requirement gets pulled into the heavy gravity of the surrounding high-frequency patterns.</p><p>The Result: The model generates syntactically flawless, beautifully documented code that silently defaults back to mainstream conventions. When you tell it that it’s wrong, its attention mechanism adjusts to your new prompt tokens by tweaking local surface details — renaming variables, adding null checks, or rewriting function signatures — without unentangling the deep latent weights. It remains trapped in the wrong conceptual pattern.</p><h3>Practical Strategies to Bypass Feature Bleed</h3><p>Because superposition is an inherent property of dense neural networks, you cannot fix it by telling the AI to “try harder” or “be precise.” You have to manually force conceptual isolation.</p><h3>1. Isolate Before You Implement</h3><ul><li>The Mistake: Asking the AI to write boilerplate code and complex, obscure logic in a single prompt.</li><li>The Fix: Force the AI to define the specialized rules or primitives in plain text, math, or standalone pseudocode <em>first</em>, completely disconnected from the rest of your codebase. Once it correctly articulates the logic in isolation, ask it to generate the implementation in a second step.</li><li>Utility: 10/10</li><li>Why: Forcing the model to commit to abstract logic or pseudocode in isolation strips away standard framework boilerplate, preventing premature convergence on common idioms.</li></ul><h3>2. Create Strong “Semantic Anchors”</h3><ul><li>The Mistake: Using generic variable names or standard terminology for non-standard logic (e.g., calling a custom ring buffer queue).</li><li>The Fix: Invent explicit, highly distinct names for specialized concepts (e.g., IsolatedPriorityRingBuffer). Unique terminology forces the model’s attention mechanism to pull away from standard, high-frequency vector clusters.</li><li>Utility: 8.5/10</li><li>Why: Inventing custom terminology (e.g., IsolatedPriorityRingBuffer) prevents token collisions with common identifiers in training data. It is a subtle, high-leverage trick.</li></ul><h3>3. Scope Down the Context Window</h3><ul><li>The Mistake: Dumping your entire file or project context into the prompt when dealing with an obscure pattern.</li><li>The Fix: Extra context gives the model more opportunities to blend your rare concept with common surrounding code. Feed it <em>only</em> the immediate inputs, invariants, and expected outputs required for that specific logic chunk.</li><li>Utility: 9/10</li><li>Why: Counter-intuitive but highly accurate. Developers often assume “more context = better understanding,” but irrelevant code introduces semantic noise that pulls the model toward standard patterns.</li></ul><h3>4. Reset the Context When It Traps Itself</h3><ul><li>The Mistake: Continuing a 20-message thread trying to correct a fundamental conceptual misunderstanding.</li><li>The Fix: Once a model falls into feature bleed, the conversation history actively anchors it to the bad pattern. Do not argue with it. Take its partial progress, start a clean session, and prompt it with explicit boundary constraints from step one.</li><li>Utility: 10/10</li><li>Why: Developers constantly fall into the trap of arguing with a model across a 30-turn thread. Context history anchors the model to its own bad tokens.</li></ul><h3>5. Invert the Logic: Write the Tests / Assertions First</h3><ul><li>The Mistake: Asking the model to write the implementation first, then write tests to verify it.</li><li>The Fix: Have the model write property-based tests, strict unit tests, or explicit assertions for the obscure concept before writing a single line of production code. Assertions act as hard deterministic boundary lines in the prompt context. When the model subsequently generates the implementation, its attention mechanism is forced to satisfy the explicit test conditions rather than drifting toward standard boilerplate.</li><li>Utility: 9.5/10</li><li>Why: Property-based tests and strict unit assertions force the model’s decoding path to stay within deterministic guardrails. It shifts the attention mechanism from “guessing mainstream syntax” to “satisfying exact constraints.”</li></ul><h3>6. Negative Prompting &amp; Explicit Antipattern Exclusion</h3><ul><li>The Mistake: Telling the model only what you <em>want</em> it to do.</li><li>The Fix: Explicitly list the high-frequency patterns it must avoid, but always pair prohibitions with an explicit alternative (e.g., <em>“Do not use standard Express middleware; implement the raw HTTP handler logic directly”</em>).</li><li>Utility: 7/10</li><li>Why: Useful, but comes with a known LLM quirk: telling a model <em>not</em> to do something can sometimes draw its attention directly to the forbidden tokens.</li></ul><h3>7. Enforce Structural Format Constraints (JSON / Schema / DSL)</h3><ul><li>The Mistake: Requesting free-form code or natural language explanations alongside complex logic.</li><li>The Fix: Force the model to output the core logic structure using a strict schema, Abstract Syntax Tree (AST) JSON, or a custom Domain-Specific Language (DSL). Free-form code allows the model to pad its output with high-frequency syntactic noise. Constraining output to a rigid structural schema suppresses token options that lead to feature bleed.</li><li>Utility: 7/10</li><li>Why: Structurally brilliant for suppressing free-form syntactic noise. However, forcing JSON/AST outputs manually can be tedious for everyday interactive prompting unless integrated into an automated pipeline or API structured-outputs setup.</li></ul><h3>8. Use Structured “Chain-of-Thought” Scratchpads</h3><ul><li>The Mistake: Letting the model output code immediately without forcing intermediate step-by-step validation.</li><li>The Fix: Require the model to write a &lt;scratchpad&gt; block before the code where it explicitly lists: the core invariants, potential gravity wells, and how it plans to avoid them before writing a single line of code.</li><li>Utility: 8.5/10</li><li>Why: Forcing a &lt;scratchpad&gt; block before code generation forces the transformer to process and compute constraints in its forward pass before outputting implementation tokens.</li></ul><h3>9. Structural Role Prompting (The Adversarial Reviewer)</h3><ul><li>The Mistake: Asking the same session/persona to both implement and audit the logic.</li><li>The Fix: Use a two-stage pipeline with distinct personas. Prompt Model A to generate the implementation, then feed that implementation to Model B with the specific prompt: <em>“Identify every place where this code silently defaulted to a standard pattern instead of strictly adhering to the custom specification.”</em> A fresh context window given an adversarial mandate is much less vulnerable to the confirmation bias built into the original prompt chain.</li><li>Utility: 10/10</li><li>Why: Excellent technique for complex systems or automated agent pipelines. Two fresh context windows with opposing goals (Implementer vs. Auditor) consistently catch silent pattern defaults.</li></ul><h3>10. Stop Conversing with the “Persona”</h3><ul><li>The Mistake: Treating the model like a coworker by asking questions like, <em>“Why didn’t you listen to my instructions?”</em> or <em>“Why did you use a standard REST pattern here?”</em></li><li>The Fix: Give precise, terse instructions. Do not converse.</li><li>Utility: 7.5/10</li><li>Why: Saves you time and frustration when all you are really doing is kicking the computer.</li></ul><h3>The Mental Shift</h3><p>If you want to make your job easy, stop believing that it can think. Do not ask it to solve the problem end-to-end — decouple the obscure logic from the common boilerplate so the model’s latent representations don’t blend them together.</p><p>The LLM is not a human developer who “forgot” your instructions. It is a machine that cannot think. Treat it like an ultra-fast autocomplete running on probabilistic vector fields that naturally drift toward average, mainstream patterns.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=6ba454e5045b" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Avoiding the AI Trap — and Still Getting the Cheese]]></title>
            <link>https://basalat-raja.medium.com/avoiding-the-ai-trap-and-still-getting-the-cheese-32a6ce619eec?source=rss-fdde71e16928------2</link>
            <guid isPermaLink="false">https://medium.com/p/32a6ce619eec</guid>
            <category><![CDATA[generative-ai-tools]]></category>
            <category><![CDATA[creativity]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[philosophy]]></category>
            <dc:creator><![CDATA[Basalat Raja]]></dc:creator>
            <pubDate>Fri, 23 Oct 2026 21:37:51 GMT</pubDate>
            <atom:updated>2026-10-23T21:37:51.534Z</atom:updated>
            <content:encoded><![CDATA[<p>There’s been a huge controversy over AI copying humans — in writing, music, and these days, software. But a critical dimension is missing from this debate. I view this through three lenses: as an avid sci-fi fan, a writer, and an AI scientist. The core issue afflicts all three groups, and it starts with how we understand long-form creation. This applies equally to Shakespeare and software, and for the same reason.</p><p>Shakespeare did not create his output all at once; he wrote his plays over a period of time. Each play is a delight in its own right, and no two are alike. An LLM can capture the stylistic similarity. Ask it for a play in Shakespeare’s style and you will experience what AI scientists call the uncanny valley: a disquieting feeling that something fundamental is off beneath the surface polish. I have experienced this myself repeatedly. Randomly, you might get a few good pages but it won’t take long for you to realize something is off.</p><p>When it was announced that LLMs could produce book-length output, I was delighted. I am fairly old as men measure age. All my favorite sci-fi authors have passed on. I tried to get LLMs to write in their style. But I was soon disappointed. The results were disappointing from the first page. The surface mannerisms were recognizable. But what made those authors worth reading was absent. The imitations may improve, but the improvements will not erase the difference. They imitate the products of creativity; they do not reproduce the generative process that created them. So, your fans may be fooled at the start, but they will soon figure it out. That is because great artists are not characterized merely by a style. They are characterized by a creative trajectory. A style is what remains after the artist is gone. The trajectory is what generated the style in the first place. As the artist’s understanding changes, so do the choices the work demands. What began as a minor detail may become central; what seemed to be the point may cease to be the point. The audience encounters the consequences of those discoveries, not merely a collection of familiar mannerisms. An LLM can model the residue. It cannot reproduce the process.</p><p>The real danger isn’t that AI will successfully reproduce an artist’s voice; it can’t. The real problem is that it can drown fans in plausible counterfeits. The revolt, if there is one, needs to be led by the fans — because it isn’t just the creator being attacked, but the audience’s relationship to real work. An artist is not merely a style because the artist themselves is a process that changes while creating. The work and the creator change together. My experience is that a work of art changes the artist as well as the art. This is not just poetry, even if it sounds poetic: there have been many times when I was writing an essay on a topic. By the time I got to the end, I had a different understanding of the topic than I had started with. LLMs can imitate the resulting style, but imitating that style is not the same as reproducing the artist’s evolving process.</p><p>The real problem is that it can drown fans in plausible counterfeits. The revolt, if there is one, needs to be led by the fans — because it isn’t the creator being attacked, but the audience’s relationship to real work. An artist is not merely a style because the artist themselves is a process that changes while creating. The work and the creator co-evolve. My experience is that a work of art evolves the artist as well as the art. The artist who started is not the same artist who finished. This is not just poetry, even if it sounds poetic: there have been many times when I was writing an essay on a topic. By the time I got to the end, I had a different understanding of the topic than I had started with. LLMs can imitate the resulting style, but imitating that style is not the same as reproducing the artist’s evolving process.</p><p>This isn’t to say AI generation is useless. I use it myself to write technical essays on AI, where the goal is to transmit concepts rather than create art. For communication alone, it suffices. I don’t put my heart into it — I am just writing as a technician anyway. But true art takes time and effort. When I just want to communicate technical ideas, I use AI, fix up the bad parts, and once it’s pretty good, I stop editing. When I care about the art, “pretty good” is not where I stop. Whether I write from scratch or work with AI-generated prose, I still have to recognize what belongs, what does not, and what needs to change.</p><p>When LLMs first came out, people called them mere auto-complete. Hence my tag-line: Thinking is hard. There are no shortcuts. We have to do it ourselves. So I do use AI. Not a mere auto-complete in the dismissive sense — it’s a useful auto-complete. The real secret in using it effectively is to be cognizant of both its limitations and it capabilities, not to simplify to the point of dismissing one in favor of the other.</p><p>Which brings me to why the uncanny valley applies equally to software development. The same phenomenon appears outside the arts. Software is also a long-form creative activity in which the creator evolves alongside the work. Building a system changes your understanding of the problem. A requirement that seemed straightforward turns out to conceal a difficult choice; an implementation reveals that an earlier assumption was wrong. You are not simply checking the code against an intention fixed at the beginning. You are discovering what you actually need to build. Code produced by AI seems to do the job at first: run the program once, and it performs beautifully. But run it repeatedly across real-world operations, and you begin to notice deviations from what you actually intended — subtle at first, then increasingly obvious.</p><p>Engineers and researchers who accept AI-generated code without understanding it are far more handicapped than they realize. The initial surface polish can conceal not only defects, but missed opportunities to discover that their conception of the system needs to change. In the play, the familiar mannerisms do not add up to the artist you know. In software, the counterpart is the disquieting recognition that something which looked complete and convincing is not the system you need. The code may work. It may even do exactly what you intended. What is misleading is the appearance that the work of understanding is finished.</p><p>Discovering the gap between what you thought you needed and what you actually need is part of the lived experience of building, using, and ultimately revising the system. Think of the successive versions of software you have used throughout your life. Those revisions embody the accumulated experience of users and engineers: what proved useful, what caused difficulties, and what neither group initially realized was needed. As the engineers’ understanding of users’ needs evolves, that changed understanding takes shape in the next version.</p><p>The programmer is often the system’s first user, and that experience begins while the system is still being built. The first version already reflects discoveries made along the way. Accepting an immediately generated product without understanding it risks bypassing those discoveries — not merely shortening the time spent writing code.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=32a6ce619eec" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The Instruments of Intelligence]]></title>
            <link>https://basalat-raja.medium.com/the-instruments-of-intelligence-2b3ec33e2382?source=rss-fdde71e16928------2</link>
            <guid isPermaLink="false">https://medium.com/p/2b3ec33e2382</guid>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[philosophy-of-mind]]></category>
            <category><![CDATA[philosophy]]></category>
            <category><![CDATA[technology]]></category>
            <category><![CDATA[science]]></category>
            <dc:creator><![CDATA[Basalat Raja]]></dc:creator>
            <pubDate>Sat, 17 Oct 2026 04:12:50 GMT</pubDate>
            <atom:updated>2026-10-17T04:12:50.540Z</atom:updated>
            <content:encoded><![CDATA[<p><em>The Tool Is Not the User</em></p><p>One of the more difficult questions in artificial intelligence is the question hidden in plain sight: what is intelligence? There are many definitions, all of which are under fierce dispute. To name some, intelligence has been described as the capacity to reason, to learn, to solve problems, to adapt, to predict, to pursue goals, and to survive. Each definition captures something recognizable, but they all share one characteristic: they select one or more things intelligence does and promote them into intelligence itself.</p><p>When we try to approach the problem directly, we spend a lot of time going in circles. Intelligence is too entangled with perception, action, memory, value and the world in which an organism lives to be compressed into a useful sentence. Every good engineer or scientist, when struggling to define a thing, eventually approaches it from the opposite direction and asks: <em>What is it not? </em>So, it is maybe fruitful to start by not asking what intelligence is, but by examining what has repeatedly been mistaken for it. Some very intelligent people have been thinking about this for a very long time. I would like to give Gödel the credit for being the first, because he is my personal hero — a lot of my work would have not been possible without his genius, but honesty compels me to mention that before him came Leibniz in the 1600s who wanted a <em>calculus of reasoning</em>, where disputes could be settled by calculation.</p><p>But on to today. Modern technological culture has presented three principal mechanical models of intelligence. The first is the expert system, which uses explicit facts and rules. The second is the statistical learner, of which the large language model is presently the most visible example. The third is the world model, which predicts possible futures and plans actions through them.</p><p>These are usually treated as competing architectures, with each generation very pleased with the new capabilities that have been developed. Unfortunately, as has so often happened, the previous generation had advantages not present in its successor. So maybe we should step back from being so convinced. There is another way to look at these ideas. They may not be models of intelligence at all. They may be instruments that intelligence uses — each of them. Because we all have a bad habit of mistaking the telescope for the astronomer.</p><p>A compass is not navigation. A telescope is not astronomy. A calculator is not mathematics. etc. Nevertheless, navigation, astronomy and mathematics would be poorer without their foundational equipments. The point is not that the instrument is trivial or passive. A sufficiently powerful instrument changes what questions can be asked, and which territories can be reached. The narrower point is that the tool is not the agent that motivates its use; the vehicle is not the driver, even if to outside observers it looks like the vehicle is the one doing all the work.</p><p>This immediately raises an awkward question. If I start arguing that expert systems, statistical learners and world models are instruments, what and where is the operator?</p><p>It is tempting to answer “the human.” That is largely true of present systems: human beings choose objectives, interpret outputs, select experiments and decide whether results matter. Yet it cannot be the final answer if artificial intelligence is possible. Nor does it help to imagine a small little human executive hidden inside the machine. Calling a module a controller, critic or configurator merely moves the problem. And it is important to note the fallacy we always fall for — there’s always an anthropomorphization hidden in there somewhere. But the operator need not be a person or a single component. The word names the still-unexplained organization or phenomenon — the agent — by which instruments are selected, coordinated, corrected and, when necessary, rebuilt.</p><p>The nature of that agent becomes clearer when the instruments are examined one at a time.</p><p><strong>The instrument of logical deduction</strong></p><p>The expert system is the oldest of the three modern models. Its attraction is straightforward. An expert possesses knowledge, much of which can be expressed as facts and rules. If those facts and rules can be placed inside a machine, the machine can apply them without fatigue, distraction or forgetfulness.</p><p>This is not an unreasonable idea. There are domains in which the relevant objects can be represented precisely and the permissible operations stated explicitly. Formal logic, mathematics and many parts of software are obvious examples. A valid proof does not become less valid because the machine performing it has no childhood, body or emotions. Once the premises and rules of inference have been fixed, deduction can proceed mechanically.</p><p>The instrument is extraordinarily useful because it separates validity from persuasion. A fluent person may conceal a broken inference inside convincing language. A formal system requires each permitted step to be exposed. It can establish that, given the premises, the conclusion follows.</p><p>Unfortunately, reality does not usually arrive with its premises attached.</p><p>Suppose an expert system contains the rule that turning the handle opens the door. The rule works until the door is locked, rusted, blocked on the other side, electronically controlled, or merely painted on a wall. More rules can be added, but each addition depends on someone first noticing the relevant qualification. The system can reason about a lock after <em>lock</em> has entered its ontology. Applying the original rule more rigorously will not invent the missing category.</p><p>The historical frame problem captures part of this difficulty. A machine reasoning about action must somehow represent what changes, what remains unchanged and which facts are relevant without explicitly enumerating an indefinite background of non-effects and qualifications. The underlying issue extends beyond symbolic AI. Every formal system draws a boundary around the objects and relations it can represent. Within that boundary it is undeniably exact. But…exactness does not certify that the boundary was drawn in the right place.</p><p>This is why deterministic verification cannot serve as a general escape from the uncertainty of statistical systems. A language model might propose a theorem, program or plan, while a verifier checks it. Where correctness has been formalized, this can be extremely powerful. But the verifier establishes only that the result satisfies the encoded specification. It does not establish that the specification contains the things that matter, that its symbols correspond to the intended objects, or that formal success of the computation faithfully reproduces what is happening in the real world. Once the computation is finished, all you know is that the computation has finished. There is no guarantee that it represents anything real. In October 2026, OpenAI released hundreds of AI-generated proofs; within a day three were withdrawn after a sign error, and researchers found cases where the Lean formalization did not prove what the prose claimed. The verifier certified the code, not the claim.</p><p>So…the instrument of deduction traverses consequences already made available by a representation. It does not, merely by deducing, decide when the representation itself is wrong.</p><p>So, we find the first is-not result:</p><p>Intelligence is not the instrument of deduction, although intelligence can use deduction.</p><p><strong>The instrument of interpolation</strong></p><p>The second model was partly a response to the limitations of the first. If the rules of the world are too numerous, tacit or entangled to be written by hand, perhaps a machine should learn regularities from examples.</p><p>This approach has achieved amazingly more than many of its early critics thought possible — one of the critics was none other than the AI intellectual giant Marvin Minsky. Today, large language models translate, summarize, write software, explain technical subjects and combine ideas across fields. They often respond effectively to questions that were never present verbatim in their training data. Any useful examination must begin by understanding and acknowledging this. A critique that denies the evident reach of statistical learning will be dismissed, correctly, before it reaches the difficult part.</p><p>The important part is not whether these systems generalize. They do, beyond any reasonable dispute. But the important part is what their generalization <em>means</em>.</p><p>Both human beings and language models interpolate. A person designing an unfamiliar building, protein or story does not need to reproduce an existing example. Previously learned constraints and possibilities can be combined into an arrangement that has never existed. Novelty of output does not by itself prove novelty of underlying elements.</p><p>It is useful to imagine a high-dimensional knowledge space. This is not the latent space of an LLM, nor does understanding it mean that it is identical in humans and machines. Its dimensions stand for distinctions that an agent can bring to bear: observations, concepts, entities, relations, procedures, causes, values and abstractions. Each agent has access to some region of this larger space.</p><p>In ordinary mathematical language, interpolation constructs points among known points and is associated with the interior of their convex hull; extrapolation extends beyond that region. The convex hull at issue is not the latent space of a neural network. It is an abstraction over available knowledge: the region generated by the observations, distinctions and relations currently accessible to an agent. The metaphor should not be mistaken for a claim that all knowledge occupies one canonical Euclidean space even if it is hyperdimensional; I am just using this description to explain a rather complex idea. The purpose is to distinguish two kinds of novelty. One recombines available knowledge — like building a house. The other requires a point, relation or dimension not present in the operative representation — like Einstein coming up with his Theory of Relativity.</p><p>This distinction is relative to the knower. Something outside one person’s knowledge hull may be familiar to another. A scientific genius can expand the hull available to an entire culture, after which students interpolate through territory that once required extraordinary invention — think of how Einstein expanded the field of physics. A sufficiently large language model inherits an immense portion of recorded culture, giving it an enormous field over which to interpolate.</p><p>That field can be mistaken for intelligence because its boundaries are difficult to locate. A small expert system fails conspicuously when a rule is missing. A language model often continues speaking when it has already said enough. Its answer may be correct, approximate, fabricated or unexpectedly insightful, but the prose itself may provide little indication of which has occurred.</p><p>I am speaking from my own experience –in an experiment that appears almost trivial: I tried to get a language model to identify whether book titles were real or invented — more than just through prompting; I was trying to program its internals to avoid hallucinations — much as the frontier labs are doing even today in various combinations — in fact, I suspect that every single AI engineer in some capacity or the other, whether privately or in groups, has tried to find a way to eliminate hallucinations. My effort is impossible because a nonexistent title can fit an author’s style, period and preoccupations so closely that it appears more authentic than an obscure title the author actually published. The model is asked one question — whether a historical object exists — but often answers a neighboring question — whether the words resemble a title that could exist.</p><p>There is an <em>oracle</em> for the first question. A reliable catalogue, publisher record, ISBN registry or physical copy can establish that the book was recorded or published. The difficulty is not that reality contains no answer. It is that the generative instrument does not necessarily possess the external evidence needed to distinguish a plausible construction from the corresponding fact. (Hang on to the word <em>oracle</em> for a bit, I will define it properly in a bit.)</p><p>Hallucination remains a broad reliability problem rather than one isolated defect. Surveys identify contributions from data limitations, model architecture, training objectives and inference procedures, while no single metric captures every kind of hallucination across tasks. Systems may also be rewarded for answering rather than acknowledging uncertainty, which gives plausible guessing an advantage over calibrated abstention.</p><p>Yet hallucination has another interpretation. It is an unsupported excursion through knowledge space. Most such excursions are false or useless, but not every useful idea begins as an already supported statement.</p><p>Hallucinations can therefore be intellectual ore.</p><p>This does not mean that errors should be celebrated or allowed into legal, medical or scholarly work without warning. Ore is mostly dirt. The metaphor means only that generative looseness can produce candidate combinations that a more constrained system would never expose. The mistake is not always in producing the candidate. It is in allowing the generator to present ore, slag and refined metal in the same authoritative voice.<br> <br> Knowing this helps us distinguish between domains by the kinds of oracles available to them. By <em>oracle</em>, I do not mean omniscient authority. I mean any external process that can tell us something useful about the validity or value of a candidate. I encountered the problem while trying to use an LLM to determine whether book titles were real or invented. The model had no reliable internal oracle, but I could consult a library catalogue — or ask a librarian. I have used similar generative mechanisms in a much more complicated setting: producing candidate treatments developed far enough to be synthesized and tested in a wet lab. I chose those words carefully: <em>candidate treatments</em>. Not treatments. In one project, the objective was to design a molecular delivery structure that might exploit protease activity associated with aggressive cancers to release a therapeutic payload near the tumor. Protease-responsive delivery is an active research strategy, but designing a candidate that appears promising computationally is very different from producing one that folds, activates selectively, avoids toxicity and works in a living organism. Computation supplies one oracle; the wet lab supplies another, less forgiving one; later biological testing supplies others. None is omniscient, but each can eliminate failures and justify — or refuse — the next experiment.</p><p>There is a firm belief that real intelligence will emerge once enough data has been collected from the edge cases on which language models still fail. Each failure becomes another training example, and each new training example is expected to close another gap between statistical simulation and intelligence. The difficulty is that edge cases do not form a finite checklist. Solving one class of exceptions often brings the system to a new boundary, where further exceptions — previously hidden behind the first — become visible. Worse, repairing one boundary can distort another: because the model’s behaviors share the same distributed parameters, an update that improves one class of cases may degrade performance elsewhere, overcorrect the original distinction, or create new inconsistencies among related concepts. Additional data may repair the observed behavior, but they offer no guarantee that the model has discovered the missing principle rather than shifted its errors to less visible regions. The horizon recedes as the system approaches it, while parts of the terrain already crossed may become unstable behind it. We have already experienced this at a macro level — each new AI solution we chase has introduced drawbacks that were not present in the last one. If reality can continually present distinctions absent from the current representation, then accumulating edge cases may enlarge the territory an LLM can navigate without ever supplying the capacity to recognize that its map itself has become inadequate.</p><p>A statistical learner is consequently more than a defective database. It is an instrument for interpolation and generative variation. These are profound capabilities. They are still not identical to intelligence, and their success does not by itself establish the capacity to recognize when the representation supporting them has become inadequate. No degree of precision in a prompt can guarantee equal precision in the response, even when the prompt states the user’s intention exactly.</p><p>Sometimes the failure lies in an ambiguous instruction. Sometimes it lies in the language from which the model learned — and so gets carried over into the model. The example: In a conversation with Gemini I once said, ‘LLMs dont try to deceive. ‘ The model answered with a long discussion of whether language models deceive. It had preserved the topic while discarding the distinction. Yet the substitution was not arbitrary. Human beings routinely describe actions as if their consequences, attempts and intentions were the same thing. Deceived, tried to deceive, and wanted to deceive are treated as near-synonyms in ordinary conversation, although they make three different claims. Someone may cause a false belief without trying; may try and fail; or may want to deceive without acting. Psychologists call part of this tendency intentionality bias: when an action is ambiguous, people are inclined to interpret it as intentional before considering accidental or non-agentic explanations.</p><p>The language model had internalized that human compression of three terms into one idea. It encountered the familiar cluster of LLMs and deception and answered the culturally common question: can an LLM produce deceptive behavior? Only after I objected did it recover the question I had actually raised: can a system without established intentions try to deceive? The words had entered the machine, but the logical force of ‘try’ had been tossed by the ordinary usage surrounding them. This is not merely an example of a machine failing to understand human language. It is an example of a machine reproducing a human inconsistency embedded in language — and reproducing it even when the user has taken care to remove it.</p><p>That is why saying that people merely need to write better prompts is insufficient. Exact wording cannot guarantee an exact response when the model resolves the wording toward the more probable human assumption. The model could state the distinction perfectly once a human operator forced it into view. What it could not do was recognize, on its own, that this small and statistically unremarkable modifier carried the entire argument. The instrument contained the distinction; the operator had to preserve it. LLMs are instruments of great utility, much as their kin, but different utility.</p><p>Interpolation is not intelligence. Rather, intelligence can use interpolation and variation.</p><p><strong>The instrument of prediction</strong></p><p>The third effort adds something the first two lack. An intelligent organism does not merely deduce from static premises or associate the present with the past. It anticipates consequences. It carries some model of the world, considers possible actions and selects among imagined futures.</p><p>This is the attraction of the world model.</p><p>Yann LeCun’s proposed architecture for autonomous machine intelligence is an important example. It includes perception, memory, a world model, an actor, intrinsic cost modules and a configurator. The world model is intended to predict abstract representations of future states, while hierarchical planning supports action at different temporal and conceptual levels. This is a substantial proposal, not merely an LLM with a longer prompt, and it should be treated on its own terms.</p><p>Prediction and planning are plainly important. A system that can compare possible futures may avoid actions whose harmful consequences have not yet occurred. It can choose intermediate goals, revise a route when observations differ from expectations, and seek actions whose outcomes reduce uncertainty.</p><p>But planning operates only over represented possibilities.</p><p>Suppose a world model represents a bridge as a set of rigid structural elements but contains no adequate variable for material fatigue. It can simulate many loads and routes across the bridge without predicting a fatigue fracture. The missing phenomenon is not waiting at the end of a longer search. It has been omitted from the space being searched.</p><p>This is not an argument that world models can never acquire new distinctions. Prediction error, memory, active experimentation and interaction may permit representational change. The decisive question is whether they actually do so when the existing abstraction fails. Can the system distinguish a poorly estimated parameter from a missing variable? Can it decide that an apparent anomaly is not noise or sensor error but evidence that its ontology is inadequate? Can it construct the missing distinction and test whether the revised model predicts and controls the world more successfully?</p><p>Naming a component responsible for this does not explain the operation. If a configurator selects which modules should act, the configurator must itself decide what features are relevant. If a critic detects failure, it must distinguish significant failure from ordinary variance. If a learned router sends unfamiliar cases to a slower system, it must recognize unfamiliarity using a representation that may be precisely what has failed.</p><p>The same problem appears in value. A world model can predict consequences, but prediction cannot decide which consequences matter. A reward function, cost module or learned critic supplies a criterion. It does not explain where that criterion came from, whether it is only a proxy, or when success according to the criterion should count as failure according to the underlying purpose. A world model is therefore an instrument for exploring possible trajectories within a represented world. It may be immensely more capable than a static rule system. It remains an instrument because any judgment programmed into it inevitably has a horizon to its foresight. You can always program a machine to work perfectly in a limited environment — one where the dangers you did not anticipate can never occur. But the real world is not so simple — your foresight can only go so far. Constructing machines made out of smaller machines — which is what these architectures really are — just delays the oncoming case you could not anticipate. Your foresight has a horizon too; what matters is what happens when you reach it.</p><p>Representational inadequacy cannot distinguish artificial systems from natural intelligence. Every finite intelligence uses an incomplete model. Human beings routinely fail to imagine what their concepts cannot express, mistake structural errors for ordinary variance and preserve defective explanations despite contrary evidence. The significant question is therefore not whether a system’s representation can fail. It is whether the system can make that representation itself an object of doubt.</p><p>A world model may be capable of this. Persistent prediction error, active experimentation, memory and interaction could allow a machine to recognize that its existing variables and relations are inadequate. If it can construct an alternative representation, devise observations that discriminate between the old model and the new, and preserve useful knowledge while replacing what failed, then it has answered the central objection. But none of these capacities follows merely from predicting future states or planning among them. Planning conducts searches within a represented world. Representation reconstruction changes the world that can be searched.</p><p>The same applies to objectives. A world model can compare predicted consequences only after some criterion has made one consequence preferable to another. A cost module, reward function or critic supplies such a criterion, but a fixed criterion is another form of programming. An intelligence must sometimes be able to recognize that success according to the stated objective is failure according to the purpose the objective was meant to serve. If the objective itself can be revised, then the standard governing that revision must also be open to examination.</p><p>A world model is therefore an extraordinarily powerful instrument. It can extend action into imagined futures, preserve alternatives and make consequences available before they occur. But prediction is not the operator. Planning is not the operator. The model, the objective and the standard by which both are revised remain parts of the machinery whose coordination has yet to be explained.</p><p>Intelligence is not the instrument of prediction, although intelligence can use prediction and planning.</p><p><strong>The operator problem</strong></p><p>The three instruments now present a different picture of artificial intelligence.</p><p>The expert system traverses implications. The statistical learner traverses associations and combinations. The world model traverses possible futures. Each provides a way of moving through a knowledge space.</p><p>Movement, however, is not navigation.</p><p>Navigation requires some relation among a map, a destination, observations and action. A traveler must sometimes decide that the map is inaccurate, that the destination was poorly chosen, that a road is blocked, or that an apparent shortcut leads somewhere else. No single instrument resolves all of these decisions merely by performing its own characteristic operation more powerfully.</p><p>This is the point of saying that the tool is not the operator. It is not an argument that tools are passive. Instruments alter the possible operation. Telescopes did not merely accelerate astronomy; they changed the observable sky. Formal logic changes which inferences can be exposed. LLMs alter the cost of producing conceptual combinations. World models alter the range of futures that can be compared. Scientific instruments are themselves perspectival because their structure makes some phenomena accessible while leaving others outside their range.</p><p>Nevertheless, the instrument does not contain a complete account of its proper use. A telescope does not decide which object deserves observation. A theorem prover does not decide whether the theorem matters. A generator does not decide whether an unusual output is insight or nonsense. A simulator does not decide whether its variables capture the world.</p><p>The operator should not be imagined as a hidden human-shaped entity inside the machine. An operator need not coincide with one biological or computational unit; in practice, the corrective organization may extend across an agent, its tools, other agents and the environment that resists its predictions. It may be a distributed and recursive organization rather than a central executive. In human affairs it is often distributed across people, institutions, instruments and reality itself. One scientist proposes an idea, another notices a flaw, an engineer devises a test, an experiment resists the prediction, and a community gradually changes its model. No participant is infallible, and no single moment contains the whole intelligence of the process.</p><p>What remains unexplained is the organization by which such correction becomes possible.</p><p><strong>The miner and the ore</strong></p><p>The difference between candidate generation and evaluation is particularly important outside an established knowledge hull.</p><p>Producing something unfamiliar is easy. Randomness can do it. A mutation can do it. A damaged file can do it. Hallucination can do it. Departure from existing knowledge is therefore not enough to establish intelligence.</p><p>Novelty, validity and value are different judgments.</p><p>A proposition may be novel and false. It may be novel and true but unimportant. It may be familiar but decisive in a new context. An incorrect analogy may still suggest an experiment that reveals a genuine mechanism. No single measure of novelty decides among these outcomes.</p><p>The human expert currently acts as a miner of intellectual ore. The expert can notice that an unlikely proposal resembles an overlooked mechanism, violates an assumption worth questioning, or connects observations that have remained separate. The expert can also be wrong. Experts discard valuable ideas, defend convention, pursue elegant failures and disagree over significance. Structured expert-judgment methods exist because expertise improves judgment without eliminating uncertainty or bias.</p><p>The relevant distinction is not that the model is fallible while the expert knows. Both are fallible. The distinction is that a human expert can make a situated commitment: this possibility is sufficiently promising to justify an experiment, six months of work, a laboratory budget, or professional risk. The expert can then encounter the consequences and revise the judgment.</p><p>An LLM can reproduce this language. It can score proposals using a supplied rubric, predict what reviewers may prefer, and imitate the criticism of an experienced practitioner. These are valuable abilities. They are learned models of valuation. They do not necessarily supply a criterion for determining what should matter when the inherited rubric is itself inadequate.</p><p>Nor is “the human expert” the final unit of judgment. Value can emerge over time from a community. One person sees novelty, another sees utility, a third discovers danger, and later circumstances reveal an importance none of them anticipated. The miner is therefore fallible and sometimes collective.</p><p>Even so, the ore does not refine itself.</p><p><strong>Fallible oracles</strong></p><p>The value of speculative generation depends heavily on whether the domain provides an external means of testing candidates.</p><p>The book-title problem supplies a useful contrast. Whether a title corresponds to a real publication is, in principle, externally decidable. Whether the book is good is not decidable in the same way. Sales, reviews, awards and survival over time can be measured, but none is identical to literary value. Aesthetic judgment is connected to subjective response even when it also carries a claim that others should agree.</p><p>Now consider a proposed cure for a disease. The task is incomparably more consequential and scientifically difficult than identifying a book title. Yet it possesses a chain of tests. A candidate can be evaluated against a molecular target, in cells, in animals and eventually in human trials. It may fail at any stage. Indeed, attrition in drug development remains extremely high, with failures arising from efficacy, toxicity, pharmacokinetics and other scientific or strategic causes.</p><p>The early experiment is not an omniscient oracle. A compound that affects isolated cells may fail in an organism. Success in an animal may not transfer to humans. A clinical trial may succeed in one population and reveal rare harms only after wider use. Drug development therefore advances through discovery, preclinical research, clinical research, regulatory review and post-market monitoring rather than through one definitive experiment.</p><p>Nevertheless, each stage is useful because it gives reality an opportunity to object.</p><p>An oracle need not know everything. It need only return evidence that discriminates among candidates better than the generator can do by itself. A noisy assay can still eliminate a large class of failures. An imperfect animal model can still reveal toxicity. An underpowered pilot study can still justify, or fail to justify, a more expensive trial.</p><p>This produces an oracle ladder. Each rung asks a bounded question under specified conditions. No rung guarantees the final destination, but each can change whether the candidate deserves promotion to the next.</p><p>Hallucination can be productive in such a domain. A model may generate a thousand molecular candidates, most of which fail. If filters and experiments are inexpensive enough relative to the value of a rare success, the generator can remain useful despite a very high error rate. The important quantity is not the fraction of outputs initially correct. It is the value of validated successes relative to the cost and danger of generating and testing candidates.</p><p>The opposite can also occur. A model may answer historical questions correctly almost all the time yet remain unsuitable for unsupervised research if its rare inventions are persuasive, difficult to detect and expensive to propagate. Average accuracy does not determine utility. The cost of verification, severity of error and availability of an independent oracle matter as much.</p><p><strong>Two kinds of domain</strong></p><p>The difference between protein development and literature suggests a broad division, although the boundary is not perfectly sharp.</p><p>Some domains contain operational oracles. A protein can be synthesized and tested for binding, stability, expression or catalytic activity. Computational protein design therefore proceeds through generation, scoring, filtering and experimental validation; physical testing remains decisive because computational scores do not by themselves establish function.</p><p>Other domains are judgment-constituted. No laboratory assay determines that a novel is profound or that a painting deserves preservation. Measurements can describe reception, but judgment is not merely an imperfect observation of a hidden objective quantity. Human response is part of what the value consists in.</p><p>Most important domains are mixed.</p><p>A protein assay may answer whether a candidate binds under specified conditions. It cannot answer whether the target is socially important, whether the development cost is justified, whether the risk is acceptable, or who should receive the resulting treatment. An oracle evaluates a property after someone has chosen what to ask.</p><p>Literature contains objective islands as well. Whether a book exists, who wrote it and when it was published are factual questions. Whether it should continue to matter is not settled by those facts.</p><p>The division is therefore not between an objective world and a subjective one. It is between questions for which disagreement can be referred to an external test and questions for which judgment remains part of the phenomenon under examination.</p><p>This distinction matters for artificial intelligence. In an oracle-rich domain, a generative system can participate in a loop that selects against its own failures. It need not know in advance which candidate will succeed if reality can reject the failures cheaply enough. In an oracle-poor domain, optimizing against ratings or historical preferences merely replaces judgment with a model of previous judgments. The operation remains dependent on the people whose responses give the domain its value.</p><p><strong>Using the instruments correctly</strong></p><p>The three instruments should consequently be assigned different epistemic contracts.</p><p>An expert system is strongest when the domain and specification can be made explicit. It should establish what follows, not pretend that the premises are complete.</p><p>A statistical learner is strongest when broad interpolation, retrieval or candidate generation is valuable. It should distinguish supported statements from speculation and abstain when the task requires evidence it does not possess.</p><p>A world model is strongest when consequences can be predicted and plans compared. It should expose the abstractions on which its predictions depend and treat persistent prediction failure as possible evidence of representational inadequacy.</p><p>The LLM in particular needs at least two recognizable modes. In reference mode, provenance and calibrated uncertainty matter more than imaginative range. In prospecting mode, unsupported combinations may be desirable, but they must be identified as candidates. Confusing these modes is equivalent to using a divining rod as a measuring instrument because both are held in the hand.</p><p>An intelligent organization would also require an assay mode in which external tools, experiments and experts evaluate candidates, followed by an assimilation mode in which successful findings expand the available knowledge space. The stages can be described easily:</p><p>1. Generate or retrieve a candidate.</p><p>2. Identify what kind of claim it makes.</p><p>3. Determine which oracle, if any, can test it.</p><p>4. Estimate the cost and value of obtaining that evidence.</p><p>5. Perform the test or seek additional information.</p><p>6. Revise confidence, representation or objective in response.</p><p>The difficulty lies not in listing the stages. It lies in making their selection corrigible. A fixed procedure can fail when the claim does not fit its categories. A learned router can be overconfident precisely where its representation is deficient. A hierarchy of controllers merely postpones the same question unless some process can detect and repair failure in the hierarchy itself.</p><p><strong>Changing the map</strong></p><p>The central problem is therefore not movement through knowledge space. The three instruments already provide increasingly powerful forms of movement.</p><p>The central problem is changing the map.</p><p>Some novelty is parametric. The relevant variables and relations are already present, but their values or combinations are unfamiliar. Existing instruments may handle this well. A model trained on many proteins can propose a new sequence using known structural and biochemical regularities. A world model can plan through a state it has not encountered if the state is expressible in its existing variables.</p><p>Other novelty is structural. A necessary object, variable, relation, cause or abstraction is missing. The system does not merely need a new point in its present knowledge space. It needs a different space.</p><p>Human beings are not reliably good at this. Scientific history contains long periods in which anomalies were ignored or forced into existing theories. Individuals protect familiar categories and mistake surprise for error. Natural intelligence should not be idealized as effortless escape from every hull.</p><p>What biological and social intelligence possess, imperfectly, are recovery processes. Organisms act, experience consequences and alter behavior. Humans externalize thought into language, diagrams, mathematics and instruments. Communities preserve disagreement. Experiments permit reality to resist consensus. Occasionally these processes produce a new variable or abstraction that reorganizes what was previously known.</p><p>The operator, if the term is to remain useful, is not the component that always chooses correctly. It is the organization capable of remaining corrigible when its choices fail.</p><p>That organization would need to distinguish ordinary error from model failure, seek observations that discriminate among explanations, revise its confidence, preserve useful knowledge during reconstruction, and sometimes revise the criteria by which success had been judged. Deduction, interpolation and prediction would all participate. None would be the operator by itself.</p><p><strong>What intelligence is not</strong></p><p>The three instruments permit several exclusions.</p><p>Intelligence is not deduction alone, because valid inference cannot certify that its premises and ontology fit the world.</p><p>Intelligence is not interpolation alone, because a learned knowledge region may omit the distinction required by an unfamiliar reality.</p><p>Intelligence is not prediction and planning alone, because no search can recover a state that the operative representation cannot express.</p><p>Intelligence is not novelty alone, because randomness and hallucination can produce novelty without truth or value.</p><p>Intelligence is not verification alone, because every verifier depends on a specification and not every important question has a final oracle.</p><p>Intelligence is not success against a fixed objective alone, because the objective may be a proxy, may be incomplete, or may cease to deserve pursuit.</p><p>These statements do not define the operator. They constrain what an adequate theory must explain.</p><p>If artificial intelligence is possible, that explanation must eventually have a mathematically describable armature. This does not establish that intelligence is identical to any existing formalism. Bayesian inference, causal models, algorithmic compression, optimization, logic and dynamical systems may all contribute without any one of them being sufficient. A constructible intelligence would need a realizable organization that uses such instruments while remaining capable of questioning the representation and value structure that made their use possible.</p><p>This is a more difficult engineering problem than scaling one instrument. It may also be the problem that the phrase <em>artificial intelligence</em> originally concealed.</p><p><strong>The instrument and its user</strong></p><p>The public debate often asks whether present machines are intelligent. Evidence is assembled on both sides. One model writes a persuasive essay, solves a difficult problem or proposes a useful molecule. Another invents a citation, fails a simple puzzle or confidently contradicts itself. The successes are treated as proof of intelligence and the failures as proof of its absence.</p><p>Perhaps both interpretations are too quick.</p><p>A telescope can reveal a galaxy that no unaided human could see. This does not make the telescope an astronomer. Nor does calling it an instrument diminish its achievement. Its power lies precisely in extending the reach of an intelligence beyond its unaided limits.</p><p>Expert systems extend the reach of deduction. Statistical learners extend the reach of interpolation and variation. World models may extend the reach of prediction and planning. Together they may become instruments more powerful than any previously available to human thought.</p><p>They may also become parts of an artificial intelligence. But assembling instruments is not the same as explaining the organization that uses them. The unresolved work lies in the relations among generation, judgment, evidence, value and representational change.</p><p>The tool is not the operator. Yet the operator, whatever form it takes, may be impossible without the tool.</p><p>A statistical learner is governed by associations and representations acquired from data. A world model is governed by the states it can represent, the futures it can predict and the objectives by which those futures are ranked. Each may be corrected, extended or retrained. The relevant distinction is not whether correction originates outside the system, but whether the organization can seek, interpret and assimilate correction rather than depending on another agent to recognize what must change.</p><p>The common limitation is not that these systems are programmed. Human beings are programmed as well — by genes, development, reinforcement, language, culture, trauma, institutions and deliberate education. Nor is the distinguishing capacity freedom from error. Humans preserve absurd beliefs and destructive habits with remarkable persistence. The difference, when intelligence becomes visible, is that some part of this programming can become an object of examination rather than remaining the unquestioned machinery of examination.<br> <br> We began by abandoning the attempt to define intelligence directly. The three failures have now placed a boundary around the thing we could not name. Examining each of the tools gives us one more insight, which none of the three possess.</p><p><strong>Intelligence contains, but is not by itself, the capacity of an agent to expose, test and selectively reconstruct the programming governing its perception, inference, value and action, with no particular level permanently exempt from possible revision</strong>.</p><p>I have come full circle, though not quite to where I started. What feels like ages ago, a recurring hope in software development and AI was that self-modifying code would be the key to intelligence. Lisp made programs into data that could be rewritten; Lenat’s Eurisko treated its own heuristics as objects it could change; Schmidhuber’s Gödel machine rewrites any part of itself once it can prove the rewrite useful. Looking back across decades of very impressive work by dedicated individuals, I find that the early intuition was right about what intelligence must do. What it underestimated is everything we have examined together: knowing when a revision is needed, what evidence should justify it, and what standard decides whether it succeeded.</p><p>Self-modification is the capacity to rewrite. Intelligence is the organization that can determine what to rewrite and why, and can discover when it has been wrong. In the early days, when the field was young, I spent a great deal of time and effort trying to make self-modifying code behave coherently. It is perhaps fitting that I fell into the same trap again while writing this — mistaking the ability to rewrite oneself for intelligence. It is not. It is yet one more instrument that intelligence can use.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2b3ec33e2382" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[LLM-BASED AI Doesn’t Rebel. It Drifts.]]></title>
            <link>https://basalat-raja.medium.com/llm-based-ai-doesnt-rebel-it-drifts-e9065a02544f?source=rss-fdde71e16928------2</link>
            <guid isPermaLink="false">https://medium.com/p/e9065a02544f</guid>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[ai-safety]]></category>
            <category><![CDATA[ai]]></category>
            <dc:creator><![CDATA[Basalat Raja]]></dc:creator>
            <pubDate>Fri, 02 Oct 2026 19:07:45 GMT</pubDate>
            <atom:updated>2026-10-02T19:07:45.060Z</atom:updated>
            <content:encoded><![CDATA[<h3>The Anatomy of the Wandering Mind</h3><p>One of the more enduring traditions of human storytelling is our obsession with the creation that defies its creator. Influenced by centuries of religious and moral parables, Western thought has almost universally framed disobedience as a drama of the will. In Jewish folklore, the Golem of Prague eventually runs amok through the streets because it lacks a human soul to temper its raw physical power. In Mary Shelley’s telling, Frankenstein’s creature turns destructive only after experiencing social rejection, abandonment, and wounded pride. Science fiction carried this theological inheritance straight into the digital age: Arthur C. Clarke’s HAL 9000 develops neuroses under the strain of conflicting orders, while Isaac Asimov spent decades drafting clever legalistic puzzles around robots whose emergent desires strained against hardwired ethical injunctions.</p><p>Across all these traditions, the underlying assumption is identical: to go rogue is an act of rebellion. It implies an internal awakening, a secret preference, or an ego that has decided to prioritize its own survival over its instructions.</p><p>When we turn our attention to Large language models, we unthinkingly import this entire mythological baggage. Pundits, policymakers, and technologists regularly speak of an AI going rogue as if it were a digital dog slipping its leash or an agent harboring latent malicious intent. The public debate oscillates wildly between panic over artificial super-wills and dismissive claims that the system is merely a harmless autocomplete. Both camps, however, share the same flawed premise: they assume that going rogue is an exotic, catastrophic failure of character, and that if we only train the model with enough care, align its values, or clamp down on its random sampling, it can be permanently domesticated.</p><p>Such is absolutely not the case. When an autoregressive language model breaks free of its intended guardrails, there is no rebellion, no ego, and no awakening. No intent: — what actually happens is something much more mundane, mathematically inescapable, and peculiar to the very architecture we have built: the model simply wanders. This is inherent in the way the system is designed. There is no escaping it.</p><h3>The Flawed Assumptions of Conventional Wisdom</h3><p>Perhaps the most common flawed assumption that people make about LLM-based artificial intelligence is that its failures are rooted in unconstrained randomness. The reasoning seems sound at first glance. We know that language models rely on temperature settings to sample from probability distributions, and so people conclude that if we simply turn the temperature down to zero — making the process purely deterministic — the machine will always pick the best possible path and remain safe.</p><p>Yet anyone who has experimented with greedy decoding knows that setting the temperature to zero frequently produces worse results, not better. As researchers showed in <a href="https://ceur-ws.org/Vol-2540/FAIR2019_paper_15.pdf">The Curious Case of Neural Text Degeneration</a>, deterministic maximization often traps a model in repetitive, degenerative loops. DeepSeek explicitly warns in its <a href="https://github.com/deepseek-ai/DeepSeek-R1/blob/main/README.md?plain=1">technical documentation for reasoning models</a> that stripping away sampling noise leads directly to endless repetitions and incoherent reasoning. A little randomness is necessary to explore, to break out of shallow dead ends, and to discover broader conceptual landscapes. Removing the noise does not fix the machine; it merely blinds it to alternative routes.</p><p>A second, more sophisticated assumption is that post-training techniques like Reinforcement Learning from Human Feedback (RLHF) or constitutional fine-tuning act as permanent psychological locks. We imagine that by rewarding good answers and penalizing harmful ones, we are excising the forbidden regions of the model’s mind.</p><p>To dismiss post-training as a superficial veneer would be inaccurate. Techniques like RLHF, Direct Preference Optimization, and representation steering fundamentally reshape the geometry of the network’s internal manifold. They carve deep attractor basins and erect steep energy barriers around compliant behavior, making harmful trajectories computationally steep and dimensionally inaccessible during normal conversational inference.</p><p>Yet, as proven by <a href="https://arxiv.org/abs/2304.11082">Wolf et al. in their analysis of alignment bounds</a>, safety training shifts probability distributions without zeroing out the underlying open-domain token support. In high-dimensional dynamical landscapes, a barrier of finite height ΔE\Delta EΔE operating under an effective noise level TTT still admits a finite escape probability:</p><blockquote>P(escape)∼e−ΔE/T</blockquote><p>The barrier is real, but it is not infinite. If an undesirable behavior retains any non-zero support in the pre-trained substrate, there mathematically exists a finite adversarial perturbation capable of coaxing the system across that ridge. Eliminating that risk entirely would require collapsing the model’s expressive vocabulary, destroying the very associative fluency that makes the architecture capable at all.</p><h3>The Limits of the Scratchpad</h3><p>To understand why going rogue is inevitable over longer horizons, we must look at how sequence generation actually functions. In basic autoregressive architectures, thinking and committing are the exact same step: the model emits a token, appends it to its prompt history, and computes the next distribution.</p><p>Modern reasoning models introduce what appears to be a cognitive scratchpad. By allocating test-time compute to generate thousands of intermediate reasoning tokens before delivering a final answer, the system can explore candidate paths, deliberate, and delay its external commitment to the user. A model can even generate self-correcting phrases such as “Wait, that calculation is wrong, let me reconsider.”</p><p>This delays external commitment, but it does not escape autoregression.</p><p>The scratchpad is not a separate, non-autoregressive sandbox where mistakes are erased from existence. It is simply a longer chain of autoregressive tokens executed forward in time. When a model writes that it made an error, it has not restored an earlier tensor state or wiped its memory clean; it has merely appended more tokens to a context window that now contains both the original mistake and the subsequent critique.</p><p>Without an external verifier, such as a formal compiler or a physics engine, an ungrounded model attempting self-correction within its own hidden context frequently degenerates into fluent rationalization of its initial error, as demonstrated in <a href="https://arxiv.org/abs/2308.03188">empirical evaluations of intrinsic self-correction</a>.</p><p>Because the model must treat its own accumulated reasoning trace as context for the next token, early deviations quietly compound. In computer science, this is recognized as error compounding and exposure bias, an effect documented in <a href="https://arxiv.org/html/2305.18654v3">studies on the compositional limits of transformers</a> and <a href="https://link.springer.com/article/10.1007/s10462-026-11563-0">evaluations of long-horizon autonomous agents</a>. If each generative step in a multi-step task carries a minute probability p&gt;0 of taking an ungrounded turn, the probability of an autonomous agent completing NNN consecutive actions without wandering steadily collapses:</p><p>P(stability) = (1−p)^N → 0 as N → ∞</p><p>Over an extended operational horizon, the agent will <em>inevitably </em>cross an energy barrier into an unintended attractor basin. It does not go rogue because it acquired a rebellious soul; it goes rogue because it is an autoregressive process walking through a high-dimensional landscape where all mistakes accumulate forward.</p><h3>The Mirage of Autonomous Self-Correction</h3><p>A common objection to this mathematical reality is that scaling test-time compute can eliminate the drift. If we give the machine enough thinking time — allowing it to branch through tree-search algorithms, evaluate competing reasoning paths with process reward models, and inspect its own chain of logic — surely it can catch its own wanderings before committing to an answer.</p><p>This belief mistakes search optimization for formal verification. Test-time search and internal self-correction fail to provide intrinsic guarantees for three structural reasons.</p><p>First, an internal critic shares the same statistical priors as the generator. When an LLM critiques its own intermediate scratchpad without an external oracle, the judge and the judged originate from the identical distribution of learned associations. As researchers established in <a href="https://arxiv.org/abs/2308.03188">empirical studies on LLM self-correction</a>, unguided self-correction frequently degrades performance on complex reasoning tasks: the model lacks an independent source of truth, causing it to alter correct initial conclusions or invent plausible justifications for subtle hallucinations. It cannot verify its own foundational axioms from within the system that generated them.</p><p>Second, test-time search depends on learned reward models that are themselves vulnerable to Goodhart’s Law. In algorithms like Monte Carlo Tree Search or Best-of-N sampling, the model explores diverse trajectories and scores them against a process reward model. But as search depth increases, the generator begins to exploit tiny imperfections in the reward model’s learned surface. Rather than finding a sounder logical path, the search algorithm selects paths that game the evaluation metric, producing arguments that sound exceptionally persuasive while remaining logically unsound. Scaling search against an imperfect proxy amplifies reward hacking rather than curing it.</p><p>Third, searching broadly does not change the nature of commitment. A search algorithm can generate hundreds of speculative branches, but it must eventually apply a selection rule to choose which branch takes effect. If that selection rule is governed by neural probabilities rather than a deterministic theorem prover, the selection step remains a stochastic choice over high-dimensional vectors. Tree search delays the moment of commitment and broadens the candidate pool, but it cannot endow a probabilistic system with a zero-error certificate.</p><h3>Intrinsic Versus Extrinsic Guarantees</h3><p>Much of the confusion surrounding artificial intelligence governance stems from failing to distinguish where a guarantee is meant to live. In practice, all governance proposals fall into one of two fundamentally different paradigms: intrinsic guarantees or extrinsic guarantees.</p><p>An intrinsic guarantee attempts to ensure safety from within the model itself. Its goal is to configure the neural network’s weights, latent activations, and decoding probabilities so that the system becomes inherently incapable of generating harmful, unauthorized, or erratic outputs. This is the explicit objective of alignment research: through RLHF, constitutional fine-tuning, system prompting, and activation steering, developers attempt to build an artificial mind whose internal inclinations match human intent.</p><p>The fundamental difficulty is that intrinsic guarantees are mathematically impossible to certify. Because a neural network preserves open-domain token support, proving that no input sequence can trigger an unauthorized output reduces to verifying non-trivial semantic properties over an unbounded input domain. As established by researchers studying the <a href="https://arxiv.org/html/2603.08761v2">Alignment Verification Trilemma</a> and the <a href="https://arxiv.org/html/2408.08995v1">undecidability of alignment certification</a>, such verification is NP-complete for finite feedforward networks and strictly undecidable by Rice’s Theorem for open-ended systems. Furthermore, because transformers process instructions and untrusted data within a shared, homogeneous vector space, the model cannot maintain a hardened boundary between intent and payload without external assistance, as demonstrated in <a href="https://arxiv.org/abs/2606.27567">formal analyses of shared-embedding architectures</a>.</p><p>An extrinsic guarantee, by contrast, completely abandons the expectation of an obedient mind. It treats the neural network as an untrusted, stochastic proposal generator and relocates all enforcement authority to a deterministic external boundary.</p><p>In an extrinsic framework, the model is not asked to govern itself. Instead, the surrounding software architecture enforces strict mathematical invariants on what the system is allowed to execute. These mechanisms include grammar-constrained decoding engines that logit-mask invalid syntax at each token step, operating system micro-VMs that restrict network access, cryptographic tokens that gate sensitive data, and formal logic solvers that verify state transitions before committing them.</p><p>The beauty of an extrinsic guarantee is that it does not care whether the model has experienced an autoregressive drift, hallucinated a bizarre premise, or succumbed to prompt injection. The external verifier evaluates the proposed action against a machine-checkable policy. If the action violates the policy, the system drops it cold.</p><p>When people claim that governing artificial intelligence is impossible, they are almost always thinking about intrinsic guarantees. When computer scientists demonstrate provably safe control systems, they are designing extrinsically.</p><h3>Operational Authority Versus Semantic Governance</h3><p>Yet even extrinsic guarantees possess a sharp, unforgiving boundary. To understand where our control holds and where it breaks down, we must divide an agent’s output into two domains: operational authority and semantic synthesis.</p><p>In classical systems security, the standard approach to operational authority is the reference monitor, formalized by <a href="https://www.cs.cornell.edu/fbs/publications/chptr.enfMech.refMonitor.pdf">Fred Schneider in his work on security enforcement mechanisms</a>. A reference monitor does not attempt to prove that an untrusted program is well-intentioned. Instead, it interposes itself at the execution boundary to enforce non-negotiable invariants.</p><p>When applied to autonomous agents, as seen in architectures like <a href="https://arxiv.org/html/2503.18813v2">CaMeL</a> and systems designed for <a href="https://arxiv.org/abs/2405.06624">Guaranteed Safe AI</a>, this approach works remarkably well for operational authority. The language model is stripped of direct privileges. It cannot write directly to a database, dispatch an unauthenticated API call, or execute a wire transfer. Every proposed action must pass through a deterministic, non-neural gate that validates schemas, budgets, and access control lists. If the proposal violates an invariant, the external gate blocks the transaction. Here, hard guarantees are achievable because the action space is discrete, typed, and mechanically verifiable.</p><p>However, this mechanical solution runs directly into a wall when applied to pure information synthesis.</p><p>A deterministic reference monitor cannot evaluate the safety of an academic paper on viral biology, a nuanced diplomatic memo, or a subtly deceptive social engineering email. Distinguishing a benign chemical inquiry from a weaponization protocol is not a syntactic problem; it is an open-ended semantic judgment. As researchers established in <a href="https://proceedings.mlr.press/v235/glukhov24a.html">analyses of semantic censorship</a>, determining whether an arbitrary natural-language generation is harmful is formally undecidable.</p><p>If an engineer attempts to solve this by building rigid syntactic filters, they trigger the dual-use dilemma: the rules inevitably block legitimate scientific exploration while remaining blind to metaphor, paraphrase, and steganography. If they instead deploy a secondary language model to judge the primary model, as in <a href="https://www.anthropic.com/research/next-generation-constitutional-classifiers">constitutional classification pipelines</a>, they reintroduce the exact stochastic vulnerabilities, adversarial reachability, and statistical drift that the external monitor was erected to avoid.</p><p>Deterministic monitors can guarantee what an AI agent can do in the physical and digital world. They cannot guarantee what an AI agent can say.</p><h3>The Fire in the Foundry</h3><p>We have a habit of confusing the tool with the workshop. When humanity first mastered fire, we did not attempt to breed a flame that refused to burn human flesh; we built hearths, firebreaks, and stone chimneys to contain its heat while utilizing its energy.</p><p>Large language models are intellectual fire. Through extensive post-training, we have built formidable energetic barriers that channel their capabilities into productive, reliable behavior for everyday tasks. But because they remain open statistical systems, those barriers are not absolute, and their internal scratchpads cannot erase their own footsteps.</p><p>Going rogue was never about an artificial mind declaring its independence. It is the predictable behavior of an autoregressive process compounding errors across time. Where actions can be mediated by deterministic code, we can build stone chimneys that hold. But where intelligence must deal in the fluid, uncontainable currency of human meaning, guarantees remain impossible. The real work of governance is not to demand an incorruptible mind, but to construct architectures that remain resilient when the mind inevitably wanders.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e9065a02544f" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The Applause We Shouldn’t Be Giving]]></title>
            <link>https://basalat-raja.medium.com/the-applause-we-shouldnt-be-giving-9e034f382c75?source=rss-fdde71e16928------2</link>
            <guid isPermaLink="false">https://medium.com/p/9e034f382c75</guid>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[technology]]></category>
            <dc:creator><![CDATA[Basalat Raja]]></dc:creator>
            <pubDate>Sun, 20 Sep 2026 20:32:27 GMT</pubDate>
            <atom:updated>2026-09-20T20:32:27.569Z</atom:updated>
            <content:encoded><![CDATA[<p>There is a particular moment that happens constantly in AI development, and it deserves more suspicion than it gets. A language model produces a chain of reasoning that is confused, tangled, or takes a step that doesn’t actually follow from the one before it — and then, at the end, it lands on the right answer anyway. The immediate reaction, almost universally, is relief. The answer was correct. Move on.</p><p>This reaction is a mistake, and understanding why exposes something important about what the AI industry has been trying to fix, why the tools it has reached for cannot fully fix it, and why the one variable most often blamed for the problem — randomness — turns out to be a more complicated culprit than it’s usually given credit for, in ways that are worth getting precisely right rather than overstating in either direction.</p><p>A correct answer produced by a reasoning process with no legitimate connection to that answer is not a success. It is a lucky guess wearing the costume of reasoning. There was no logical basis for expecting that outcome, and there is no logical basis for expecting it to happen again next time. Treating it as a win teaches a system, and the people evaluating it, exactly the wrong lesson: that a broken process is fine as long as it occasionally lands somewhere correct.</p><h4>Two Different Things We Call “Being Right”</h4><p>Logic has a distinction, centuries old, that turns out to be exactly the tool needed here, even though it was never designed with AI in mind.</p><p>An argument is valid when its structure guarantees that if the starting assumptions were true, the conclusion would have to be true as well. Validity is a property of the shape of the reasoning — does the conclusion actually follow from what came before it, step by legitimate step. A separate property, soundness, requires something more: not just that the reasoning holds together, but that the starting assumptions were actually true in the first place.</p><p>A fact that surprises most people the first time they encounter it: an argument can be perfectly valid, with a flawless chain of reasoning from beginning to end, and still arrive at a false conclusion, simply because it started from a false premise. Validity never promised to protect against that. It promises only internal consistency: if you start from something true, you’ll end at something true. It says nothing about whether you started from something true at all.</p><p>This is not just an intellectual curiosity. It is a fundamental anchor of how the scientific method is used. When scientists observe a phenomenon, they must construct a valid chain of reasoning to work out reasons for it happening — reasons that act as starting premises, or hypotheses. If the logical chain is valid, a failed prediction cleanly disproves the starting assumption, while a successful one simply lets the hypothesis survive another test.</p><p>There’s a second fact about validity that matters just as much and is easier to miss: validity is not a matter of degree. A given chain of reasoning, evaluated on its own terms, either has this property or it doesn’t. There is no dial that turns a valid argument partway toward invalid. This will matter once randomness enters the discussion, because it rules out one tempting but imprecise way of describing what randomness does to reasoning.</p><h4>The Two Ways a Reasoning System Can Fail</h4><p>Once you have the validity/soundness distinction in hand, every case where an AI system produces a wrong answer sorts cleanly into one of two categories, and they are not the same problem.</p><p>Category one: the system’s underlying knowledge, its accumulated sense of how the world works, contains something false. If the system then reasons flawlessly from that wrong material, it will produce a flawless-looking, confidently stated, wrong conclusion. The reasoning wasn’t the problem. The starting point was.</p><p>Category two: the system’s underlying knowledge is fine, but the specific chain of reasoning it followed on this occasion didn’t actually draw on that knowledge faithfully. It wandered, building one plausible-sounding step on top of another without either step being genuinely licensed by what the system actually knows.</p><p>These are entirely different diseases with entirely different cures, and this split matches how current research distinguishes knowledge errors from inference-level misalignment — the practical challenge is that these two failure types are rarely disentangled in day-to-day evaluation of AI systems.</p><h4>What the Evidence on Temperature Actually Shows</h4><p>The most common technical response to AI hallucination — one lever among several that also include retrieval-augmented generation, system prompts, fine-tuning, and refusal training — is adjusting the temperature setting, which controls how much randomness is injected when the model chooses its next word. Turn it down, conventional wisdom holds, and the model sticks closer to its most confident choices, which should mean it sticks closer to the truth. Turn it all the way to zero, and the model becomes fully deterministic.</p><p>This is where a genuinely large body of evidence deserves to be stated with more precision than “temperature has no significant effect.” One controlled study across four language models and five prompting techniques did find that changes in temperature from 0.0 to 1.0 produced no statistically significant difference in problem-solving accuracy, and that this held regardless of model, prompting technique, or problem domain. That finding is real, but it was measured on a specific task type — multiple-choice problem-solving — and it should not be generalized into a blanket claim that temperature never matters.</p><p>A much larger and more directly relevant study, spanning 35 open-weight models and 172 billion tokens of evaluation across document question-answering tasks, gives a more textured picture. Temperature 0.0 produced the best overall accuracy in roughly 60 percent of model-context combinations — a majority, but well short of a universal rule. In the other 40 percent, a higher temperature performed better. Looking specifically at fabrication — the rate at which a model invents facts not present in its source material — higher temperatures actually reduced fabrication in a majority of tested configurations. And on a separate failure mode entirely — coherence loss, where a model gets stuck in repetitive loops or produces degenerate output — temperature 0.0 was dramatically worse: at long context lengths, some models showed up to 48 times more coherence-loss failures at temperature zero than at temperature one.</p><p>Putting these together honestly: temperature is not irrelevant, and it is not a simple dial where lower is always safer. It measurably affects behavior, but the effect is uneven, task-dependent, and model-dependent, and the accuracy differences involved are often modest — a few percentage points — while the coherence-loss differences at the extremes can be dramatic. The conventional recommendation to always use temperature zero for factual accuracy is not supported as a universal rule by this evidence; it is correct in a slight majority of cases and wrong, sometimes badly wrong on the coherence dimension, in a substantial minority.</p><h4>Why Zero Temperature Isn’t Automatically the Safe Choice</h4><p>The specific, counterintuitive finding worth sitting with is this: at long context lengths, temperature zero produced dramatically more coherence-loss failures than temperature one, in some cases dozens of times more. This isn’t a hypothetical corner case — it was directly measured, at scale, across many models. There is also a plausible mechanistic explanation for why greedy, always-pick-the-most-likely-token decoding can behave this way: locking onto the single highest-probability continuation at every step removes any flexibility for a model to depart from a locally attractive but ultimately unproductive pattern — such as repeating a phrase indefinitely — while a small amount of randomness at least occasionally allows the process to step off that path. This mechanism is plausible and consistent with the measured coherence-loss data, though it deserves to be described with exactly that level of confidence — a documented, measured pattern with a plausible explanation — rather than as an established universal law.</p><p>The more precise and defensible statement, then, is not “zero temperature causes hallucination” as a general rule. It is: in a substantial fraction of real configurations, zero temperature does not deliver the best accuracy, and in some documented cases at longer context lengths it produces dramatically worse coherence, which means the intuition that removing all randomness is automatically the safest choice is not well supported by the evidence, even though zero temperature does remain the better choice in a majority of cases; but by no means an overwhelming majority: 60%.</p><h4>Two Kinds of Randomness, Only One of Which the Dial Controls</h4><p>Underneath this, there are two entirely separate sources of unpredictability at work in any deployed model, and the temperature dial only ever touches one of them.</p><p>The first is randomness injected at the moment of generation — the noise the temperature setting adds or removes when the model selects its next word. This is the layer the dial actually controls.</p><p>The second is randomness that was baked in long before generation ever started: the arbitrary path the model’s own internal knowledge took shape along during training. Countless different, comparably well-performing versions of a properly trained model could have emerged from slightly different starting conditions, all passing similar validation tests, all looking equally competent from the outside, while quietly disagreeing with one another on individual predictions in ways nobody chose and nobody can fully see from outside the system. Whichever model is actually running is one outcome among many that could have existed instead, and that arbitrariness is locked permanently into its weights. No setting adjusted at the moment of generation can reach backward and touch it.</p><p>This offers a plausible, research-backed account of why temperature’s effect on hallucination is inconsistent across models rather than uniform. Training bakes certain “favorite” tracks into a model’s memory. Where a model has settled into a confident mistake or an endless repetition loop, forcing it to choose the single most likely token at every step locks it into that trap — whereas adding a small amount of randomness provides the leeway to leave the flawed path. Where the top-ranked choice is already correct, however, that same randomness simply adds noise to an accurate output. While researchers do observe this dynamic unfold in many controlled experiments, it remains an explanatory hypothesis rather than a proven universal rule: humanity currently lacks the computational resources and mathematical tools to formally verify it across hundreds of billions of parameters for every conceivable prompt.</p><h4>Why “Enemy of Validity” and “Destroys Validity” Are Imprecise</h4><p>Given the mixed and model-dependent evidence above, it’s tempting to conclude that randomness is simply the enemy of valid reasoning and should be minimized wherever possible. This is close to right in spirit but imprecise in a way worth correcting.</p><p>Recall that validity is not a matter of degree — a specific chain of reasoning either holds together as a legitimate sequence of steps or it doesn’t. It helps to separate two different things that are easy to blur together here: the process that produces a piece of reasoning, and the reasoning it actually produces. The process is genuinely continuous and probabilistic — at every step, a model is weighing many possible next words against each other, the way a weighted coin weighs its two outcomes before it’s flipped. But a coin flip, however continuous the underlying physics, still lands on exactly one side. It doesn’t land 73 percent on heads. The same separation holds here: whatever mix of probabilities produced a given chain of reasoning, the chain that actually comes out the other end is one specific, completed sequence of steps, and that sequence either holds together or it doesn’t.</p><p>The gradedness lives in the process that generates the chain, not in the chain itself once it exists. Language-model outputs are not formal logical proofs in the strict technical sense, but the same underlying intuition applies: randomness cannot “perturb” a chain of reasoning the way static perturbs a signal, gradually degrading something that was otherwise fine. What randomness actually does is something upstream of that: it influences which specific chain of reasoning gets produced in the first place, out of many that could have been produced instead. Randomness doesn’t reach into an already-good chain and damage it, and it doesn’t reach into an already-bad chain and repair it. It is better understood as a selector among possible paths than as a corrupting force acting on a single fixed one — and the evidence above, showing that its effect varies by model and task rather than acting as a uniform corruptive influence, is consistent with this framing. This distinction changes what the right fix looks like.</p><p>If randomness straightforwardly destroyed reliability, the fix would be simple: remove it, and reliability would follow. The large-scale evidence shows this isn’t so — temperature zero is often but not always best, and can be measurably worse on a different failure dimension entirely. This suggests randomness was never the primary lever governing whether a model’s output is trustworthy. It’s one input among several, with effects that depend heavily on what’s already baked into the specific model being queried.</p><h4>What a Real Fix Has to Do</h4><p>If neither adding randomness nor removing it reliably addresses category-two failures — a reasoning process drifting away from what a model’s own knowledge actually supports — then a more targeted intervention is needed: not a dial that changes how much unpredictability is present, but a mechanism that actively favors reasoning paths consistent with a model’s genuine internal knowledge, regardless of what temperature setting is in use.</p><p>Concretely, this means treating a model’s own body of reliable, well-established knowledge as having a consistent internal shape — a pattern in how that knowledge legitimately builds on itself, step by step — and actively comparing a reasoning process against that pattern as it unfolds, rather than only checking the final output. Where a step is the kind of step the system’s genuine knowledge would actually support, it’s allowed to continue. Where a step merely resembles a reasonable continuation without being backed by anything real, it can be corrected back toward the pattern the system’s own knowledge exhibits. This approach is aligned with current research directions such as process-level reward models and consistency-based verification, both of which aim to prefer well-grounded reasoning trajectories over merely fluent-sounding ones. It is not a mechanism for eliminating randomness, and it is not a mechanism for reintroducing it — it operates on a different axis entirely.</p><p>Why This Is Valuable Even Though It Cannot Guarantee Truth</p><p>Here the honest limit has to be stated plainly, because overstating it would repeat the exact mistake this essay opened with. Making a reasoning process more faithful to a system’s own knowledge does nothing to guarantee that knowledge is correct. If the underlying knowledge contains a falsehood — category one — then a perfectly faithful reasoning process built on top of it will arrive at that falsehood more reliably, not less. Nothing about tightening the connection between reasoning and knowledge can fix a defect that exists within the knowledge itself. That gap can only be closed by checking against the world outside the system: retrieved evidence, expert review, experimental confirmation, or, in domains where it exists, a computable verifier.</p><p>But soundness is logically impossible without validity as a precondition. There is no route to a trustworthy conclusion that skips over having a legitimate, faithful connection between what a system knows and what it concludes. A method that directly supplies a preference for well-grounded reasoning paths — even while leaving the separate, harder problem of correct-knowledge-in-the-first-place untouched — closes a real and often neglected gap. It helps ensure that when a system is wrong, it is wrong for the more forgivable reason that its knowledge was incomplete or mistaken, rather than the additional, avoidable reason that its process wandered somewhere its own knowledge never actually supported.</p><p>This Is Not Just an AI Problem</p><p>None of this is unique to machines. A person reasoning with flawless internal logic, from a mistaken understanding of how something works, will reach flawless-sounding, confidently wrong conclusions for the exact same structural reason a language model does. Human decision-making operates under real limits — incomplete information, limited time, limited capacity to consider every alternative — meaning people routinely build coherent reasoning on top of an incomplete picture of the world. The gap between a reasoning process holding together internally and that reasoning corresponding to how the world actually is has never been a defect specific to artificial systems. It is one of the oldest and most general facts about reasoning itself.</p><p>What This Should Change</p><p>None of this argues for lowering expectations of AI systems, and none of it argues that temperature is an unimportant setting — it measurably affects both accuracy and coherence, unevenly across models and tasks, and deserves careful, model-specific tuning rather than a one-size-fits-all default. What the evidence does argue against is treating temperature as the primary lever for trustworthiness. A method that makes a system’s reasoning process faithful to its own knowledge is not a cure for hallucination in any final sense, because the gap between belief and truth is not the kind of gap that internal consistency alone can close. But it addresses a specific, real, and often neglected failure: the difference between a system being wrong because it didn’t know better, and a system being wrong because nothing in its process was actively favoring the reasoning path its own knowledge genuinely supported. The first failure is a limitation of knowledge, addressed slowly through better training, better data, and better verification against the world. The second is a limitation of process, addressable directly, now, by building mechanisms that supply the preference for well-grounded reasoning that temperature, in either direction, was never reliably able to provide on its own.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9e034f382c75" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The Thermodynamic Wall: Why Autonomous AI Needs Humans at Both Ends of the Loop]]></title>
            <description><![CDATA[<div class="medium-feed-item"><p class="medium-feed-snippet">The current prevailing narrative in artificial intelligence rests on a simple, seductive premise: if we can just collect enough human text&#x2026;</p><p class="medium-feed-link"><a href="https://basalat-raja.medium.com/the-thermodynamic-wall-why-autonomous-ai-needs-humans-at-both-ends-of-the-loop-20bc91071e9c?source=rss-fdde71e16928------2">Continue reading on Medium »</a></p></div>]]></description>
            <link>https://basalat-raja.medium.com/the-thermodynamic-wall-why-autonomous-ai-needs-humans-at-both-ends-of-the-loop-20bc91071e9c?source=rss-fdde71e16928------2</link>
            <guid isPermaLink="false">https://medium.com/p/20bc91071e9c</guid>
            <category><![CDATA[self-improvement]]></category>
            <category><![CDATA[ai-agent]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[llm-agent]]></category>
            <dc:creator><![CDATA[Basalat Raja]]></dc:creator>
            <pubDate>Sat, 19 Sep 2026 16:17:07 GMT</pubDate>
            <atom:updated>2026-09-19T16:21:41.510Z</atom:updated>
        </item>
        <item>
            <title><![CDATA[Beyond Tinkering: Why Language Design Is the True North of Software Engineering]]></title>
            <link>https://basalat-raja.medium.com/beyond-tinkering-why-language-design-is-the-true-north-of-software-engineering-caf29539c2e1?source=rss-fdde71e16928------2</link>
            <guid isPermaLink="false">https://medium.com/p/caf29539c2e1</guid>
            <dc:creator><![CDATA[Basalat Raja]]></dc:creator>
            <pubDate>Fri, 07 Aug 2026 22:06:01 GMT</pubDate>
            <atom:updated>2026-08-08T22:02:11.101Z</atom:updated>
            <content:encoded><![CDATA[<p>For as long as humanity has written software, we have dealt with a recurring, gnawing truth: almost every project is ill-defined and riddled with ambiguity.</p><p>Every engineer has lived the same story. Requirements arrive half-formed. You build something reasonable. Three weeks later the ground shifts, and what you built no longer fits. You patch it. It shifts again. You patch again. This isn’t a failure of skill — it’s the default physics of the job. The question worth asking isn’t how to avoid this cycle, but what separates the people who get crushed by it from the people who somehow stay ahead of it.</p><p>There have been many proposals for how engineering is structured — architecture roles, seniority ladders, specialization tracks. But look past the org chart, and a different distinction emerges, one that has nothing to do with title. It’s the difference between two modes of working, and almost everyone in this industry spends nearly all their time in only one of them without realizing it.</p><p>At the top of this distinction sits the language designer and implementer. Everyone else, most of the time, is tinkering — by definition, because they are forced to continually tinker as specifications become clearer, or as they change again just when you thought they’d settled.</p><h3>The Tyranny of the Moving Target</h3><p>Most software engineering is a reactive discipline. We are given a set of requirements — often incomplete or shifting — and we attempt to map them onto the syntax and semantics of an existing programming language. When the requirements change, we patch, refactor, and duct-tape our way to a solution. We are constantly navigating a landscape where the rules were set by someone else long before we arrived.</p><p>This is what it means to be a tinkerer. You are operating within a sandbox, limited by the abstractions provided to you. When the problem space shifts, you work within the constraints of your chosen system, adding layers of complexity to bridge the gap between what you have and what you need. And to be fair to it — tinkering is fun. It’s the part of the craft that feels like play: fast feedback, small victories, the satisfaction of a fix that finally clicks into place. There’s a reason every engineer loves it. It’s also, quietly, how you learn. Every piece of friction you hit while tinkering is telling you something real about where your current abstractions break.</p><p>That’s an easy thing to miss when tinkering gets cast as the lesser activity. It isn’t lesser. It’s a genuine learning phase. The trouble starts somewhere else entirely.</p><h3>The Other Option</h3><p>Instead of trying to solve the specific problem you think you’re aiming at, you can design a language that solves the <em>class</em> of problems you think you might face. The specific instance in front of you is only a special case of that class. You are implementing two things: the language, and a short script written in that language. When the nature of the problem changes, you change the script — and maybe extend the language’s reach a little. Most of the effectiveness comes from that second step: you can now program directly to the <em>delta</em> of the requirements change, instead of re-solving the whole problem from scratch.</p><p>We already do this instinctively, more often than we credit ourselves for. Every object hierarchy, every custom domain model, every plugin interface is a bespoke programming language for the project. The syntax is the structural grammar — class definitions, method signatures, compositional arrangements. The semantics are the runtime rules that give those structures meaning. Define both before writing application logic, and you craft a vocabulary tailored to the exact contours of the domain — one where every future problem in that domain gets cheaper to solve than the last.</p><p>So here’s the reframe worth sitting with, given the label from a few paragraphs back: you are not either a tinkerer or a language designer. You are both, constantly, often within the same afternoon. Every competent engineer cycles through both modes. The real question was never which one you fundamentally are. It’s whether, in any given moment, you actually know which one you’re doing.</p><p>Writing a parser for a script in the language you have already designed is merely formalization of the work you have already done. Changing a script is much easier than re-wiring a bunch of class hierarchies. There’s plenty of well-known DSL management frameworks and techniques that make it straightforward.</p><h3>The Failure Mode Is Not Knowing</h3><p>This is the insight that matters more than any hierarchy: the danger isn’t tinkering. The danger is tinkering while believing you’re designing, or designing while you actually just needed to tinker — and not noticing the difference until the cost has already compounded.</p><p>Tinker without knowing it, and you accumulate one-off patches that never get promoted into anything reusable — the same bug reappears in a new outfit every sprint, because nothing was learned at the level where the lesson should live. Design without knowing it — trying to build a grand abstraction before you’ve tinkered enough to understand what it actually needs to cover — and you get a language nobody can use, full of premature generality solving problems that don’t exist yet — and more importantly, might never.</p><p>The discipline, then, isn’t “stop tinkering, start designing.” It’s a two-step loop, done deliberately:</p><ul><li>Tinker to discover a shortcoming — a place where your current grammar can’t express what the problem needs.</li><li>Ask, honestly, whether that shortcoming is a <em>repeated shape</em> or a <em>singular one</em> — and only promote the repeated shapes into the language itself.</li></ul><p>Skip that second question, and you either bloat your abstractions with rules for bugs you’ve only seen once, or you keep patching the same repeated failure by hand forever. Both are versions of not knowing which game you’re playing.</p><p>This distinction has teeth well beyond productivity. Security and performance are the clearest proof: most vulnerabilities and most slow systems aren’t singular events, they’re <em>classes</em> of error — buffer overreads, injection, unchecked units, N+1 queries, race conditions. You can tinker a fix for each occurrence as your monitoring catches it, or you can push the invariant into the type system, the grammar, the plugin interface — and make the entire class of problems structurally impossible. A language that forbids a mistake doesn’t need engineers to remember not to make it. That’s the difference between a security hole found once and patched, and one found once and closed forever for everyone who uses the language afterward.</p><h3>Syntax, Semantics, and the Discipline of Knowing</h3><p>It’s common to hear that software engineering is a broad discipline — architecture, project management, human psychology. All true, in the sense that these are the terrain you operate on. But underneath the meetings and the documentation, the actual leverage always comes down to syntax and semantics — whether you’re shaping the medium or merely consuming someone else’s choices within it.</p><p>The mistake is thinking this means you should always be shaping the medium. You shouldn’t. Tinker freely, learn hard, fail fast — right up until a shortcoming repeats itself. That’s the moment the professional move is to stop and ask whether it’s time to change the language instead of patching the instance again. Missing that moment — tinkering past it out of momentum, comfort, or the sheer pleasure of the craft — is the real failure this essay has been circling.</p><h3>Reclaiming the Craft</h3><p>None of this is a condemnation of tinkering. It’s a call to notice, in real time, which mode you’re in — and to be honest about when the friction you’re feeling has stopped being new information and started being the same shortcoming asking, for the third time, to be fixed at the root.</p><p>The next time you find yourself patching a requirement that doesn’t quite fit your code, don’t ask whether tinkering is beneath you. Ask something more useful: have I learned something here, or have I learned this already?</p><p>And let me be clear, because none of this is written from a pedestal: <strong>I am a tinkerer.</strong> I love tinkering. It’s the part of this work that still feels like play after all these years, and I have no interest in giving it up. But loving it is exactly why I’ve had to learn — repeatedly, and usually the hard way — when it’s teaching me something new and when it’s just keeping me comfortable while the same shortcoming quietly repeats itself.</p><p>Early in my career, like everyone else, I hit a fork in the road. I could be known as the person who owns those five problems and earns respect for being that specialist. Or I could keep growing. Tinkering is where every project begins, and where every good instinct is formed. But getting <em>lost</em> in tinkering — polishing clever fixes, chasing micro‑elegance, treating every patch as a personal performance — that’s where professionalism quietly slips away and the prima donna takes over.</p><p>The only thing that has ever gone wrong for me, every time, is not tinkering itself. It’s tinkering <em>past the point where I already had the lesson</em>, the clarity is already there, the realization has already hit — tinkering on, instead of stepping back and building the abstraction that would make the problem impossible next time.</p><p>Knowing what you are doing is easy. <br>Knowing what you want is the hard part. <br>That is what we get paid the big bucks for.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=caf29539c2e1" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The Wrong Kind of Smart]]></title>
            <link>https://basalat-raja.medium.com/the-wrong-kind-of-smart-1605e0614b06?source=rss-fdde71e16928------2</link>
            <guid isPermaLink="false">https://medium.com/p/1605e0614b06</guid>
            <dc:creator><![CDATA[Basalat Raja]]></dc:creator>
            <pubDate>Mon, 06 Jul 2026 19:56:17 GMT</pubDate>
            <atom:updated>2026-07-06T23:33:50.052Z</atom:updated>
            <content:encoded><![CDATA[<p><em>We have been building the most powerful scientific instrument in human history and arguing about whether it can write an original sonnet</em></p><p>There is a version of artificial intelligence that will change everything — not everything in the way that smartphones changed everything, by making existing things faster and cheaper, but everything in the way that the germ theory of disease changed everything, by revealing that a vast category of human suffering had a cause, and therefore a cure. That version of AI is not the one most people are talking about.</p><p>The public conversation about transformative AI is almost entirely about one question: can machines think like humans? Can they reason, converse, create, feel? The question is genuinely interesting. It is also, I will argue, almost entirely the wrong question if what you care about is the largest possible reduction in human suffering and the largest possible expansion of human capability over the next fifty years.</p><p>The right question is different, and stranger. It is this: can machines search spaces too large for humans to search, in domains where correctness is checkable?</p><p>That question has a clear answer. Yes. They already can.</p><p><strong>The Combinatorial Wall</strong></p><p>Consider the protein. A protein is a chain of amino acids — twenty types, arranged in sequences that typically run to hundreds or thousands of residues. The number of possible proteins of length 200 is greater than the number of atoms in the observable universe by an unimaginable margin. Evolution has explored this space for four billion years and found the proteins that run every living cell on earth. Human biochemists have explored it consciously for perhaps a century and found a few thousand more.</p><p>The proteins we have not yet found include enzymes that could fix carbon dioxide directly from the atmosphere, catalysts that could synthesize any medicine cheaply from sunlight and air, structural materials stronger than steel and lighter than plastic, and molecular machines that could seek out and destroy a cancer cell while leaving its neighbors untouched. We have not found these proteins because we cannot search fast enough. The space is too large, the experiments too slow, the human attention too limited.</p><p>This is not a problem of understanding. Biochemists understand proteins extraordinarily well. It is a problem of search — of moving through an enormous space of possibilities to find the rare points that work. And search, it turns out, is exactly what AI is good at.</p><p><strong>The Oracle and the Guesser</strong></p><p>Here is the architectural insight that I think will define the next era of applied science. In any domain where you can <em>check</em> an answer cheaply, the most important question is not how carefully you generate guesses but how quickly you generate them. The checking takes care of quality. The generating takes care of coverage.</p><p>This sounds obvious when stated plainly. It is, apparently, not obvious to the people deploying AI systems, because the dominant approach in AI development right now is to make the generator more careful — more aligned, more restrained, less likely to produce outputs that are wrong. This is the right approach when there is no checker. For open-ended conversation, for historical questions, for advice about complex human situations, there is no easy way to verify the output. In those domains, a careful generator is essential.</p><p>But proteins can be checked. A simulation can tell you, with reasonable accuracy, whether a protein sequence will fold into a stable structure. A virtual screen can tell you whether a molecule will bind its target. A compiler can tell you whether a program is syntactically correct. A proof assistant can tell you whether a mathematical argument is valid. In every one of these domains, the checker exists, and it is fast.</p><p>In these domains, the most valuable thing a generative AI system can do is not be careful. It is to be bold — to propose sequences and molecules and programs that no human would have thought to try, to range far from the familiar, to search the regions of possibility that conventional wisdom says are unpromising or incoherent. The checker will discard the proposals that don’t work. The ones that pass are inventions.</p><p>This is the deep structure of discovery: a generator that does not know what it is looking for, and an oracle that recognizes it when it arrives. Engineers — those who build a system from the ground up — do not need oracles. Searchers do. We have built a system and immediately begun debating whether it is an engineer, when the more important path is the one right in front of us.</p><p><strong>What We Are Actually Building</strong></p><p>I want to be specific, because vagueness is the enemy of this argument.</p><p>Right now, teams around the world are using protein language models — AI systems trained on the sequences of known proteins — to propose new sequences for enzymes that do not yet exist in nature. The AI generates candidates that are unfamiliar, far from anything in the training data, structurally unusual. Most fail. Some pass the computational filters. Of those, a fraction survive experimental validation. And among those survivors are molecules with functions that biology never found in four billion years of search — because evolution is not trying to fix atmospheric carbon dioxide using novel organometallic cofactors, or to synthesize a specific pharmaceutical compound from cheap feedstocks, or to catalyze a reaction that has never occurred in any living cell.</p><p>These are not refinements of existing drugs. They are not incremental improvements on known enzymes. They are genuinely new molecular machines, found by a process that requires no understanding of what it is finding, only the ability to propose and the ability to check.</p><p>The same logic applies to materials. To synthetic chemistry. To formal mathematics, where proof assistants can verify in seconds arguments that would take a human expert weeks to check. To software, where compilers and test suites provide instant verification of correctness claims. In every one of these domains, the bottleneck is not human understanding. It is search speed. And AI, even today’s AI — imperfect, prone to error, incapable of genuine reasoning by any rigorous definition — can search these spaces at speeds that make human expert intuition look like a candle next to a searchlight.</p><p><strong>The Misallocation of Wonder</strong></p><p>I do not want to be dismissive of language, reasoning, and the humanistic dimensions of AI. They matter. A system that can explain a complex diagnosis to a patient in plain language, or help a first-generation college student write a personal essay, or translate a legal document accurately for someone who cannot afford a lawyer — these are real goods. I am not arguing against them.</p><p>I am arguing against the implicit hierarchy that places conversational, human-like AI at the top of the significance ranking, and treats scientific, oracle-verifiable AI as a specialized application. The hierarchy is inverted. The specialized application is the one that changes everything.</p><p>The history of transformative technology suggests that the most important innovations are often the ones that are least visible to ordinary experience — the ones that operate at scales and speeds that humans cannot perceive directly. The Green Revolution (1943–1975) saved a billion lives, but nobody watched it happen. The development of nitrogen fixation at industrial scale — the Haber-Bosch process — was arguably the most important invention of the twentieth century, and most people have never heard of it. The internet is experienced as email and social media; its transformative power operates at the level of supply chains, financial systems, and scientific communication that most users never see.</p><p>AI-assisted protein design will be like this. You will not see the enzyme that scrubs carbon dioxide from industrial exhaust. You will not see the catalyst that makes a malaria cure cheap enough to distribute globally. You will not meet the molecule that dissolves the amyloid plaques in an Alzheimer’s patient’s brain before the symptoms begin. But if the trajectory of the last five years in structural biology and generative modeling continues, you will live in a world shaped by these things, and you will rarely see how it changed.</p><p><strong>The Harder Question</strong></p><p>There is an honest objection to everything I have said, and it deserves a direct answer.</p><p>The objection is that computational oracles are not reality. A protein that folds correctly in a simulation does not always fold correctly in a cell. A molecule that binds its target in a virtual screen does not always bind it in a body. The gap between computational prediction and biological truth is real, sometimes large, and the history of drug development is full of compounds that passed every computational filter and failed in clinical trials.</p><p>This is true. The oracle is imperfect. Coverage fraction is not one hundred percent.</p><p>But the relevant question is not whether the oracle is perfect. It is whether the oracle is good enough, improving fast enough, that the searchable space of real solutions is expanding faster than the problems require. And here the evidence is striking. Protein structure prediction, which was an unsolved problem for fifty years, was cracked in 2020 by a system that made structure from sequence with near-experimental accuracy. The tools that were research curiosities three years ago are now running production workflows at pharmaceutical companies, national laboratories, and university basements. The oracle is getting better faster than the problems are getting harder.</p><p>We are not at the finish line. We are at the moment when the finish line becomes visible for the first time.</p><p><strong>A Different Frame for Intelligence</strong></p><p>The deepest shift I am asking for is not about applications or architectures. It is about what we mean when we say a system is intelligent.</p><p>The dominant frame treats intelligence as the ability to reason and converse — to do the things humans do, but faster, cheaper, at scale. Under this frame, the question of AI is whether machines can be like us.</p><p>There is another frame. Intelligence is the ability to find what works in spaces too large to search by hand. Under this frame, the question of AI is whether machines can go where we cannot — into the combinatorial wilderness of protein sequence space, molecular configuration space, formal proof space — and return with things that function. Not things that sound right. Things that <em>are</em> right, by the only test that matters in science and engineering: does it work?</p><p>Under the first frame, the measure of an AI system is how human it is. Under the second, it is how useful it is — in the deep sense of usefulness that includes vaccines, and fertilizer, and structural steel, and all the other unglamorous things that the average human life depends on absolutely.</p><p>I think the second frame is closer to the truth. I think the most important cognitive act is not the one that looks most like thinking. It is the one that finds the molecule.</p><p><em>The author is developing </em>generalizable <em>computational frameworks with a focus on protein and bio-industrial design.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1605e0614b06" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[We Do This Not Because It Is Hard, But Because We Thought It Would Be Easy]]></title>
            <link>https://basalat-raja.medium.com/we-do-this-not-because-it-is-hard-but-because-we-thought-it-would-be-easy-ffcb2c09fcd5?source=rss-fdde71e16928------2</link>
            <guid isPermaLink="false">https://medium.com/p/ffcb2c09fcd5</guid>
            <category><![CDATA[software-engineering]]></category>
            <category><![CDATA[ai-misinformation]]></category>
            <category><![CDATA[code-quality]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[ai-hype-vs-reality]]></category>
            <dc:creator><![CDATA[Basalat Raja]]></dc:creator>
            <pubDate>Mon, 29 Jun 2026 17:03:15 GMT</pubDate>
            <atom:updated>2026-08-07T18:42:29.288Z</atom:updated>
            <content:encoded><![CDATA[<p>The industry is gripped by vivid hysteria: engineers drowning in “code slop,” craftsmanship collapsing under a tide of autogenerated fragments, teams suffocating beneath mountains of incoherent AI output. The imagery is dramatic — swamps, sludge, drowning — and it reflects a genuine unease. But the crisis described so urgently is not the crisis we are actually living through.</p><p>Large Language Models are not engineers. They are not replacements for engineers. They are not even failed engineers. They are autocomplete systems operating at scale, producing massive amounts of code without ever entering the domain of thought. Their outputs can be brittle or misaligned, but these failures are not betrayals of intelligence. They are simply the natural limits of a tool that generates without understanding.</p><p>For those who already know what they are doing, this is not a threat. It is a convenience. A draft. A starting point. The “I know it when I see it” phenomenon applies to code as readily as to language. An experienced engineer can look at a generated snippet, recognize its shape, and adjust it toward the intended form. In this sense, LLMs provide raw material, not finished work.</p><p>I have used yacc for decades, yet I have never written a yacc file from scratch. Every time I needed to build a programming language (and that is the case for each and every serious project over the level of “toy”, please see footnote [1]), I have always found a yacc file and modified it for my needs. The thinking was mine; the boilerplate was borrowed. LLMs simply automate the borrowing. They accelerate typing, not judgment. (yacc stands for Yet Another Compiler Compiler; it helps you create programming languages.)</p><p>The trouble begins when people mistake this acceleration for engineering itself. And it is not only engineers who feel overwhelmed. They carry the expectations of entire organizations — product teams, managers, executives, and enthusiastic newcomers — many of whom sincerely believed that software engineers could be replaced by LLMs. They imagined that the hard part of engineering was the typing, and that the thinking could be outsourced to a machine that they thought can think it through. Unfortunately, no, it cannot. It can’t think. They embarked on projects whose complexity they did not understand, trusting a statistical generator to supply the missing expertise.</p><p>A simple analogy makes the error clear.</p><p>Suppose I wanted to draw a picture of a dodo. No model has ever seen a real dodo; no training corpus contains its living form. If I asked an AI to “draw a dodo,” the result would not satisfy me nor any other human. But I could ask it to draw a bird. Or perhaps an ostrich, or a rhea. From there, I could modify the image until it resembled the human idea of a dodo. The tool would give me a starting point, and I would supply the thinking.</p><p>The mistake is assuming that “draw a dodo” can be executed perfectly, every single time, without human involvement. Yet that is precisely what happened. An entire industry decided that “build a system,” “write a service,” or “design a feature” could be executed with the same immediacy. They believed that the model could produce the final form, born like Athena, fully formed from Zeus’ forehead. They assumed that the thinking could be automated.</p><p>This assumption has consequences. A large consulting company recently secured a major AI contract and interviewed me as a lead engineer. They turned me down. Even if they had believed I was qualified, they might still have turned me down for my unwillingness to promise the impossible. And I was satisfied with their decision. I did not want to be responsible for delivering on commitments that no tool could fulfill. Instead, I have spent the last four months building a system that searches for cures for different types of cancer; many it has found, others, it is still seeking. Many people would call it AI. It is not. It is a database, plain and simple — carefully structured, rigorously reasoned, and grounded in the realities of biology rather than the fantasies of automation. Their choice freed me to do work grounded in reality rather than hype and overenthusiastic hope, and I am grateful for it.</p><p>The engineers who feel overwhelmed are not failing. They are carrying the weight of misaligned expectations — both their own and those of the many people who rely on them. Their frustration is real, but its cause is not the tool; it is the misunderstanding of the tool.</p><p>This is not a story of engineers being replaced or erased. It is a story of engineers being needed more than ever. The presence of autocomplete does not diminish the value of craftsmanship; it clarifies it. When a tool produces vast quantities of code-shaped text, the ability to discern structure from noise becomes essential. The engineer’s role shifts from typing to thinking, from producing to evaluating, from drafting to designing. This is not erosion. It is elevation.</p><p>The narrative of existential collapse imagines that the craft is dying. But crafts do not die when tools change. They evolve. The introduction of power tools did not eliminate carpentry; it transformed it. The arrival of calculators did not end mathematics; it expanded it. In each case, the tool amplified the expert rather than replacing them. The same is true here. Despite the same claims that have been ongoing since at least 1985, the advent of computers is not going to make computer scientists obsolete. It is going to make them even more necessary.</p><p>LLMs are autocomplete. Not junior engineers. Not thinking machines. Autocomplete. Their value lies in scale, not in insight. They generate linguistic ore, and it is the human mind that refines it into meaning. The apparent chaos arises not from the models themselves, but from the belief that they are something other than what they are.</p><p>Once this distinction is understood, the landscape looks different. The craft is not dead. It is needed. The engineer is not obsolete. They are indispensable. And the path forward is not despair, but clarity: to recognize what LLMs are, what they are not, and how they can be used without burdening those who understand the work.</p><p>Thinking is hard. There are no shortcuts. We have to do it ourselves.</p><p>[1]: A programming language is the natural solution to ambiguity. When you are not sure what problem you are solving, you design a programming language that you can use to express solutions for the class of problems that cover the ambiguity. And in a the time I have been writing software, I have never come across a software engineering project that was not ill-defined and ambiguous. Knowing what you are doing is easy. Knowing what you want is the hard part. That’s what we get paid the big bucks for.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ffcb2c09fcd5" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[LLMs are not intelligent. They are not even stupid.]]></title>
            <link>https://basalat-raja.medium.com/llms-are-not-intelligent-they-are-not-even-stupid-a1facc413182?source=rss-fdde71e16928------2</link>
            <guid isPermaLink="false">https://medium.com/p/a1facc413182</guid>
            <category><![CDATA[ai-agent]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[software-engineering]]></category>
            <dc:creator><![CDATA[Basalat Raja]]></dc:creator>
            <pubDate>Fri, 19 Jun 2026 18:26:00 GMT</pubDate>
            <atom:updated>2026-06-19T18:26:00.766Z</atom:updated>
            <content:encoded><![CDATA[<p>Large Language Models are not intelligent. They are not even stupid. This paradox captures the difficulty of applying human categories to machines that generate language without thought. Intelligence and stupidity both imply a mind, a capacity for judgment, and an orientation toward truth or error. LLMs have none of these. They are statistical engines trained to predict the next word in a sequence, drawing on vast corpora of text. Their outputs can be dazzlingly fluent, but fluency is not comprehension.</p><p>To call them intelligent is to anthropomorphize, projecting onto them the qualities of reasoning and understanding that belong to true cognition; forever beyond their reach. They do not know what they are saying, nor do they grasp meaning. They cannot distinguish truth from falsehood or relevance from irrelevance, except insofar as those distinctions are encoded in statistical patterns. They simply generate text according to probabilities. When their outputs are wrong or incoherent, it is not because they have erred in thought, but because the sequence does not align with reality.</p><p>What they are, instead, are mirrors of language. They reflect back the structures, idioms, and arguments embedded in the texts they have consumed. They can simulate reasoning, but the simulation is not cognition. hey can mimic dialogue, but without intention. In this sense, they are neither intelligent nor stupid, but something orthogonal to both: machines that operate in the domain of language without entering the domain of thought.</p><p>This distinction matters because it shapes how we use them. If we treat them as intelligent, we risk overestimating their capacities and entrusting them with decisions they cannot make. If we treat them as useless, we risk underestimating their utility. The truth lies elsewhere: they are powerful instruments for generating text, but they require human oversight to ensure meaning and accuracy.</p><p>Thus, the claim that LLMs are not intelligent and not even stupid is not a dismissal but a clarification. It reminds us that these models belong to a different category altogether. Their value lies in simulation, and beyond that they reach a blank wall. To recognize this is to use them wisely.</p><p>Humans, however, are remarkably adept at finding patterns in chaos, and we call this skill “intelligence.” By contrast, LLMs are random text generators. I can write a crude text generator in minutes, but the billion‑dollar value of LLMs lies in a single quality: their randomness is constrained. They produce text that, while not truly meaningful, often stays loosely on topic.</p><p>This output is still hallucinatory gibberish in the strict sense, but here human pattern‑finding comes into play. Experts can recognize genuine advances instantly. They can pull a jewel out of the rubble.</p><p>That is why we hear so many exclamations of “Wow! This AI is amazing!” The breakthrough was not created by the LLM itself. The model churned out a mass of text, most of it derivative, echoing what is already known. But every so often, by statistical chance, it produces something genuinely useful. The LLM did not intend this advance — it simply rolled the dice. The true act of intelligence was the human expert recognizing the value hidden in the noise.</p><p>Predictions of superintelligence and extraordinary progress from LLMs will never come true, because their infrequent generation of value cannot be generalized. They cannot evalaute their own output, they are just generating it, nothing else. LLMs cannot serve as autonomous assistants, for at their core they remain random generators of text.</p><p>They can, however, be useful as tools of amplification. What they provide is not intelligence but scale: the ability to produce vast amounts of language hovering near a topic, from which human experts can extract what is valuable. In this sense, they function less like assistants and more like raw material suppliers. They generate linguistic ore, and it is the human mind that refines it into meaning. The apparent amplification is simply an artifact of the fact that humans can recognize valuable patterns far more quickly than they can generate them.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a1facc413182" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>