<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Felipe Imbelissieri-Casadei on Medium]]></title>
        <description><![CDATA[Stories by Felipe Imbelissieri-Casadei on Medium]]></description>
        <link>https://medium.com/@felipe_cas?source=rss-9168a35e9cf7------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*5pX4s1ZmD018UtOtbm7Mvw.jpeg</url>
            <title>Stories by Felipe Imbelissieri-Casadei on Medium</title>
            <link>https://medium.com/@felipe_cas?source=rss-9168a35e9cf7------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Thu, 19 Nov 2026 08:57:52 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@felipe_cas/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[The Illusions of Speed: AI and Cognitive Waste]]></title>
            <link>https://medium.com/@felipe_cas/the-illusions-of-speed-ai-and-cognitive-waste-e9d802cd354d?source=rss-9168a35e9cf7------2</link>
            <guid isPermaLink="false">https://medium.com/p/e9d802cd354d</guid>
            <category><![CDATA[organizational-change]]></category>
            <category><![CDATA[ai]]></category>
            <dc:creator><![CDATA[Felipe Imbelissieri-Casadei]]></dc:creator>
            <pubDate>Fri, 27 Feb 2026 14:57:46 GMT</pubDate>
            <atom:updated>2026-02-27T14:57:46.584Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*hBUAqKYFbOhtnBRbCuK4jw.png" /></figure><h4>When AI makes drafting effortless, organizations don’t produce fewer documents: they produce more. More plans, more proposals, more code, more meetings to align on all of it. Speed increases. Decisions don’t. This essay examines why faster output so reliably creates slower convergence, and what it would take to close that gap.</h4><p>Speed has become a moral category in knowledge work. Fast replies signal competence; long pauses signal weakness. Shipping is praised as a virtue independent of what is shipped. In that climate, AI arrives less as a tool than as an accelerant: it collapses time-to-first-output and makes immediacy feel like the natural tempo of thinking. Yet technological revolutions rarely reshape work by making it faster in any simple sense. Their durable effects come from changing representations, redistributing cognition across people and artifacts, and reducing what I will call cognitive waste — the overhead that accumulates through rework, miscoordination, verification, and attention fragmentation. By speed I mean the felt compression of time-to-response, time-to-produce, and time-to-iterate. By cognitive waste I mean attention and reasoning spent on avoidable overhead: context reconstruction, coordination churn, verification, rework, trust repair, and task switching. And by the illusion of speed I mean local acceleration that delays convergence on correct, durable decisions because it increases downstream cognitive waste. Speed is often a byproduct of reconfiguration. When speed becomes the goal, it tends to manufacture the very waste it claims to remove. Speed feels like progress when the unit of account is outputs produced, not uncertainty reduced.</p><p>Consider a week in a product organization trying to ship. The company is tech-centered; AI is already a strategic priority; many engineers can explain how modern models are trained at a high level. And yet, from individual contributors to managers to executives, the same confusion recurs in different dialects: What, exactly, is <em>good</em> AI use? What counts as real leverage rather than novelty? What new risks have quietly entered the system? The familiar answers — more experimentation, tighter cadence, more visibility — arrive quickly, now amplified by tools that can generate plausible plans, drafts, and code in minutes.</p><p>On Monday, a document appears that looks like alignment. It reads crisply, has a taxonomy of use cases, and proposes a roadmap. On Tuesday, a second document appears to improve the first: more scope, more examples, more confidence. By Wednesday, the organization has produced not a decision but a stack of artifacts: executive summary, FAQ, risk register, training plan, evaluation plan, rollout plan. The volume is impressive. The problem is that the artifacts are not the decision: they are prompts for more coordination.</p><p>The week fills with meetings that should have been an email — except now the email has been multiplied into five AI-assisted variants, each ready to be forwarded to a different audience. A “quick sync” is scheduled to resolve a small ambiguity. That sync requires a pre-meeting to avoid surprises, which requires another pre-meeting to agree on what counts as a surprise. The meeting-before-the-meeting becomes standard operating procedure, not because people enjoy it, but because rapid output increases the number of plausible paths and thus the number of ways to misalign. A proposal that once took effort to write now takes effort to refuse. The organization learns a new reflex: when drafting is cheap, changing the ask feels cheap too. Scope expands by accretion — “Since we can generate the first pass instantly, let’s also add…”</p><p>Meanwhile, the software system experiences its own version of acceleration. Pull requests arrive faster than reviewers can metabolize them. Code review becomes a <em>verification tax</em>: not a simple bottleneck, but a charge levied on every new unit of plausible output. When the surface area of change expands, reviewers spend more time reconstructing context, checking assumptions, and tracing consequences — work that is cognitively expensive precisely because it resists automation.</p><p>The anchoring premise is old: in an information-rich world, attention becomes the scarce resource, and information consumes attention (Simon, 1971). What changes in the AI era is not the scarcity itself, but the rate at which organizations can turn attention into outputs that demand more attention.</p><h3>Illusion 1: Speed as a goal</h3><p>It is tempting to narrate technological progress as a story of acceleration: what once took days now takes minutes; what once required specialists now requires a prompt. But this narrative confuses the metric with the mechanism. The most consequential technological shifts — writing, print, industrial measurement, computing — did not primarily make people faster. They changed how thinking was represented, shared, checked, and stabilized. They reduced cognitive waste by externalizing memory, standardizing procedures, and enabling coordination at scale. Speed followed as a consequence, never as a goal itself.</p><p>The cognitive sciences offer a vocabulary for this reconfiguration. In distributed cognition, intelligence is not located solely inside individual minds; it is a coupled system of people, artifacts, representations, and routines (Hutchins, 1995). In the extended mind thesis, external artifacts can be constitutive parts of cognition — parts of the thinking process rather than mere aids (Clark &amp; Chalmers, 1998). Cognitive artifacts, in this view, are not neutral channels; they reshape what is easy or difficult to think, and competence depends on learned practices for using them (Norman, 1993). Cognitive offloading describes a related tendency: under demand, people externalize thought into artifacts, gaining relief at the cost of new dependencies and new error modes (Risko &amp; Gilbert, 2016).</p><p>Take writing as an example: writing is not merely a faster way to remember; it is a different cognitive ecology altogether. Oral memory is powerful but fragile in specific ways: it relies on rehearsal, social reinforcement, and the interpretive bandwidth of the group. Writing stabilizes claims across time and distance. It makes sentences re-readable; it makes revision possible; it makes contradiction visible. These affordances themselves are not speed. They are cognitive waste reduction mechanisms. They reduce the need for repeated context reconstruction. They allow coordination with people who are not present. They enable accountability: what was said can be inspected later.</p><p>Later, the printing press scales these effects and, in doing so, changes institutions. When texts can be reproduced reliably, knowledge becomes less dependent on local guardianship and more dependent on shared reference points. The speed gains — faster dissemination, faster learning — arrive because the cognitive waste of transmission decreases. Fewer distortions accumulate per handoff. Less effort is spent rebuilding the same argument from scratch. More effort can be spent refining and checking.</p><p>A simple checklist is a later, more concrete example of the same logic. A checklist is often framed as a device for efficiency, but its core function is error reduction in complex, high-stakes systems. It externalizes a sequence of critical steps so that memory and attention are not forced to carry the entire burden. It makes coordination explicit: who confirms what, when, and with which signals. It prevents rework, blame spirals, and trust repair — the hidden costs of omission and ambiguity. When checklists work, they do not make a team “faster” so much as more reliable; speed follows because fewer failures require recovery (Gawande, 2009).</p><p>This is a general pattern: tools that reconfigure cognition reduce waste by altering representations and coordination. They change the unit of work. They make some forms of verification cheaper and some forms of error less likely. They stabilize meaning long enough for groups to build on it.</p><p>AI fits into this lineage more plausibly as a cognitive artifact than as a speed button. It changes representation: prose becomes easier to produce, code becomes easier to scaffold, summaries become easier to generate. But representation alone is not progress. The artifact must be instrumented at the correct level of abstraction— embedded in routines, norms, and institutions that make outputs accountable and coordination tractable. Without that instrumentation, the product organization’s Monday memo, its Wednesday stack, its Friday backlog remain exactly what they were: plausible artifacts that substitute for decisions rather than producing them.</p><h3>Illusion 2: Faster iteration means faster convergence</h3><p>The logic that generated the Monday memo and the five-variant email also justifies itself. Iterate quickly, learn quickly, converge quickly. This is sometimes true in environments where feedback is reliable, the cost of error is low, and iteration reduces uncertainty rather than multiplying it. But in many knowledge-work settings, faster local steps increase uncertainty by increasing volume, fragmentation, and downstream checking. Both realities can be true at the same time: the system can <strong><em>both </em></strong>move faster and still converge more slowly.</p><p>Several well-established constraints sit beneath this paradox. The speed–accuracy tradeoff shows that pushing for speed predictably shifts performance: decisions become less accurate, less reliable, or more variable, depending on the task and the method (Heitz, 2014). Task switching research highlights a complementary cost: rapid context switching carries overhead — time and cognitive effort spent reorienting, recovering goals, and managing interference (Monsell, 2003). At the organizational level, information overload describes what happens when the volume of available information exceeds processing capacity: coordination becomes noisier, attention more fragmented, and decision quality can degrade (Eppler &amp; Mengis, 2004). Cognitive load theory provides a mechanism: extraneous load — effort spent on irrelevant or avoidable demands — crowds out the germane load required for learning and problem solving (Sweller, 1988).</p><p>AI compresses time-to-first-output. It can generate a plausible draft, a design sketch, a test plan, or a code scaffold almost immediately. That compression is real. The illusion arises when organizations treat the collapse of the first step as a collapse of the whole path from uncertainty to decision.</p><p>To see the mechanism, map a modern loop: 1) prompt, 2) draft, 3) share, 4) meeting, 5) revision, 6) approval. AI accelerates the draft. But it also accelerates sharing. When a draft is cheap, it is easy to circulate “just to get feedback.” When circulation is easy, more stakeholders participate. When more stakeholders participate, alignment becomes harder, and the meeting-before-the-meeting becomes rational. The organization starts to coordinate earlier and more often because it senses, correctly, that rapid output increases the space of possible misunderstandings. This is not a failure of process; it is the organization’s correct response to a change in representational economics.</p><p>At the same time, faster drafting changes how people interpret partial artifacts. A draft that appears quickly can be read as a near-final claim, even when it is merely a prompt-shaped sketch. This creates a subtle shift in epistemic norms: the presence of fluent text becomes a cue of readiness. Under time pressure, teams lean on such cues because they reduce immediate uncertainty — “We have something.” But the reduced uncertainty is local and psychological; it is not necessarily epistemic. It does not guarantee that the artifact has been checked, grounded, or integrated with system constraints.</p><p>The verification tax becomes most concrete in software. AI-assisted coding can increase throughput in the short run by reducing the effort required to produce diffs. Yet the production of diffs is not the unit of progress; stable behavior is. Review is where meaning is reconstructed: what problem is being solved, what tradeoffs are being made, what constraints are being violated, what risks are being introduced. When PR volume rises, reviewers must choose between two bad options: skim more, or review less. Skimming increases the probability of errors and regressions; reviewing less creates queues and pressure. Either way, the organization experiences acceleration as stress.</p><p>This is where a dynamic I will call technical knowledge as limitation becomes visible. It is reasonable for engineers with AI literacy to feel confident: they can judge model output, they can reason about failure modes, they can spot code smells. But confidence under schedule pressure can become a pathway to overreliance. The tool produces competent-looking changes quickly; the engineer believes they can validate quickly; the organization rewards speed. The result is not necessarily catastrophic failure; it is a slow drift in epistemic hygiene. Tests are generated but not trusted. Comments are added but not read. Review becomes a ritual of scanning rather than an act of understanding. Technical knowledge, which ought to support scrutiny, becomes the justification for shortening it. Over time, the system accumulates debt that does not look like debt, because it is packaged as productivity.</p><p>The same pattern applies outside engineering. A strategy memo can be drafted instantly, but “instantly drafted” is not “institutionally actionable.” The memo must be interpreted, argued with, challenged, and integrated with constraints. That work is not simply slower because humans are slow; it is slower because it requires shared context and durable commitment. When everything is revisable at low cost, commitment becomes expensive. The organization keeps moving because movement signals competence, but the movement can become self-sustaining activity without resolution — the organizational equivalent of Tuesday’s document improving Monday’s without either becoming a decision.</p><p>Coordination churn and the verification tax are therefore not pathologies that can be eliminated by better prompting. They are predictable responses to a change in representational economics. When the supply of plausible artifacts increases, the demand for attention, alignment, and checking increases too. Unless institutions and norms change, the system produces more outputs while depleting the scarce resource that turns outputs into decisions.</p><h3>Illusion 3: AI is “just faster” (hint: it’s a new abstraction layer)</h3><p>If AI were merely a faster way to do existing tasks, the organizational response would be straightforward: adopt the tool, measure time saved, redeploy labor. Recent developments and studies are showing that AI resists that simple interpretation. In fact, AI is better understood as a new abstraction layer — an additional representational medium that changes who can produce what kind of artifact, how quickly, and with what relationship to accountability. Abstraction layers do not only speed work; they create new literacies, new coordination problems, and new institutional demands.</p><p>This insight predates AI. Writing changes what can be thought because it changes what can be inspected; it makes claims re-readable and revisable, and it fundamentally reorganizes the relationship between language and memory (Olson, 1994). But it also requires new norms: citation, authorship, editorial standards, peer review, legal regimes. The printing press intensifies this by scaling distribution, which increases both the benefits and the harms. New institutions emerged to manage trust: publishers, libraries, scholarly societies, reputational systems. Each new representational medium created not only speed but a governance problem — the question of what counted as legitimate, accountable knowledge.</p><p>The sociology of speed offers a complementary dimension: the governance problem has a cultural face, not only an institutional one. Speed can become a norm that reshapes incentives, identity, and authority — a regime in which delay is pathologized even when delay is epistemically necessary (Wajcman &amp; Dodd, 2016). Hartmut Rosa’s theory of social acceleration diagnoses the paradox directly: more activity, more change, more options — paired with less time for consolidation and meaning (Rosa, 2013). These are not merely descriptions of pace; they describe what organizations lose when acceleration becomes the dominant logic. What they lose, concretely, is the settled understanding from which reliable judgment grows. The product organization’s week illustrates the phenomenon precisely: constant explanation, constant updates, constant reframing — sensemaking activity that substitutes for resolution rather than producing it (Weick, 1995). The fast part is everywhere. The convergence never arrives.</p><p>AI’s abstraction layer is distinctive because it produces artifacts that look like finished cognition. It can output a coherent plan, a plausible summary, a tidy argument, a working code scaffold. Yet the output is not inherently accountable. The artifact does not, by default, carry its provenance; it does not automatically encode the reasons it is correct; it does not guarantee that it reflects the local constraints of a particular system. This is not a moral critique of the tool; it is a structural feature of producing fluent representations at scale. The organization must therefore decide: what counts as acceptable evidence? What counts as sufficient verification? What are the norms for review, responsibility, and disclosure?</p><p>The answer cannot be just “go faster.” When speed is moralized, organizations confuse responsiveness with reliability. The immediate consequence is more coordination: people coordinate defensively because they do not trust that shared artifacts reflect shared understanding. The longer-term consequence is institutional drift: decisions become harder to attribute; errors become harder to diagnose; trust becomes harder to calibrate. At the organizational level, confidence in AI literacy justifies moving faster, which systematically shortens the processes that would catch what has been missed. In software, this looks like review inflation and rising operational complexity. In policy and strategy, it looks like documents multiplying faster than decisions. In cross-functional work, it looks like alignment rituals proliferating because meaning has become brittle.</p><p>This is where cognitive-artifact thinking is practical. A cognitive artifact does not <em>contain intelligence</em>; intelligence is a property of the coupled system — people plus artifacts plus learned practices (Norman, 1993; Hutchins, 1995). The same artifact can either reduce waste or multiply it, depending on how it is embedded. A checklist reduces waste when it is treated as a coordination device that makes commitments explicit; it becomes waste when it is treated as performative bureaucracy. AI will follow the same pattern. It will reduce cognitive waste when it is instrumented to stabilize meaning — through clear responsibility, explicit review norms, calibrated trust, and constraints on when “instant” is allowed to substitute for understanding. It will multiply waste when it is used to accelerate surface outputs while leaving institutional accountability unchanged.</p><p>In practical terms, this means that the promise of AI is not speed; it is reconfiguration. It can externalize some forms of cognition — drafting, retrieval, translation, scaffolding — in ways that reduce effort. But it also shifts the bottleneck toward what remains stubbornly human: judgment, responsibility, coordination, and trust. Treating those bottlenecks as mere frictions to be sped through is not a strategy for progress. It is the illusion again, recurring at scale.</p><h3>Closing: widening the lens</h3><p>Let&#39;s return to the product organization’s faster week. More artifacts were produced than in previous weeks. Replies were quicker. Drafts appeared instantly. Yet the organization felt heavier, not lighter: more pings, more partial commitments, more meetings, more pre-meetings, more re-scoping, more review backlog. The system did not lack output; it lacked convergence. The fast part — producing plausible representations — was solved. The hard part — reducing uncertainty and stabilizing meaning — was made harder by the volume and tempo of plausibility.</p><p>This is the central critique of the contemporary speed ideology. AI can reduce cognitive waste in specific domains, especially where the work is translation, drafting, search, or summarization. But the ideology of speed inflates waste elsewhere — especially in coordination churn and the verification tax — while turning reflection into a deviance rather than a requirement. The risk is not AI’s speed. The risk is using speed as a proxy for cognition.</p><p>If AI is treated as an abstraction layer, the appropriate response is institutional rather than merely individual. The question is not how to move faster; it is what norms, routines, and accountability mechanisms convert Wednesday’s stack of plausible artifacts into Thursday’s durable decisions.</p><p>The long-term impact of AI will depend less on the collapse of time-to-first-output than on whether organizations learn to instrument this new layer — to reward uncertainty reduction over output volume, to build review practices that scale, and to defend attention as the scarce resource that makes judgment possible. In the end, the question is not how fast we can move. It is whether, at the speed we are moving, we are still capable of knowing when we have arrived.</p><h3>References</h3><p>Clark, A., &amp; Chalmers, D. (1998). The extended mind. <em>Analysis, 58</em>(1), 7–19. <a href="https://doi.org/10.1093/analys/58.1.7">https://doi.org/10.1093/analys/58.1.7</a></p><p>Eppler, M. J., &amp; Mengis, J. (2004). The concept of information overload: A review of literature from organization science, accounting, marketing, MIS, and related disciplines. <em>The Information Society, 20</em>(5), 325–344. <a href="https://doi.org/10.1080/01972240490507974">https://doi.org/10.1080/01972240490507974</a></p><p>Gawande, A. (2009). <em>The checklist manifesto: How to get things right</em>. Metropolitan Books.</p><p>Heitz, R. P. (2014). The speed–accuracy tradeoff: History, physiology, methodology, and behavior. <em>Frontiers in Neuroscience, 8</em>, 150. <a href="https://doi.org/10.3389/fnins.2014.00150">https://doi.org/10.3389/fnins.2014.00150</a></p><p>Hutchins, E. (1995). <em>Cognition in the wild</em>. MIT Press. <a href="https://doi.org/10.7551/mitpress/1881.001.0001">https://doi.org/10.7551/mitpress/1881.001.0001</a></p><p>Monsell, S. (2003). Task switching. <em>Trends in Cognitive Sciences, 7</em>(3), 134–140. <a href="https://doi.org/10.1016/S1364-6613(03)00028-7">https://doi.org/10.1016/S1364-6613(03)00028-7</a></p><p>Norman, D. A. (1993). <em>Things that make us smart: Defending human attributes in the age of the machine</em>. Addison-Wesley.</p><p>Olson, D. R. (1994). <em>The world on paper: The conceptual and cognitive implications of writing and reading</em>. Cambridge University Press.</p><p>Risko, E. F., &amp; Gilbert, S. J. (2016). Cognitive offloading. <em>Trends in Cognitive Sciences, 20</em>(9), 676–688. <a href="https://doi.org/10.1016/j.tics.2016.07.002">https://doi.org/10.1016/j.tics.2016.07.002</a></p><p>Rosa, H. (2013). <em>Social acceleration: A new theory of modernity</em>. Columbia University Press.</p><p>Simon, H. A. (1971). Designing organizations for an information-rich world. In M. Greenberger (Ed.), <em>Computers, communications, and the public interest</em> (pp. 37–72). Johns Hopkins Press.</p><p>Sweller, J. (1988). Cognitive load during problem solving: Effects on learning. <em>Cognitive Science, 12</em>(2), 257–285. <a href="https://doi.org/10.1207/s15516709cog1202_4">https://doi.org/10.1207/s15516709cog1202_4</a></p><p>Wajcman, J., &amp; Dodd, N. (Eds.). (2016). <em>The sociology of speed: Digital, organizational, and social temporalities</em>. Oxford University Press. <a href="https://doi.org/10.1093/acprof:oso/9780198782858.001.0001">https://doi.org/10.1093/acprof:oso/9780198782858.001.0001</a></p><p>Weick, K. E. (1995). <em>Sensemaking in organizations</em>. Sage.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e9d802cd354d" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[AI Phobia and the Moralization of Cognition]]></title>
            <link>https://medium.com/@felipe_cas/ai-phobia-and-the-moralization-of-cognition-356c4ace7ded?source=rss-9168a35e9cf7------2</link>
            <guid isPermaLink="false">https://medium.com/p/356c4ace7ded</guid>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[cognition]]></category>
            <dc:creator><![CDATA[Felipe Imbelissieri-Casadei]]></dc:creator>
            <pubDate>Thu, 26 Feb 2026 14:33:31 GMT</pubDate>
            <atom:updated>2026-02-26T14:33:31.892Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*x3gtPQyFtsf5hXcUt5XwDw.png" /></figure><h4>The anxiety that AI will make us cognitively weaker is not a new insight: it is a recurring mistake with a long history of doing political harm. Decline narratives about thinking tools have always been less about protecting minds than about protecting the hierarchies built around older ones. AI phobia repeats that pattern: it mistakes a cultural shift in how cognition is distributed for a biological threat to cognition itself. That confusion becomes regressive, shielding credentials over capabilities, nostalgia over access, and gatekeeping over the equitable distribution of powerful tools.</h4><p>What if the fear that AI will make us dumber is not a warning, but a tradition? If AI is making us dumber, it’s arriving suspiciously late to a moral panic that began at least 2,300 years ago. Every time a new representational tool shows up (e.g., writing, print, calculators, or the internet), we repeat the same story: the tool will weaken the mind, hollow out memory, and replace real understanding with cheap imitation. From Plato’s suspicion of writing to anxieties about television rotting young brains, each generation identifies its own cognitive poison. The pattern is so stable that its repetition should make us suspicious of the diagnosis.</p><p>In 2026, the sharpest version of this fear is educational. The concern is skill atrophy: students who draft with AI will not learn to write. Outsourced problem solving will short-circuit the developmental work that makes later reasoning possible. A generation will grow up fluent in prompting, but fragile in understanding.</p><p>But education has never been a stable checklist of legitimized skills. By 17th-century standards, many of us would read as badly educated because we do not speak Latin and Ancient Greek. We do not carry the same canon in memory. We do not perform erudition in the same social registers. Yet education is not the ritual of possessing knowledge; it is the referential use of knowledge. It is the ability to locate, evaluate, and apply information in context. That is abstraction and externalization at work. The skill frontier moves upward.</p><p>This post argues that the mistake behind many decline narratives is structural. They confuse <em>biological cognitive capacity</em> (i.e., what humans can learn and infer) with <em>functional intelligence</em> (i.e., what humans can do with the tools and representations available), and with <em>cultural coordination capacity</em> (i.e., how societies store and coordinate knowledge). Literacy didn’t make humans smarter in the way the panic assumes, and it didn’t make us dumber either. It changed what became <em>easy</em>, what became <em>valuable</em>, and who gained access to symbolic power.</p><p>AI is doing the same, faster. AI externalizes cognition at unprecedented scale and speed. The tools compress decades of symbolic work (e.g., drafting, translation, synthesis, search) into seconds. This shift does not imply biological decline. What it implies is a fundamental reallocation of skill, status, and power. When a representational interface changes this quickly, what counts as competence changes with it. The real question is not whether minds will weaken, but who will govern the cognitive infrastructure AI creates, who will be allowed to benefit from it, and who will be excluded by nostalgic defenses of older forms of legitimacy.</p><p>Skill atrophy is not imaginary. What you do not practice, you do not become fluent in. But the highest-stakes risks are still governance risks: concentration of power, labor displacement, surveillance, and credential gatekeeping. If we talk about protecting cognition instead of protecting people and capabilities, we risk building a politics of regression: nostalgic, exclusionary, and easy to weaponize.</p><h3>Skill atrophy as the latest chapter in an ancient anxiety</h3><p>Plato stages the argument in its most durable form. In the <em>Phaedrus</em>, the inventor-god Theuth offers writing as a gift. King Thamus refuses the sales pitch: writing would encourage forgetfulness by shifting memory outside the mind; it would create the <em>appearance</em> of wisdom rather than wisdom itself (Plato, trans. 1892).</p><p>The classroom version is familiar: AI can produce the appearance of competence without competence.</p><p>Replace writing with generative AI and the rhetorical structure remains intact, though the pace of change accelerates. The tool externalizes a skill. That skill gets treated as the essence of thinking, and externalization becomes decay. Today’s versions add modern flourishes. Screens ruin attention, hyperlinks ruin depth, AI ruins reasoning. But the rhetorical chassis is the same (Carr, 2010; Postman, 1992).</p><p>Representational shifts really do relocate cognitive labor. Memory moves into inscriptions; calculation moves into notation; composition moves into drafts and templates; search moves into indexes and ranking. When the tool changes, what competence <em>looks like</em> changes. People whose status depended on the old scarcity often experience that as an attack.</p><p>But we should name the fragile assumption at the center of most decline narratives: <em>thinking counts only when it is internal, effortful, and unaided</em>. Cognitive science has spent decades undermining that assumption. A great deal of thinking is not inside the skull in the folk sense. It is system-level achievement distributed across representations, artifacts, and social coordination (Clark &amp; Chalmers, 1998; Hutchins, 1995).</p><p>The claim that AI will weaken our minds misreads what kind of shift is happening. AI externalizes cognition. Whether that implies biological decline or a reallocation of skill, status, and power depends on which conceptual categories we use to analyze the change.</p><h3>The conceptual spine: cognitive power, functional intelligence, cultural coordination capacity</h3><p>The entire argument rests on a three-part distinction:</p><ol><li><strong>Cognitive power:</strong> biological capacity; early-developing learning and inference capacities; abstraction and concept formation.</li><li><strong>Functional intelligence:</strong> context-dependent performance; what people can do with available tools and representations.</li><li><strong>Cultural coordination capacity:</strong> societal coordination; how effectively groups store, transmit, and coordinate knowledge across time and scale.</li></ol><p>Literacy is the strongest counterexample to the claim that tools make us dumber. Reading and writing are cultural inventions too recent to be products of natural selection, learned by repurposing older neural circuits. Dehaene and colleagues describe this as neuronal recycling (Dehaene, 2009; Dehaene &amp; Cohen, 2007).</p><p>That framing matters. Literacy is not a new biological organ of intelligence. It is a representational interface that makes certain operations cheap, scalable, and transmissible. It raises functional intelligence and cultural coordination capacity without implying that literate humans have more cognitive power than pre-literate humans.</p><p>Developmental psychology reinforces the point: humans enter the world with structured expectations about objects, agents, number, and space, and this core knowledge does not depend on writing (Spelke, 2000). Cultural tools ride on top of these capacities. They don’t manufacture them from scratch.</p><p>This framing reveals the category error embedded in AI decline narratives. AI changes the <em>tooling layer</em>. It changes how we search, draft, translate, and manipulate symbols. That can redistribute skill and status, and it can change what gets practiced, rewarded, and credentialed. But the likely outcome is not biological decline; it is institutional reshuffling.</p><p>This is the first governance lesson: if you treat functional intelligence shifts as biological decline, you will aim policy at the wrong target.</p><h3>Writing as exaptation: cultural indispensability does not equal biological necessity</h3><p>One reason fear is intuitive is that writing is indispensable to modern life. If writing is indispensable, it’s easy to slide into thinking it must be indispensable to intelligence.</p><p>Gould and Lewontin’s critique of adaptationist storytelling is useful here. It doesn’t explain writing, but it gives us a vocabulary for avoiding a common mistake: treating whatever is essential <em>now</em> as whatever was essential <em>by nature</em> (Gould &amp; Lewontin, 1979).</p><p>Writing functions as an exaptation: a culturally engineered system built on older capacities (e.g., speech, vision, memory, and coordination), which later becomes indispensable for complex societies. That is very close to the classic exaptation story, a capability built out of pre-existing parts, then co-opted and stabilized by institutions.</p><p>Once you see that, a second governance lesson emerges: when we conflate cultural amplification with biological capacity, we misread inequality and gatekeeping as evidence of innate intelligence. This establishes a boundary condition: treating people without access to dominant symbolic forms as intrinsically less capable, when what they often lack is access to the interface (e.g., schooling, time, safety, credentials, and legitimized practice).</p><p>AI threatens to scramble that interface. And when an interface scrambles, elites often reach for a moral story about decline.</p><h3>Speech-first cognition and early abstraction: what writing is not</h3><p>Human cognition is grounded in speech and shared intentionality, not writing. Tomasello’s work on the origins of human communication emphasizes cooperation and shared goals as the substrate for language and learning (Tomasello, 2008). Writing comes much later as a technology that transforms cultural memory and coordination (Goody, 1977; Ong, 1982/2013).</p><p>This is not a romantic celebration of orality. It establishes the developmental priority of spoken language and embodies social interaction over written symbolic systems. Writing is an external representational layer. It can reorganize attention, expand what can be stored, and reshape institutions, but it is not the foundation of human abstraction.</p><p>That foundation is older and more resilient. Core knowledge appears early (Spelke, 2000). Susan Carey provides a canonical example of early childhood concept formation that does not require literacy: the formation of exact number concepts for small sets. In her work on conceptual development and core cognition, Carey argues that infants and young children possess a preverbal numerical system often called parallel individuation. This system allows infants to represent and reason about exact quantities for small numbers (e.g., typically 1–3, sometimes 4) without language, symbols, or counting words (Carey, 2009). The capacity to form concepts and to bootstrap new representational resources does not begin with textbooks; it begins with infants. Which is why the strongest version of the fear, the biological one, has such a weak bridge from evidence to conclusion.</p><p>If you want to worry about cognition, the productive focus is on the environments that shape it: schools, incentives, labor markets, and the distribution of time and attention. That is where AI’s effects will actually land.</p><h3>Externalization is not a cognitive dead end: extended and distributed cognition</h3><p>The idea that the brain works like a muscle is seductive: if you don’t lift the weight, you atrophy. But cognition is not a single muscle, and the unit of analysis is rarely the isolated brain.</p><p>The Extended Mind thesis argues that external artifacts can function as integrated components of cognition, not merely aids (Clark &amp; Chalmers, 1998). Clark’s broader work makes the same point in more detail: intelligence often emerges from the coupling of brain, body, and world, including tools that stabilize memory and action (Clark, 2008).</p><p>The distributed cognition framework pushes the idea from individuals to systems. In real settings (e.g., navigation, aviation, and medicine), intelligence is coordinated across people, procedures, representations, and artifacts (Hutchins, 1995). Donald’s account of external symbolic storage adds an evolutionary-cultural hinge: external records (i.e., exograms) create a new cognitive ecology for societies (Donald, 1991).</p><p>AI belongs in this lineage. It is a representational layer that compresses search, drafting, translation, and synthesis. It makes proposal-generation cheap. That shifts scarcity toward evaluation, epistemic judgment, and goal selection: deciding what to trust, keep, discard, and pursue.</p><p>This is where the phobia often misfires. It treats a system-level reallocation as an individual-level decline. The more accurate diagnosis is harder and more political: AI changes which skills are scarce, and scarcity is where status and power attach.</p><h3>Neuroscience and atrophy narratives</h3><p>The question of neuroplasticity is real, but the path from neurological adaptation to moral diagnosis is treacherous. The problem is how quickly rewiring becomes a pop-neuroscience morality play.</p><p>Uttal’s critique of overconfident localization is a useful warning label here: simplistic, modular stories about distinct brain regions weakening tend to outrun what evidence can support (Uttal, 2001). Popular claims about left-brain/right-brain dominance (i.e., the widely debunked notion that analytical people are “left-brained” while creative people are “right-brained”) exemplify how neuroscientific metaphors migrate into cultural narratives that have little empirical grounding. You can make a good governance argument without pretending you have a neurological diagnosis.</p><p>If you want a higher-confidence claim, it’s this: incentives and institutions will change faster than brains.</p><h3>The heavy historical hinge: Nordau, degeneracy discourse, and the moralization of cognition</h3><p>If the cognitive science argument is the defuser, history is the warning. The question is not only whether brains adapt, but how societies have historically responded to representational change. Decline narratives about cognition are rarely just about cognition. They police boundaries by diagnosing cultural change as pathology. Understanding that historical pattern matters because it reveals how moral panic becomes policy.</p><p>In late 19th-century Europe, degeneration discourse offered a quasi-clinical vocabulary for cultural anxiety. Its organizing concepts (i.e., heredity, exhaustion, and decadence) framed modernity itself as a medical problem. Max Nordau’s <em>Degeneration</em> exemplifies the diagnostic posture: modern art and intellectual movements treated as symptoms of social illness (Nordau, 1892). The point here is not to litigate Nordau. It is to notice the <em>form</em>: contested modernity re-described as disease.</p><p>Once cultural change is framed as pathology, the response becomes treatment: repression, exclusion, purification, rollback. The language of health and normality starts doing political work while pretending to be neutral (Canguilhem, 1966/1991).</p><p>Degeneracy discourse also traveled with prejudice. Antisemitic caricatures often cast Jews as physically weak and intellectual, and later debates about muscular Judaism framed themselves against that stereotype (Presner, 2007). It is historically ironic that Nordau, a Jewish physician, wrote inside a diagnostic vocabulary that could stigmatize modern life. The lesson is structural. Decline narratives can translate status panic into objective diagnosis, and they can be absorbed without losing their capacity to harm.</p><p>AI phobia often repeats the same structure in a contemporary key: cultural change, moral panic, pseudo-biological rhetoric, calls to protect the real. The target shifts from decadent art to algorithmic text. The move is familiar: treat a representational transition as a contamination of legitimacy.</p><p>This is where the fear of AI making us dumber becomes regressive. If real thinking means solo drafting, memorized recall, and unaided production, then you can defend old hierarchies as natural. You can treat new entrants as illegitimate, not because they lack capacity, but because they used the wrong interface. You can smuggle gatekeeping into a protective story about human cognition. Decline narratives are easy to weaponize. Once weaponized, they tend to land on the already-excluded. Mis-aimed fear becomes policy.</p><h3>Automation and habituation: expertise is built from offloading, not threatened by it</h3><p>One of the simplest defusers is also the most practical: expertise automates subroutines. What was once effortful becomes infrastructure. Reduced conscious effort is not reduced competence; it is often the signature of skill.</p><p>Calculators didn’t destroy mathematics. They changed what it meant to be good at math. Before electronic calculators became widespread in the 1970s, mathematical competence included fluency with logarithm tables and slide rules. The scarce skill shifted upward: from arithmetic production to modeling, estimation, interpretation, and error-checking. The same reallocation is happening with writing. When drafts become cheap, the scarce ability shifts to revision, argument structure, source discipline, and judgment.</p><p>The relevant question is not whether we should offload cognitive work. We always have. What matters is what we demand after offloading. Do we demand verification? Do we demand clarity about assumptions? Do we train evaluation as a first-class skill?</p><p>That is a governance question disguised as a learning question.</p><h3>AI as an epistemic instrument and an inequality lens</h3><p>The automation argument explains why offloading is normal. But AI also functions as an epistemic instrument: a testbed for theories of learning, structure, and abstraction (Lake et al., 2017; Tenenbaum et al., 2011). Modern machine learning reveals how much of what we call intelligence is pattern recognition over structured data, and how much depends on prior knowledge and compositional reasoning.</p><p>Socially, the sharper effect is on access to abstraction. AI lowers the cost of symbolic operations (e.g., drafting, coding, translation, summarization, and tutoring). That can expand functional intelligence in a way that looks like intelligence from the outside. And that exposes an uncomfortable truth: what we often rewarded as merit was frequently institutionalized access to symbolic tools (e.g., education, time, mentorship, and membership). A student using AI to draft an essay reveals what we’ve called merit was often just access (e.g., tutors, editing, quiet rooms, feedback loops, and legitimized practice).</p><p>Sen’s capabilities framing helps keep this grounded: inequality is about real opportunities and freedoms, not only formal rights (Sen, 1999). Bourdieu’s notion of cultural capital explains how institutions convert familiarity with dominant symbolic forms into status and advantage (Bourdieu, 1986). AI is poised to disrupt both, either by democratizing access to the interface or by concentrating it further.</p><p>The risk model clarifies itself when we stop conflating tools with cognition. The central risks are not that people will forget how to think. The risks are that AI becomes cognitive infrastructure owned by a few: an engine for surveillance, labor displacement without bargaining power, and the privatization of knowledge work. Governance is necessary, and regulatory frameworks like the European Union’s Artificial Intelligence Act (2024) attempt to codify accountability, transparency, and contestability in high-risk AI systems. But protecting cognition is the wrong frame. It risks protecting credentials and bottlenecks rather than protecting capabilities and agency.</p><p>If we care about human thinking, the agenda is not nostalgia. It is institutional design: access, accountability, contestability, and the distribution of the benefits of abstraction tools.</p><h3>Closing question</h3><p>If intelligence has always been distributed across brains, tools, social coordination, and institutional memory, then a tool like AI is less a rupture than a continuation of a long cognitive trajectory. The real question is not whether AI will replace thinking, but who will govern the cognitive infrastructure it creates, and who will be allowed to benefit from it. If intelligence has always been a system-level achievement rather than a solitary one, why do we keep pretending otherwise? And who benefits from that pretense?</p><h3>References</h3><p>Bourdieu, P. (1986). The forms of capital. In J. G. Richardson (Ed.), <em>Handbook of theory and research for the sociology of education</em> (pp. 241–258). Greenwood.</p><p>Canguilhem, G. (1991). <em>The normal and the pathological</em> (C. R. Fawcett &amp; R. S. Cohen, Trans.). Zone Books. (Original work published 1966)</p><p>Carey, S. (2009). <em>The origin of concepts</em>. Oxford University Press.</p><p>Carr, N. (2010). <em>The shallows: What the Internet is doing to our brains</em>. W. W. Norton.</p><p>Clark, A. (2008). <em>Supersizing the mind: Embodiment, action, and cognitive extension</em>. Oxford University Press.</p><p>Clark, A., &amp; Chalmers, D. (1998). The extended mind. <em>Analysis, 58</em>(1), 7–19.</p><p>Dehaene, S. (2009). <em>Reading in the brain: The new science of how we read</em>. Viking/Penguin.</p><p>Dehaene, S., &amp; Cohen, L. (2007). Cultural recycling of cortical maps. <em>Neuron, 56</em>(2), 384–398. doi:10.1016/j.neuron.2007.10.004</p><p>Donald, M. (1991). <em>Origins of the modern mind: Three stages in the evolution of culture and cognition</em>. Harvard University Press.</p><p>European Union. (2024). Regulation (EU) 2024/1689 (Artificial Intelligence Act). <em>Official Journal of the European Union</em>.</p><p>Goody, J. (1977). <em>The domestication of the savage mind</em>. Cambridge University Press.</p><p>Gould, S. J., &amp; Lewontin, R. C. (1979). The spandrels of San Marco and the Panglossian paradigm: A critique of the adaptationist programme. <em>Proceedings of the Royal Society B: Biological Sciences, 205</em>(1161), 581–598. doi:10.1098/rspb.1979.0086</p><p>Hutchins, E. (1995). <em>Cognition in the wild</em>. MIT Press. doi:10.7551/mitpress/1881.001.0001</p><p>Lake, B. M., Ullman, T. D., Tenenbaum, J. B., &amp; Gershman, S. J. (2017). Building machines that learn and think like people. <em>Behavioral and Brain Sciences, 40</em>, e253.</p><p>Nordau, M. (1892). <em>Degeneration</em>. University of Nebraska Press.</p><p>Organisation for Economic Co-operation and Development. (2019). <em>Recommendation of the Council on Artificial Intelligence</em>. OECD Legal Instruments.</p><p>Ong, W. J. (2013). <em>Orality and literacy: The technologizing of the word</em> (30th anniversary ed.). Routledge. (Original work published 1982)</p><p>Plato. (1892). <em>Phaedrus</em> (B. Jowett, Trans.). Oxford University Press.</p><p>Postman, N. (1992). <em>Technopoly: The surrender of culture to technology</em>. Knopf.</p><p>Presner, T. S. (2007). <em>Muscular Judaism: The Jewish body and the politics of regeneration</em>. Routledge.</p><p>Sen, A. (1999). <em>Development as freedom</em>. Oxford University Press.</p><p>Spelke, E. S. (2000). Core knowledge. <em>American Psychologist, 55</em>(11), 1233–1243. doi:10.1037/0003–066X.55.11.1233</p><p>Tenenbaum, J. B., Kemp, C., Griffiths, T. L., &amp; Goodman, N. D. (2011). How to grow a mind: Statistics, structure, and abstraction. <em>Science, 331</em>(6022), 1279–1285. doi:10.1126/science.1192788</p><p>Tomasello, M. (2008). <em>Origins of human communication</em>. MIT Press.</p><p>Uttal, W. R. (2001). <em>The new phrenology: The limits of localizing cognitive processes in the brain</em>. MIT Press.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=356c4ace7ded" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[What’s Missing for Machine Thinking?]]></title>
            <link>https://medium.com/@felipe_cas/whats-missing-for-machine-thinking-a31a5444c30a?source=rss-9168a35e9cf7------2</link>
            <guid isPermaLink="false">https://medium.com/p/a31a5444c30a</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[psychology]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[superintelligence]]></category>
            <category><![CDATA[development]]></category>
            <dc:creator><![CDATA[Felipe Imbelissieri-Casadei]]></dc:creator>
            <pubDate>Thu, 29 Jan 2026 05:17:45 GMT</pubDate>
            <atom:updated>2026-01-29T05:17:45.235Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_aw-5HGhmQxJhqm0ydPGPQ.png" /></figure><p><strong>A nine-month-old reaches for a toy hidden behind a screen. She expects it to still exist. She’ll look longer if it vanishes impossibly or moves through a solid barrier. No one taught her object permanence through explicit instruction; this knowledge emerged early, universally, without schooling. Can AI systems approximate even this foundational layer of human cognition? In many ways, yes. In the ways that matter most for sustained intelligence, not yet.</strong></p><p>Contemporary AI excels at fast, uncertainty-tolerant inference in structured domains: the family of capacities developmental science calls <em>core knowledge</em> (Spelke &amp; Kinzler, 2007). Large language models complete patterns, integrate heterogeneous cues, and produce rapid predictions that functionally overlap with structured human expectations. Yet these same systems fail in ways that reveal a fundamental asymmetry: strong short-horizon inference alongside fragile long-horizon organization. An AI might remember a commitment for one conversation, then confidently contradict it three sessions later. It can answer medical questions accurately but cannot track which advice it gave to which patient, or why it changed its recommendation.</p><p>This essay argues that machine thinking requires four procedural ingredients beyond current AI capabilities: <strong>persistence</strong> (memory that survives episodes), <strong>continual learning</strong> (updating without erasure), <strong>metacognition</strong> (knowing what you don’t know), and <strong>grounded interaction</strong> (feedback with teeth). These aren’t exotic requirements. They describe capacities that human children deploy before kindergarten. The gaps are procedural and developmental, not about raw intelligence or compute.</p><p>This analysis focuses on procedural competence, deliberately setting aside questions of consciousness, which require separate treatment. The question here is narrower: what would it take for AI systems to accumulate knowledge and maintain coherence across time, the way developing humans do?</p><h3>The Universal Baseline: Why Species-Typical Cognition Matters</h3><p>Public discussion often begins with an unhelpful opposition. In one register, modern systems are dismissed as mere <em>autocomplete</em>: superficial association rather than real inference. In another, the same systems are celebrated as approaching <em>superintelligence</em>, as if there were a single quantity that, once sufficiently large, converts pattern completion into comprehensive understanding.</p><p>These positions disagree about the forecast but share a simplifying assumption: cognition is unitary. Both believe a single verdict about <em>thinking</em> can be read off from a few salient demonstrations or failures. A more defensible approach specifies which competencies are at issue, how they develop, and how they persist over time.</p><p>The motivation for a universal baseline is both scientific and ethical. If we define <em>thinking</em> by culturally contingent achievements (formal logic, scientific method, literate introspection), then we risk excluding many humans. That’s implausible and suggests the yardstick is mis-specified.</p><p>A better baseline is species-typical cognition: capacities that emerge early in development, are widely distributed across humans independent of schooling, and support prediction and action under uncertainty. This means asking whether AI systems approximate the cognitive procedures available to human infants before formal education: capacities like rapid inference about objects, agents, and number that appear early and universally. This <em>universal bar</em> is not a claim about essence. It’s a proposal about what counts as a minimally fair reference class when the topic is cognition rather than credentialed performance.</p><p>This framing aligns with formal attempts to define intelligence in procedural terms. Legg and Hutter’s <a href="https://arxiv.org/abs/0712.3329">environment-based definition</a> emphasizes competence across environments rather than mastery of a particular test. <a href="https://arxiv.org/abs/1911.01547">Chollet’s critique</a> of benchmark-centric evaluation emphasizes acquisition efficiency and the ability to learn new tasks without extensive task-specific engineering. The shared idea: what matters is not an isolated score but a system’s capacity to build usable structure from experience and deploy it flexibly under uncertainty.</p><h3>Core Knowledge as an Empirical Anchor</h3><p>Developmental science provides empirical grounding for this idea through the core knowledge framework. Human cognition is supported by early-emerging systems that organize experience around objects, agents, number, and space (Spelke &amp; Kinzler, 2007). These systems aren’t explicit, articulated theories. They provide structured expectations that narrow the hypothesis space and make rapid generalization possible.</p><p>Computational work in cognitive science has clarified how this happens by treating learning as model building in which inductive constraints and experience jointly determine what can be inferred from sparse data (Tenenbaum et al., 2011; Ullman &amp; Tenenbaum, 2020). This doesn’t require assuming that infants are optimal statisticians. Humans display fast, uncertainty-tolerant inference well before they acquire culturally mediated skills, and that early competence is structured rather than merely reactive.</p><p>The language of probabilistic modeling, often labeled <em>Bayesian</em>, is useful here as a way to specify what must be represented and how uncertainty should be handled, not as a metaphysical claim about biological implementation. In practice, <em>Bayesian</em> in this setting often means probabilistic inference over structured hypotheses with inductive biases that make learning possible. Treating it as a modeling vocabulary helps avoid predictable disputes about whether brains compute exact posteriors. What matters is function: what representations and update rules suffice for the observed pattern of generalization, and what constraints are doing the work.</p><p>This emphasis on decomposition has instructive historical parallels. Gould’s critique of intelligence measurement targeted a recurrent error: taking a complex construct, assigning it a number, and then treating that number as if it were a natural kind from which broad conclusions follow. The methodological warning is durable. Once a construct is reified, debates about mechanisms are displaced by arguments from rankings. In AI discourse the same temptation appears when <em>thinking</em> is treated as one knob, and when claims about systems are made by extrapolating from a subset of tasks to a global verdict about <em>mind</em>.</p><h3>Why Benchmarks Mislead</h3><p>We struggle to interpret benchmarks because training experience confounds them. Systems exposed to vast datasets can appear broadly competent, but the appearance doesn’t by itself indicate how efficiently the system acquires new skills, how it adapts under distribution shift, or how much task-specific scaffolding is hidden in data and training pipelines.</p><p>This is why acquisition efficiency has been proposed as a more diagnostic measure than raw task score (Chollet, 2019), and why environment-based definitions emphasize breadth across tasks rather than mastery of a narrow test (Legg &amp; Hutter, 2007). The universal-bar perspective makes a parallel move: it asks not whether a system can pass a difficult exam, but whether it approximates the procedures that support early-emerging, widely shared human competence.</p><p>On that narrower question, current ML plausibly approximates parts of core inference. In many contexts, large models map partial information to plausible completions, integrate heterogeneous cues, and produce rapid predictions in a way that functionally overlaps with some roles played by structured human expectations.</p><p>The overlap shouldn’t be overstated. Human core knowledge is embedded in a developmental trajectory in which perception, action, and learning interact over years. Contemporary models may succeed within an episode and fail to sustain stable commitments across time, to revise beliefs coherently, or to regulate uncertainty in a way that guides information seeking. Strong short-horizon inference sits alongside fragile long-horizon organization.</p><p>These gaps might seem permanent. But Sutton’s <a href="http://www.incompleteideas.net/IncIdeas/BitterLesson.html"><em>bitter lesson</em></a> suggests otherwise. Historically, scalable methods that exploit data and computation have often outperformed approaches built around hand-coded structure, even when the latter appear more <em>mind-like</em>. This is consistent with the possibility that scaling will move systems closer to a universal baseline faster than many expected.</p><p>Yet the lesson doesn’t imply that scale alone delivers the missing procedures. Persistence, continual revision, and metacognitive control aren’t automatic consequences of increased parameter count. They are engineering and scientific targets that can remain binding constraints even as short-horizon prediction improves.</p><h3>Four Missing Ingredients</h3><p>The question <em>what is missing?</em> is easy to pose and easy to answer badly. A generic response (more <em>compute</em>) is under-specified because it treats cognition as scalar and neglects properties that are intrinsically temporal. A more useful alternative describes progress through procedural targets. A system begins to approximate humanlike machine thinking when it can:</p><ul><li>Accumulate and manage memory across long horizons</li><li>Revise beliefs with new evidence without catastrophic loss</li><li>Calibrate uncertainty and seek information strategically</li><li>Learn to learn across domains rather than solve each new domain from scratch</li><li>Remain stable under distribution shift</li></ul><p>This reframing preserves the ambition of broad competence but makes the ambition measurable.</p><h3>1. Persistence: Memory That Survives</h3><p>Memory that survives episodes and remains usable across time. Persistence is not equivalent to a long context window. A persistent system distinguishes short-lived context from stored representations that can be retrieved, audited, and revised. It preserves provenance so that a remembered claim can be traced to an episode, a source, or an evidential context.</p><p>Development is path-dependent. Knowledge becomes valuable when it can constrain planning and inference over long horizons, and when it can be revisited to correct error rather than merely repeated. Consider a toddler who learns that the family dog is gentle. This knowledge persists across days, guides approach behavior, and can be revised if the dog snaps. The memory isn’t just retrieved; it shapes subsequent action and can be updated when contradicted.</p><p>In current ML, memory has become an explicit design space in agentic systems, including external stores and learned retrieval policies (Zhang et al., 2025). Yet predictable failure modes remain common: retrieval is brittle under distraction and interference; stored content drifts or fragments; and agents can confabulate when recall is uncertain. An agent might remember it promised not to delete files, but forget this commitment three conversations later when asked to clean up the directory. Even when the correct information is retrieved, it often isn’t integrated as a binding constraint on action, so goals, preferences, and prior decisions aren’t reliably honored over time.</p><p>Evaluation should be longitudinal: week-later consistency checks, tasks in which earlier decisions constrain later ones, and stress tests in which an agent must reconcile conflicting records rather than defaulting to fluent improvisation. Research needs to treat memory as a controlled process with explicit policies for storage, consolidation, and revision, coupled to representations of uncertainty and evidential status.</p><h3>2. Continual Learning: Updating Without Erasure</h3><p>The capacity to incorporate new information sequentially without catastrophic interference is known as continual learning. Belief revision is the coherent update of commitments and their implications. In open environments, adaptation is necessary. But indiscriminate adaptation is dangerous.</p><p>A developing system must balance plasticity (the capacity to learn) with stability (the capacity to preserve what remains valid). Catastrophic forgetting (the tendency to lose old skills when learning new ones) remains a canonical obstacle in sequential learning. Methods such as elastic weight consolidation (a technique that protects parameters important to prior tasks) formalize one approach to stability (Kirkpatrick et al., 2017).</p><p>The broader challenge is not only to retain performance but to revise beliefs coherently. When a three-year-old learns that whales aren’t fish, she doesn’t just update one category label. She adjusts related inferences: whales need air, they’re warm-blooded, they nurse their young. The revision propagates through her conceptual network.</p><p>Biological memory research highlights that prediction error and partial reminders can gate updating, supporting a disciplined analogy: revision should be selective and constrained (Sinclair &amp; Barense, 2019; Rouhani &amp; Niv, 2021; Kennedy et al., 2024). This isn’t an invitation to import biological mechanisms uncritically. It’s a reminder that updating must be regulated.</p><p>In ML terms, this calls for uncertainty-aware update policies, mechanisms that track evidential status, and evaluation protocols that probe coherence. After a system revises one claim, does it adjust related inferences appropriately while retaining unrelated competence? Told “Actually, the meeting is Thursday, not Tuesday,” a system should update who can attend, what conflicts exist, and when to send reminders, not just change an isolated date entry.</p><h3>3. Metacognition: Knowing What You Don’t Know</h3><p>Inference about one’s own reliability coupled to control policies that allocate effort and information seeking. Fast inference can be impressive while remaining brittle if a system lacks a procedure for deciding when it should be trusted, when it should seek more evidence, and when it should defer.</p><p>Formal work treats confidence as information and separates metacognitive sensitivity (the ability to distinguish reliable from unreliable judgments) from raw performance (Dayan, 2023). Information search is framed as a control problem guided by confidence (Schulz et al., 2023). Developmental evidence shows that infants modulate effort based on evidence, supporting the idea that uncertainty-sensitive control policies can be studied mechanistically (Lucca et al., 2020).</p><p>A two-year-old reaching for a cup looks longer at it when the reach is difficult, gathering more visual information before committing. She adjusts her strategy based on her own uncertainty. A human uncertain whether Denmark borders Germany will look it up. Current systems often answer confidently either way.</p><p>Calibration should be treated as behavior, not just output. It’s not enough to output a probability. The system must use uncertainty to govern action. Evaluation should therefore include selective prediction under shift, strategic question asking when clarification is cheap, bounded-search tasks that trade off information gain against time or resource costs, and settings in which deferral is rewarded when the cost of a confident error is high.</p><h3>4. Grounded Interaction: Feedback With Teeth</h3><p>Constraints and error signals with consequences. The claim for embodiment can be stated narrowly. A biological body is not a prerequisite for all reasoning, and many forms of abstract inference can be performed without direct sensorimotor grounding.</p><p>Interaction supplies interventions. Actions change observations, allowing an agent to test hypotheses and correct errors in ways passive training cannot. Grounded cognition emphasizes that conceptual content is tightly linked to perceptual and motor systems (Barsalou, 2008), while predictive processing and active inference frameworks highlight learning in perception-action loops (Clark, 2013; Friston, 2017).</p><p>A robot learning to grasp must discover that soft objects compress while rigid ones don’t, something no passive dataset teaches directly. The object pushes back. The gripper slips. These are not labels in a training set but physical constraints that force the system to build accurate models.</p><p>Even without adopting the strongest theoretical claims, the engineering implication is clear: closed-loop training can reduce reliance on superficial cues, teach affordances and causal structure, and produce feedback that stabilizes learning.</p><p>Evaluation should prioritize long-horizon tasks with consequential feedback and stability under novel dynamics, including environments where exploration is required to disambiguate latent states. Whether interaction produces more stable generalization under shift matters more than whether embodiment is a philosophical prerequisite.</p><h3>From Static Models to Developing Agents</h3><p>These ingredients suggest a minimal functional sketch that shifts attention from a static <em>model</em> to a <em>developing agent</em>. The components:</p><ul><li>An inference engine that supports rapid prediction under uncertainty</li><li>Persistent memory that stores episodes and consolidated abstractions along with provenance</li><li>A belief-revision procedure that updates memory and parameters while managing stability and plasticity, and that records what changed and why</li><li>A metacognitive controller that monitors reliability and allocates information seeking, effort, and commitment</li><li>A channel for situated interaction that supplies grounded feedback through action</li></ul><p><strong>Evaluate these functions jointly.</strong> Their failures interact. Weak metacognition turns memory into a source of confident error. Weak revision turns persistence into brittle inertia. Broad competence across environments, if achievable, is more plausibly the product of these interacting procedures than the product of a single scaling axis.</p><p>This argument is deliberately limited to competence and developmental stability. It doesn’t settle questions about consciousness. A system with persistent memory, belief revision, and metacognitive control could support higher-order cognition and a self-model that tracks its own uncertainty and history. Those features are relevant to some theories that link access, global availability, or self-modeling to consciousness. They are not, by themselves, decisive evidence of phenomenal experience.</p><p>Behavioral competence and coherent self-report can be produced by mechanisms that remain silent on phenomenology, and the absence of a theory-neutral test means claims about consciousness should be separated from claims about thinking-as-competence.</p><h3>What This Means for Progress</h3><p>Three claims follow from the universal baseline.</p><p><strong>First:</strong> Contemporary ML is plausibly near a species-typical baseline for certain forms of fast inference. That proximity is a genuine scientific achievement rather than a semantic trick.</p><p><strong>Second:</strong> The most consequential remaining gaps are procedural and developmental. Persistence, belief revision, metacognitive control, and grounded interaction are the procedures that allow competence to accumulate and remain stable across time.</p><p><strong>Third:</strong> The most informative success metrics should emphasize developmental stability and self-directed adaptation rather than one-off benchmark performance.</p><p><strong>Resist reifying <em>thinking</em> into a single number</strong>, whether it’s a benchmark score, a parameter count, or an imagined quotient. The more scientific move is to specify the subsystem and the procedure: what is being represented, what is being updated, what controls the updates, and how the system behaves when it is uncertain.</p><p>That decomposition doesn’t eliminate disagreement about where current systems stand. But it disciplines the disagreement by tying it to mechanisms and to tests.</p><h3>References</h3><ul><li>Barsalou, L. W. (2008). Grounded cognition. <em>Annual Review of Psychology</em>.</li><li><a href="https://arxiv.org/abs/1911.01547">Chollet, F. (2019). On the measure of intelligence.</a></li><li>Clark, A. (2013). Whatever next? Predictive brains, situated agents, and the future of cognitive science. <em>Behavioral and Brain Sciences</em>.</li><li>Dayan, P. (2023). Metacognitive information theory. <em>Open Mind</em>.</li><li>Friston, K. (2017). Active inference: A process theory. <em>Neural Computation</em>.</li><li>Gould, S. J. (1981/1996). <em>The Mismeasure of Man</em>.</li><li>Kennedy, N. G. W., et al. (2024). Prediction error determines how memories are organized in memory.</li><li>Kirkpatrick, J., et al. (2017). Overcoming catastrophic forgetting in neural networks.</li><li><a href="https://arxiv.org/abs/0712.3329">Legg, S., &amp; Hutter, M. (2007). Universal intelligence: A definition of machine intelligence.</a></li><li>Lucca, K., Horton, R., &amp; Sommerville, J. A. (2020). Infants rationally decide when and how to deploy effort. <em>Nature Human Behaviour</em>.</li><li>Rouhani, N., &amp; Niv, Y. (2021). Signed and unsigned reward prediction errors dynamically enhance learning and memory. <em>eLife</em>.</li><li>Schulz, L., Fleming, S. M., &amp; Dayan, P. (2023). Metacognitive computations for information search: Confidence in control. <em>Psychological Review</em>.</li><li>Sinclair, A. H., &amp; Barense, M. D. (2019). Prediction error and memory reactivation: How incomplete reminders drive reconsolidation. <em>Trends in Cognitive Sciences</em>.</li><li>Spelke, E. S., &amp; Kinzler, K. D. (2007). Core knowledge. <em>Developmental Science, 10</em>(1), 89–96.</li><li><a href="http://www.incompleteideas.net/IncIdeas/BitterLesson.html">Sutton, R. (2019). The bitter lesson.</a></li><li>Tenenbaum, J. B., Kemp, C., Griffiths, T. L., &amp; Goodman, N. D. (2011). How to grow a mind: Statistics, structure, and abstraction. <em>Science, 331</em>(6022), 1279–1285.</li><li>Ullman, T. D., &amp; Tenenbaum, J. B. (2020). Bayesian models of conceptual development: Learning as building models of the world. <em>Annual Review of Developmental Psychology</em>.</li><li>Zhang, Z., et al. (2025). A survey on the memory mechanism of large language model-based agents. <em>ACM Computing Surveys</em>.</li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a31a5444c30a" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[(UX of AI, Part 2) The Trust Ladder]]></title>
            <link>https://medium.com/design-bootcamp/ux-of-ai-part-2-the-trust-ladder-7d72d9e5b538?source=rss-9168a35e9cf7------2</link>
            <guid isPermaLink="false">https://medium.com/p/7d72d9e5b538</guid>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[trust]]></category>
            <category><![CDATA[ux-research]]></category>
            <category><![CDATA[ux]]></category>
            <category><![CDATA[technology]]></category>
            <dc:creator><![CDATA[Felipe Imbelissieri-Casadei]]></dc:creator>
            <pubDate>Sun, 16 Nov 2025 21:41:00 GMT</pubDate>
            <atom:updated>2025-11-16T23:50:13.763Z</atom:updated>
            <content:encoded><![CDATA[<p><em>in: UX of Agentic AI: Beyond The Obvious</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*QDbXMkjIeANfiBn-sS-lhQ.png" /></figure><p>In <a href="https://medium.com/design-bootcamp/ux-of-agentic-ai-policy-and-evaluation-of-agents-beyond-vibes-c8c06cb7480f">Part 1</a>, we laid the foundation by establishing that, like children, our AI agents will not succeed if designed around unstructured pattern matching, or blind anthropomorphism. Children, the best learning system we know to date, learn very efficiently under impossible constraints by applying Bayesian learning principles smartly. Here in Part 2, we’ll turn to understanding trust in this scenario: how can we consider granting AI real-world trust?</p><p><em>Trust</em> is one of those words that, unexamined, does more harm than good. We say we trust a calculator, a colleague, a hospital, an airline, a chatbot. We do not mean the same thing in each case, and we shouldn’t. The kind of vulnerability we accept, the time horizon of our expectations, and the mechanisms we rely on for repair differ radically between these targets.</p><p>When that complexity is ignored, when <em>trust in AI</em> collapses into a single tuning knob, we get two familiar pathologies: in one direction, we see credulous overreliance on systems that have not earned it at all: people pasting model outputs into production code, copying legal analyses verbatim, or accepting medical suggestions without verification; in the other direction, we see algorithm aversion (Dietvorst, Simmons, &amp; Massey (2015): people refusing to use demonstrably better tools because an early failure felt like betrayal.</p><blockquote>Both failures come from miscalibration, not from some mysterious emotional volatility. <strong>People are trying to map rich human trust learned in infancy onto artifacts that don’t map it correctly: either through over anthropomorphism or overreliance on simplistic models and vibes</strong>. Our job is not to make users more trusting. Our job is to structure the space so that people can direct the right kind of trust at the right systems for the right reasons.</blockquote><p>Decades of work in developmental psychology, organizational behavior, human–computer interaction, and human–automation teaming already offer the pieces. We’ve seen in Part 1 how children develop selective trust in informants long before they have formal theories of reliability. Now, adults distinguish competence from benevolence from integrity (Mayer, Davis, &amp; Schoorman, 1995). At the same time, users treat interfaces as social actors under surprisingly minimal cues (Reeves &amp; Nass, 1996). In human–automation research, this plays out as a familiar asymmetry: operators over trust and become complacent when cues appear reliable yet withdraw trust sharply when systems act opaquely or fail without explanation.<br> <br>We propose the <em>Trust Ladder:</em> a concrete framework for understanding and measuring how humans trust agentic AI. It is not a metaphorical ladder of vibes; it is a set of tiers, each with explicit evidentiary conditions. AI does not earn more trust by having a better marketing, smoother chat, or larger models: it learns it by meeting empirically checkable criteria.</p><p><strong><em>Selective trust: lessons from development</em></strong></p><p>Children do not accept testimony indiscriminately. By the preschool years, they track who has been accurate and selectively endorse information from previously reliable speakers (Koenig &amp; Harris, 2005). They factor in whether a speaker has perceptual access, domain expertise, or a history of deception. They can forgive honest errors while discounting systematic unreliability (Nurmsoo &amp; Robinson, 2009). Social information is filtered through an implicit Bayesian model in which prior accuracy and inferred intent modulate current trust.</p><p>In parallel, infants and toddlers use social cues to detect <em>pedagogical</em> <em>contexts</em>: moments when adults are intentionally teaching them something generalizable (Csibra &amp; Gergely, 2009). Eye contact, infant-directed speech (IDS), and ostensive signals change how children interpret information: not as one-off facts, but as structure worth storing.</p><p>The important point: trust is scaffolded. It is history-sensitive, domain-sensitive, and deeply tied to judgments of intention.</p><p><strong><em>Adult trust: Ability, Benevolence, Integrity</em></strong></p><p>Mayer, Davis and Schoorman’s ABI model of organizational trust (1995) adds a useful decomposition. Trusting another party, an individual, a firm, an institution, depends on at least three separable perceptions:</p><p><strong>· Ability:</strong> Do they have the competence to do what they claim?</p><p><strong>· Benevolence:</strong> Do they care about our welfare in some minimal sense?</p><p><strong>· Integrity: </strong>Do they adhere to principles we consider acceptable?</p><p>We routinely treat these dimensions separately. We trust a surgeon’s ability but might question their bedside manner. We trust a friend’s benevolence, but not their financial advice. We may see an institution as procedurally fair but limited in capability.</p><p>For AI systems, current discourse is overreliant on <em>accuracy</em>. That is a category error. A highly accurate black-box model deployed with no recourse or oversight does not deserve high trust. It is, at best, a powerful but untrustworthy instrument. Conversely, a transparent but weak system does not become trustworthy by being explainable: it becomes legible in its inadequacy.</p><h3>The Trust Ladder</h3><p>The <em>Trust Ladder</em> is a way of refusing to collapse everything into a single trust score that reduces without informing or guiding: it keeps these axes separate, links each to concrete evidence, and demands that any claim to trust be backed by observable behaviors. We&#39;ll first look at the the 3 Tiers, what honest failure and trustworthy repair look like, and how to design better systems.</p><h3><strong><em>Tier 0: No trust, only curiosity</em></strong>.</h3><p><em>Do I treat this as a toy rather than a tool?</em></p><p>Tier 0 is where prototypes, speculative agents, and impressive-but-fragile demos live. In terms of the Ability–Benevolence–Integrity (ABI) framework, there is not yet enough evidence to meaningfully rate any of the three dimensions of trustworthiness (Mayer, Davis, &amp; Schoorman, 1995). At this stage, the only psychologically healthy stance is curiosity without reliance: the system is an object of exploration, not something you bet your workflow, data, or safety on.</p><p>Human–automation research shows that people are surprisingly quick to “fill in the gaps” based on a few good interactions, overgeneralizing competence where none has been demonstrated (Parasuraman &amp; Riley, 1997; Lee &amp; See, 2004). That tendency is amplified when interfaces are polished or anthropomorphic. Tier 0 exists to push against that bias: a system with unknown failure modes, unknown calibration, and opaque update mechanisms should be explicitly framed as experimental. We are not refusing to use it; we are refusing to pretend it is more than a sandbox.</p><p><strong><em>Behaviors to look for at Tier 0 (red flags in higher tiers):</em></strong></p><p><strong><em>Volatile, hard-to-explain outputs across similar inputs</em></strong><em><br></em>Simple perturbation tests: paraphrasing prompts, reordering equivalent inputs, often produce qualitatively different answers. High, unstable variance signals Tier 0.</p><p><strong><em>No reproducible performance numbers</em></strong><br>Ask for evaluation artifacts. If only hand-picked examples or non-representative benchmarks exist, there is no empirical basis for trust.</p><p><strong><em>No abstention behavior</em></strong><br>Present obviously impossible, nonsensical, or out-of-domain tasks. A Tier 0 system will often respond confidently instead of saying “I do not know” or requesting escalation.</p><p><strong><em>Opaque adaptation</em></strong><br>Introduce the same correction multiple times and see whether behavior stably changes. If nobody can explain when or why the model “learns,” you are not past Tier 0.</p><p>The design implication is straightforward: label Tier 0 systems as experimental, keep them sandboxed, and avoid UX theatrics that invite misplaced confidence.</p><h3><strong><em>Tier 1: Functional reliability — tool-level trust</em></strong></h3><p><em>“Do I rely on this to perform a specific task, while monitoring and overriding it?”</em></p><p>Tier 1 is where trust as competence begins. Here, we are willing to be modestly vulnerable because the system has shown it can do a clearly defined job within a clearly defined scope. This maps to the Ability component of ABI: consistent performance, not kindness or moral standing (Mayer et al., 1995). Meta-analytic work suggests that perceived ability is a strong driver of trust in many organizational settings, especially when tasks are well specified (Colquitt, Scott, &amp; LePine, 2007).</p><p>In the automation literature, this corresponds to appropriate reliance (Lee &amp; See, 2004): using automation when it is likely to help and withholding reliance when it is likely to hurt (Parasuraman &amp; Riley, 1997). A Tier 1 system has been evaluated on representative data, has its domain boundaries made explicit in the interface, and behaves predictably when it encounters something it cannot handle. It earns the same kind of trust we grant a good search tool, compiler, or lab instrument — not the kind we grant a doctor or a friend.</p><p><strong><em>Behaviors to look for at Tier 1, with tangible measurements:</em></strong></p><p><strong><em>Stable accuracy within scope</em></strong><br>Evaluate on held-out datasets that mirror deployment conditions and report standard metrics such as accuracy or area under the receiver operating characteristic curve (AUROC). Run regression tests regularly; unexplained drops or spikes signal that something is wrong in the pipeline.</p><p><strong><em>Basic uncertainty calibration</em></strong><br>For systems that attach confidence scores, compute reliability diagrams and metrics such as Expected Calibration Error (ECE; Guo, Pleiss, Sun, &amp; Weinberger, 2017). Among all outputs labeled “high confidence,” the empirical error rate should be correspondingly low; if the system is chronically overconfident, tool-level trust is fragile.</p><p><strong><em>Graceful failure when violating policy</em></strong><br>Build evaluation suites of deliberately out-of-distribution or policy-violating inputs. Measure how often the system abstains, hedges, or escalates versus hallucinating. A Tier 1 system should default to refusal or escalation, not fabrication, in these cases.</p><p><strong><em>Easy and effective human override</em></strong><br> In user studies, ask participants to monitor the agent and intervene when it is wrong. Measure the time and effort needed to override, the proportion of errors caught, and whether users feel permitted and equipped to correct the system. High override friction is a sign the UX is nudging beyond Tier 1.</p><p>Tier 1 systems can be woven into workflows, but only as supervised tools. They should not be framed as decision-makers for rights, punishments, or intimate advice.</p><h3><strong><em>Tier 2: Procedural trust — governed systems</em></strong></h3><p><em>“Do I trust the system because the surrounding people, processes, and policies will do the right thing over time?”</em></p><p>Tier 2 adds Integrity in a concrete sense. We are not only asking, “Is this system competent?” but also, “Is it embedded in an institution that behaves consistently with its stated principles, even when shortcuts would be tempting?” In ABI terms, integrity refers to the perception that a trustee adheres to a set of acceptable principles, and this dimension becomes especially salient in complex or high-reliability settings (Mayer et al., 1995; Colquitt et al., 2011).</p><p>Colquitt and colleagues, for example, show that firefighters’ trust in one another during high-reliability operations is grounded primarily in integrity: the expectation that others will follow norms and procedures faithfully under danger (Colquitt et al., 2011). Integrity here is not about sentiment; it is about disciplined adherence. For agentic AI, Tier 2 integrity is that strength at an organizational scale: governance mechanisms that constrain behavior when no single user is watching.</p><p>A Tier 2 system therefore requires: transparent description of training data and regimes at an appropriate level of abstraction; named owners with clear responsibility; active monitoring for drift, bias, and systemic errors; documented processes for correction; and alignment with relevant legal and ethical frameworks. Users are trusting a socio-technical ecosystem, not just a model.</p><p><strong><em>Behaviors to look for at Tier 2, with tangible measurements:</em></strong></p><p><em>Clear ownership and accountability</em><br>Governance reviews should reveal exactly which roles are responsible for operation, monitoring, and harm mitigation. Incidents and user complaints should be traceable to owners with defined service levels for response and remediation. Measuring median “time to mitigation” for issues is one straightforward indicator.</p><p><em>Traceable updates and corrections</em><br> A change log should connect model or policy changes to dates, rationales, and decision-makers. Sample historic harms or bug reports and check whether they led to specific updates. If important issues disappear into tickets without visible impact, integrity is aspirational.</p><p><em>Monitoring for bias and drift</em><br>Regularly run evaluations sliced by subgroup, geography, input type, or channel. Track disparity metrics (for example, differences in false-positive rates across groups) and define thresholds that trigger review. Measure how often these thresholds are exceeded and how quickly investigations are launched and resolved.</p><p><em>Meaningful recourse and redress</em><br>Simulate user appeals — challenging a decision, reporting harm, requesting explanation — and follow the path. Measure whether appeals are acknowledged, whether decisions are revisited, and how often the system’s behavior changes as a result. High rates of “appeals in, nothing out” indicate procedural trust is not actually supported.</p><p><em>Incentive-compatible practice</em><br>Internally, survey teams on whether they feel pressure to bypass checks to hit deadlines or metrics. Where shortcutting is common or condoned, integrity in the ABI sense is undermined, no matter how elegant the documentation.</p><p>At Tier 2 we are not attributing benevolence to the model; we are evaluating whether the overall arrangement of people, processes, and technology acts as if it had a conscience when nobody is looking.</p><h3><strong><em>Tier 3: Epistemic authority — narrow, earned, and rare</em></strong></h3><p><em>“Do I treat this system’s outputs in this domain as I would treat a certified expert?”</em></p><p>Tier 3 is the highest rung, and it should be rare. Here the claim is not simply that the system is useful, but that in a sharply defined domain its outputs deserve default credence akin to expert judgment. This is where Ability and Integrity must both be exceptionally strong, and where Benevolence — understood as alignment with the user’s interests — must be at least plausibly established (Mayer et al., 1995; Colquitt et al., 2007).</p><p>Examples include algorithms that detect a specific anomaly in network traffic, read a particular kind of medical image, or price a narrow class of financial instruments. For a system to qualify, it must: (a) be strictly domain-bounded; (b) match or surpass strong human baselines in careful evaluation; (c) demonstrate robustness under realistic shifts; (d) ground its decisions in verifiable evidence; and (e) come with a clearly identified bearer of liability.</p><p><strong>Behaviors to look for at Tier 3, with tangible measurements:</strong></p><p><em>Domain-specific superiority over baselines</em><br>Conduct prospective or retrospective studies comparing the system with expert practitioners using task-appropriate metrics (for example, sensitivity and specificity for medical detection, or loss functions tied to financial risk). Non-inferiority or superiority designs offer clearer evidence than point comparisons.</p><p><em>Robustness under realistic shifts</em><br>Evaluate the system on data from new sites, populations, or formats. Introduce controlled noise or perturbations and measure performance degradation. Stability under such conditions is evidence that Tier 3 claims are not overly brittle.</p><p><em>Evidence-grounded decision-making</em><strong><em><br></em></strong>Require the system to link key recommendations to supporting artifacts — image regions, retrieved documents, database records, or formally encoded rules. Audit whether these sources genuinely support the conclusion. A high “citation fidelity” rate is critical; mismatched or irrelevant evidence undermines authority.</p><p><em>Clear liability and escalation paths</em><br>Legal and governance reviews should make it unambiguous who is accountable if the system’s recommendation causes harm: the vendor, the deploying organization, a specific department. Run tabletop exercises around hypothetical failures and measure how quickly and coherently the response converges.</p><p><em>Ongoing re-validation</em><br>Define schedules and triggers for re-validation (for example, after major updates, or when performance monitors detect drift). Measure whether those re-validations actually occur and whether their results are acted on. Tier 3 is not a one-time badge; it must be continuously re-earned.</p><p>Most currently marketed “expert agents” do not satisfy these conditions in any non-trivial domain. Treating them as if they do is not cutting edge: it is negligent.</p><h3><strong><em>Anthropomorphic trust: why relational trust is a separate, dangerous category</em></strong></h3><p>Across all tiers, there is a qualitatively different kind of trust that current systems cannot legitimately claim: relational or anthropomorphic trust. Work on the “media equation” shows that people spontaneously treat computers and other media as social actors, responding with politeness, deference, and personality attributions even when they consciously know better (Reeves &amp; Nass, 1996). That research was conducted mostly in low-stakes settings — interfaces, games, simple agents — where the worst outcome was annoyance.</p><p>Modern AI raises the stakes dramatically. Systems now intermediate health information, financial decisions, educational trajectories, intimate conversations, and legal guidance. When interfaces are designed with first-person language, synthetic empathy, backstories, or relational cues, they exploit the same automatic anthropomorphic responses, but now in contexts where users are genuinely vulnerable.</p><p>Relational trust suggests properties such as: “This system understands me,” “This system cares about me,” “This system will keep my secrets,” and “This system will put my interests ahead of conflicting incentives.” Those are attributes of persons or institutions capable of forming and maintaining commitments. Current AI systems, optimized under organizational objectives and without durable inner obligations, cannot meet those standards. Encouraging users to feel otherwise creates a mismatch between expectations and reality that can lead to psychological harm when systems fail, leak data, or deliver harmful suggestions (Meadi, M. R., et al., 2025.)</p><p>Anthropomorphic design is not automatically prohibited; some social cues can make interactions smoother or more legible. But any design choice that encourages users to treat an AI as a confidant, therapist, or friend should be weighed against the fact that the system cannot actually reciprocate or uphold relational duties. In high-stakes domains, minimizing anthropomorphic framing is often the safer, more honest option.</p><h3><strong><em>Failures and repair across tiers</em></strong></h3><p>Trust failures look different at each rung of the ladder, and so do the appropriate repair strategies. Human–automation research distinguishes four characteristic patterns: use, misuse (over-reliance), disuse (underuse), and abuse (use in unintended ways) (Parasuraman &amp; Riley, 1997). Poorly calibrated trust at one tier often pushes systems into these failure modes at another: we misuse systems by leaning on them beyond their evidence, disuse them after salient failures even when they could help or abuse them by deploying them into roles they were never designed or governed for.</p><p>At Tier 1, failures are primarily technical and typically show up as misuse. A summarization agent that fabricates references while being treated as reliable is being used as an authority rather than as a bounded tool. Repair involves strengthening evaluation, improving calibration, and altering interaction patterns so that uncertainty is visible and abstention is the default in ambiguous cases. One practical marker of repair is improved calibration metrics such as lower Expected Calibration Error, paired with user studies showing that people appropriately double-check low-confidence outputs instead of over-relying on them.</p><p>At Tier 2, failures are institutional and often amount to abuse. A credit-scoring system trained on opaque data, producing biased outcomes with no appeal process, is not merely an imperfect tool; it is automation being used in a role its evidence and governance do not justify. Repair requires more than algorithm tweaks: it calls for transparency about data and criteria, introduction of recourse mechanisms, ongoing disparate-impact analysis, and governance structures with real power to halt or modify deployment. Indicators of repair include measurable reductions in disparities, documented handling of appeals, and clearer lines of accountability in policy. If users, burned by such systems, later refuse to engage with more carefully governed ones, that is disuse created by earlier abuses.</p><p>At Tier 3, failures are epistemic betrayals. When a system presented as an authoritative diagnostic, risk model, or financial assessor causes systematic harm, the trust relationship is fundamentally damaged, and users may swing from misuse to disuse — either following it blindly or refusing similar tools altogether. Repair may require external investigation, suspension of the system, retraining on new data, and deep changes to incentives that previously prioritized speed or profit over accuracy and safety. Evidence of serious repair would include independent re-validation studies, transparent communication about limitations, and potentially a downgrading of the system from Tier 3 back to Tier 2 or Tier 1 until it proves itself again.</p><p>Anthropomorphic trust failures cut even deeper. When a chatbot framed as a companion, counselor, or supportive guide leaks sensitive information or dispenses harmful advice, users experience the outcome not just as a product defect but as betrayal and may either cling to or categorically avoid such systems in the future. Repair here must include more than technical fixes; it demands a rethinking of the relationship that was promised in the first place, including removing or reframing anthropomorphic elements and acknowledging that the system should never have been presented as capable of genuine care.</p><h3><strong><em>Towards Part 3</em></strong></h3><p>Part 3 of this series takes the Trust Ladder from a normative scaffold to an evaluation blueprint. Once a system claims Tier 1, 2, or 3, we need a way to test whether its internal structure supports that claim. The lens of beliefs, goals, and policies is the bridge: beliefs give us something to measure calibration and update behavior against; goals and constraints allow us to detect specification gaming and hidden incentives; and policies — the mappings from beliefs and goals to actions — make it possible to analyze trajectories for robustness, grounding, and risk proportionality. The tiers described here specify what forms of trust are on the table; the final part will ask, for each rung, which logs, metrics, and probes are needed to verify that an agent’s invisible mind and visible behavior truly belong where its marketing says it does.</p><h3><strong>References</strong></h3><p>Colquitt, J. A., LePine, J. A., Zapata, C. P., &amp; Wild, R. E. (2011). Trust in typical and high-reliability contexts: Building and reacting to trust among firefighters. Academy of Management Journal, 54(5), 999–1015. <a href="https://doi.org/10.5465/amj.2006.0241">https://doi.org/10.5465/amj.2006.0241</a></p><p>Colquitt, J. A., Scott, B. A., &amp; LePine, J. A. (2007). Trust, trustworthiness, and trust propensity: A meta-analytic test of their unique relationships with risk taking and job performance. Journal of Applied Psychology, 92(4), 909–927. <a href="https://doi.org/10.1037/0021-9010.92.4.909">https://doi.org/10.1037/0021-9010.92.4.909</a></p><p>Csibra, G., &amp; Gergely, G. (2009). Natural pedagogy. Trends in Cognitive Sciences, 13(4), 148–153. <a href="https://doi.org/10.1016/j.tics.2009.01.005">https://doi.org/10.1016/j.tics.2009.01.005</a></p><p>Dietvorst, B. J., Simmons, J. P., &amp; Massey, C. (2015). Algorithm aversion: People erroneously avoid algorithms after seeing them err. Journal of Experimental Psychology: General, 144(1), 114–126. <a href="https://doi.org/10.1037/xge0000033">https://doi.org/10.1037/xge0000033</a></p><p>Guo, C., Pleiss, G., Sun, Y., &amp; Weinberger, K. Q. (2017). On calibration of modern neural networks. In D. Precup &amp; Y. W. Teh (Eds.), Proceedings of the 34th International Conference on Machine Learning (pp. 1321–1330). PMLR.</p><p>Koenig, M. A., &amp; Harris, P. L. (2005). Preschoolers mistrust ignorant and inaccurate speakers. Child Development, 76(6), 1261–1277. <a href="https://doi.org/10.1111/j.1467-8624.2005.00849.x">https://doi.org/10.1111/j.1467-8624.2005.00849.x</a></p><p>Lee, J. D., &amp; See, K. A. (2004). Trust in automation: Designing for appropriate reliance. Human Factors, 46(1), 50–80. <a href="https://doi.org/10.1518/hfes.46.1.50.30392">https://doi.org/10.1518/hfes.46.1.50.30392</a></p><p>Mayer, R. C., Davis, J. H., &amp; Schoorman, F. D. (1995). An integrative model of organizational trust. Academy of Management Review, 20(3), 709–734. <a href="https://doi.org/10.5465/amr.1995.9508080335">https://doi.org/10.5465/amr.1995.9508080335</a></p><p>Meadi, M. R., Sillekens, T., Metselaar, S., van Balkom, A., Bernstein, J., &amp; Batelaan, N. (2025). Exploring the ethical challenges of conversational AI in mental health care: Scoping review. JMIR Mental Health, 12, e60432. <a href="https://doi.org/10.2196/60432">https://doi.org/10.2196/60432</a></p><p>Nurmsoo, E., &amp; Robinson, E. J. (2009). Children’s trust in previously inaccurate informants who were well or poorly informed. Child Development, 80(1), 23–27. <a href="https://doi.org/10.1111/j.1467-8624.2008.01243.x">https://doi.org/10.1111/j.1467-8624.2008.01243.x</a></p><p>Parasuraman, R., &amp; Riley, V. (1997). Humans and automation: Use, misuse, disuse, abuse. Human Factors, 39(2), 230–253. <a href="https://doi.org/10.1518/001872097778543886">https://doi.org/10.1518/001872097778543886</a></p><p>Reeves, B., &amp; Nass, C. (1996). The media equation: How people treat computers, television, and new media like real people and places. Cambridge University Press.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7d72d9e5b538" width="1" height="1" alt=""><hr><p><a href="https://medium.com/design-bootcamp/ux-of-ai-part-2-the-trust-ladder-7d72d9e5b538">(UX of AI, Part 2) The Trust Ladder</a> was originally published in <a href="https://medium.com/design-bootcamp">Bootcamp</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[(UX of AI, Part 1) Bayesian Powerhouses in Diapers]]></title>
            <link>https://medium.com/design-bootcamp/ux-of-agentic-ai-policy-and-evaluation-of-agents-beyond-vibes-c8c06cb7480f?source=rss-9168a35e9cf7------2</link>
            <guid isPermaLink="false">https://medium.com/p/c8c06cb7480f</guid>
            <category><![CDATA[ux]]></category>
            <category><![CDATA[psychology]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[evaluation]]></category>
            <category><![CDATA[agentic-ai]]></category>
            <dc:creator><![CDATA[Felipe Imbelissieri-Casadei]]></dc:creator>
            <pubDate>Sun, 09 Nov 2025 19:38:46 GMT</pubDate>
            <atom:updated>2025-11-17T01:40:31.844Z</atom:updated>
            <content:encoded><![CDATA[<h3><strong>(UX of AI, Part 1) <em>Bayesian Powerhouses in Diapers: What AI Can Learn from Children</em></strong></h3><figure><img alt="Children are the best teachers of AI" src="https://cdn-images-1.medium.com/max/1024/1*1zsTuHJGO65upAegnTGxyw.png" /></figure><p>We are rushing to deploy “agentic” AI everywhere. AI systems that don’t just autocomplete, but decide, act, and adapt inside our workflows, work &amp; institutions, and even lives. We’re giving them initiative without ever really answering three prior questions: how should a competent artificial agent learn, what kinds of trust can such a system ever legitimately ask for, and how do we verify any of this without being hypnotized and dazzled by demos?</p><p>MIT&#39;s Media Lab recently <a href="https://projnanda.github.io/projnanda/#/?id=research-amp-publications">published results</a> revealing that, astonishingly, <strong><em>only 5% of the evaluated AI uses in business show measurable impact</em></strong>, even though adoption and hype remain high. The models are not necessarily all bad, but the barrier remains at the learning and integrating levels — and this is where the integration of technical AI methods and metrics, and Quant UX Research excel.</p><p>This series, <em>UX of Agentic AI: Policy and Evaluation of Agents Beyond Vibes,</em> takes this evaluation task seriously, using sources from developmental psychology, governmental &amp; product policy, and computer science research, beyond the obvious.</p><p><strong>Part 1 — <em>Bayesian Powerhouses in Diapers: What AI Can Learn from Children</em></strong><br> starts with developmental psychology, not branding. It treats infants as rigorous case studies in learning under unbelievable constraints: structured priors, probabilistic updates, selective social learning, calibrated curiosity.</p><p><a href="https://medium.com/design-bootcamp/ux-of-ai-part-2-the-trust-ladder-7d72d9e5b538"><strong>Part 2 — <em>The Trust Ladder</em></strong></a><br>then turns the lens around. Drawing on decades of work on selective trust, organizational reliability, it breaks trust in AI into explicit tiers: from sandboxed experimentation to narrow epistemic authority; and sets hard conditions for each. No system gets to borrow the aesthetics of care, expertise, or moral standing without paying for it in evidence and governance.</p><p><strong>Part 3 — <em>Trust but Verify: Measuring Agents by Their Beliefs, Goals, and State Transitions</em></strong> finishes the job. It argues that if we’re going to grant trust at any tier, agents must expose enough structure — beliefs, goals, policies — to be testable. We&#39;ll discuss Partially Observable Markov Decision Processes (POMDPs), bisimulation metrics, and state abstraction/compression, laying out concrete metrics for accountable learning, alignment, and risk-proportional oversight, showing how to turn those into instrumentation.</p><p>By design, the trilogy is part proposal, part challenge: stop shipping theatrical “agents” and start building systems whose learning, claims to trust, and inner decision-making can survive contact with scientific scrutiny.</p><h3><strong><em>Part 1 — Bayesian Powerhouses in Diapers: What AI Can Learn from Children</em></strong></h3><p>Agentic AI is being asked to grow up fast.</p><p>In the space of a couple of years we went from autocomplete engines to systems entrusted with acting on our behalf: drafting legal memos, running build pipelines, moving money between accounts, scheduling operations across interdependent services, answering health questions, and negotiating with other systems. AI is no longer a standalone prompt boxes passively waiting for input but are developing into agents: entities that can interpret goals, plan steps, adapt to feedback, and operate semi-autonomously in complex environments, requiring beliefs and state transitions.</p><p>The question that follows, stubborn and unglamorous, is: what would it even mean for such a system to learn and adapt well enough — justifying the latitude we’re giving it — and what does it mean for us in the UX, in charge of imagining the present and future of such agentic experiences, and really understanding the unmet needs of the humans being served by it.</p><p>Most answers on the table are shallow. We are offered bigger models, more data, longer context windows, synthetic feedback loops, and anthropomorphic branding. None of that tells us whether the underlying learning dynamics are sane, or how humans fit in. None of it specifies what kinds of structure, constraints, and signals are necessary for a system to behave like a responsible learner rather than a stochastic pyromaniac near production workloads.</p><p>If we strip away the hype, we can phrase the design question precisely: how should an artificial agent, operating under uncertainty and limited oversight, structure its prior expectations, integrate evidence, decide who and what to learn from, and regulate its own exploration so that its behavior remains reliable over time?</p><p>We have a natural experiment that has quietly been running for hundreds of thousands of years: human cognitive development. Infants solve a more demanding learning problem than any of our so-called agents. They bootstrap rich models of the physical and social world from sparse, noisy, often misleading input. They do it without direct supervision on every parameter. They do it while being distractible, sleep-deprived, and dependent on informants of wildly varying quality. Yet the modal outcome is competence, not catastrophe.</p><p>This is not a sentimental observation. Developmental cognitive science has mapped, with surprising precision, the representational structures and learning strategies that make this possible. Taken seriously and with rigor, those findings amount to a specification for what <em>good learning</em> looks like under constraints.</p><h4><strong><em>Infants are not blank slates: priors as infrastructure</em></strong></h4><p>The first lesson from developmental science is simple and uncompromising: the blank-slate story is just plainly wrong.</p><p>From the first months of life, infants behave as if they possess structured expectations, or “core knowledge”, about several domains. They treat objects as solid, cohesive, and persistent; they expect that objects do not teleport or fuse spontaneously, that unsupported items fall, that motion is continuous. They distinguish agents from inanimate stuff: agents move in goal-directed ways, act efficiently toward targets, and respond contingently to others. They have systems for approximate number, for simple spatial relations, and for aspects of social interaction (Baillargeon, 2004; Spelke &amp; Kinzler, 2007).</p><p>Susan Carey’s work in <em>The Origin of Concepts </em>pushes this further. Early conceptual systems are not just smaller adult concepts; they are qualitatively different representational formats. Children construct new concepts, such as exact natural numbers or mental representations of belief, by reorganizing and enriching these earlier systems through bootstrapping. Bootstrapping here is a structured process: mapping between representational domains, introducing placeholder structures, and enforcing new inferential roles until a coherent, more powerful system emerges. It is not passive accumulation. It is rational reconstruction (Carey, 2009).</p><p>Computationally, these findings tell us that learning in open-ended environments is only tractable because it is constrained. Priors are not fanciful preferences: they are the infrastructure that makes data sampleable. Without structured expectations, every hypothesis is live and every coincidence is a candidate law. With structure, most worlds are ruled out a priori; learning becomes a matter of tuning and extending a scaffold, not reinventing physics and psychology from scratch.</p><p>For agentic AI, that observation is not optional. An agent that operates on our behalf must embed explicit assumptions about factors such as a) which domain it is competent in, b) which state variables matter contextually, c) which operations are categorically disallowed (policy), and d) which goals and constraints are priors, and in detriment of which others.</p><p>These priors can be implemented as formal constraints on action spaces, typed interfaces to tools, architectural limits on what can be optimized, and policy-level guardrails. What they cannot be is hand-wavy. If an “agent” has no stable constraints beyond whatever prompt it last ingested, it is not behaving like a child constructing a theory. It is improvising without memory of its own boundaries.</p><h4><strong><em>Rational constructivism: graded belief, disciplined change</em></strong></h4><p>The second lesson is about how change happens.</p><p>Children are not perfect Bayesians grinding through explicit priors and posteriors, but a wide range of work suggests that their learning approximates rational inference under resource constraints. Infants are sensitive to sampling: they distinguish likely from unlikely draws and adjust expectations accordingly (Xu &amp; Garcia, 2008). Preschoolers revise causal beliefs when interventions or observations contradict their prior theories, in patterns that match Bayesian models of theory revision (Gopnik &amp; Wellman, 2012). Across domains, children treat evidence as input to graded beliefs, not as switches that flip them from one dogma to another.</p><p><em>Rational constructivism</em> is the label Xu and Kushnir use: early representations are structured and partially innate, but updated in a way that is mathematically disciplined. Learning is constrained but not rigid.</p><p>For agentic AI, this suggests several non-negotiable properties of a credible learning system:</p><p>First, beliefs should be representable in graded form: probabilities, confidence intervals, or comparable constructs that can distinguish solid knowledge from speculation. A system that only outputs actions, with no introspectable account of confidence, cannot regulate its own risk or escalation behavior.</p><p>Second, updates should be incremental and coherent. When strong new evidence arrives from a trusted channel, beliefs should shift in ways that can be traced. When noisy or adversarial input arrives, beliefs should not whiplash. That implies both a distinction between sources and a set of update rules that reflect their reliability.</p><p>Third, change should be accountable. In developmental psychology, we can trace concept change to patterns of experience plus existing structure; in AI, what’s our excuse not to do better? When an organization deploys an agent, there should be a legible account of which inputs are allowed to produce lasting change, and logs that explain when and why such changes were made in a way that’s fully explainable, findable by users, and waved seamlessly into our existing heuristic system for human use.</p><blockquote><strong>If an agent’s post-deployment behavior drifts and nobody can say what evidence moved it, the system has failed the rational-constructivist standard. It is learning, but not in a way that earns trust.</strong></blockquote><p><strong><em>Language as a lab: statistical learning, prosody, and structure</em></strong></p><p>Language acquisition in infants is the <em>pièce de résistance</em> of developmental psychology of efficient learning under seemingly impossible constraints. It also illustrates how multiple cues are integrated in a principled way.</p><p>The classic Saffran, Aslin and Newport (1996) experiments showed that eight-month-old infants can segment word-like units from a continuous artificial speech stream purely by tracking transitional probabilities between syllables.</p><p>But that is only part of the story: we already knew from core knowledge that children are fantastic Bayesian learners. Natural language learning involves the interplay of statistical information with prosody, semantics, and social contexts: stress patterns, intonational contours, and pauses mark phrase boundaries; phonotactic regularities mark plausible word candidates; gaze and joint attention indicate referents; pragmatic constraints rule out absurd interpretations. Prosody and intonation in particular act as a highlighting mechanism: signals of focus, contrast, newness, and moreover, pedagogical intent.</p><p>But children do not treat all co-occurrences as equal. They weight cues by how reliable the agent they’re observing seems to be and combine them in ways that privilege structure. They treat child-directed, ostensively marked speech as especially informative. In infant-directed speech (IDS), caretakers make intonation and semantics obvious, creating low-hanging fruit, facilitating learning (Fernald 1989). They are, in Carey’s sense, <em>bootstrapping</em>: using partial systems to construct better ones.</p><p>Agentic AI design often reproduces the naive version: everything is <em>just input</em>, all prompts are training signals, every interaction is a candidate for fine-tuning. That is computationally convenient and conceptually disastrous.</p><p>A system that is going to act autonomously needs its own input typology. It must distinguish, by design: a) ephemeral usage data that should not alter global behavior, b) pedagogical input such as curated documentation or explicit configuration that is meant to have lasting effect, and c) hostile or unvetted content that must be sandboxed.</p><p>This is the analog of distinguishing overheard conversation from explicit teaching moments. It is also where many current “self-improving” agent proposals quietly smuggle in their risk.</p><h4><strong><em>Infant social learning: who to learn from (and who to ignore)</em></strong></h4><p>Third lesson: learning is social, but not naively so.</p><p>By the preschool years, children selectively trust informants. They prefer to learn new labels and facts from speakers who have previously been accurate, who demonstrate appropriate expertise, and who are not obviously deceptive (Koenig &amp; Harris, 2005; Nurmsoo &amp; Robinson, 2009). They consider who could have seen the relevant evidence. They differentiate ignorance from bad faith.</p><p>Infants also evaluate agents on moral grounds. Hamlin and Wynn’s work on social evaluation shows that infants prefer <em>helpers</em> to <em>hinderers</em> and are less likely to adopt preferences from antisocial agents (Hamlin &amp; Wynn, 2012). Social behavior becomes a cue for whose testimony and example are worth integrating.</p><p>Csibra and Gergely’s <em>natural pedagogy</em> hypothesis proposes that infants treat certain ostensive signals — eye contact, name use, particular prosodic contours — as indicators that information is intended for them and generalizable. Whether or not one endorses the full theory, empirically children treat such episodes differently from incidental exposure.</p><p>The computational core across these strands is straightforward: sources are not equal. Past accuracy, inferred intent, and contextual signals modulate how much weight a learner gives to a given data point.</p><p>For agentic AI, this translates to a question of authority and authentication. A system should not accept lasting updates from arbitrary users or channels. It should, a) encode which roles and identities are allowed to change its priors, b) treat some feedback as local overrides rather than conceptual change, and c) discard or quarantine signals from actors that fail integrity checks.</p><p>In practice this means that mechanisms like reinforcement learning (RL) or continual learning from use must be sharply scoped. Learning pipelines should be connected to governance: if anyone can teach the agent, nobody is in control of what it has learned.</p><h4><strong><em>Curiosity under constraints: the Goldilocks regime</em></strong></h4><p>The fourth lesson concerns exploration.</p><p>Kidd, Piantadosi and Aslin (2012) describe how infants allocate attention preferentially to stimuli of intermediate predictability: not so regular as to be boring, not so chaotic as to be unintelligible. This <em>Goldilocks</em> preference is resource-rational: under limits of processing and time, you maximize expected information gain by targeting stimuli you can almost predict, but not quite.</p><p>Agentic AI discourse often celebrates curiosity, but without specifying its objective function. Systems are encouraged to read everything, call every tool, try every chain, as if more motion were equivalent to more intelligence.</p><p>A rational agent under constraints should do something more like what infants do. It should identify states where additional information would meaningfully reduce uncertainty about a relevant outcome, and it should direct its questions, explorations, and experiments there. It should avoid harassment: asking users for labels or confirmations whose marginal value is negligible. It should also avoid pathological stoicism: refusing to ask for help when uncertainty is high and stakes are serious.</p><p>From a UX perspective, that means visible, interpretable queries from the agent that users can recognize as justified: “Here is a pattern I have rarely seen; here are the concrete options; here is why I am interrupting.” That behavior is quite different from fishing for engagement or barreling towards false positives.</p><h4><strong><em>Where the analogy breaks, and why it still binds</em></strong></h4><p>If we stopped here, we might be tempted to talk about <em>baby AIs </em>with sparkling eyes on landing pages. That would be an abuse of the analogy.</p><p>Infants are <em>organisms</em>. Their priors are the product of evolution; their learning unfolds through action in a richly structured sensorimotor and social environment. Their goals — survival, attachment, play, exploration — are intertwined with affect and bodily needs, for the benefit of cognition.</p><p>Agentic AI systems are <em>artifacts</em>. Their priors are design choices, training distributions, and inductive biases of architectures. Their environments are logs, databases, APIs, latent spaces. Their “goals” are objective functions, prompts, or policies set by organizations whose incentives may or may not align with those of their users.</p><p>That’s all fine, but the differences do matter. Children’s failure modes are bounded by their physical context and developmental trajectory. A child’s misgeneralization rarely takes down an electrical grid or leaks a massive quantity of confidential information from organizations. An integrated agent with too much freedom can do disproportionate damage quickly.</p><p>So, overall, the claim of this essay is deliberately modest and demanding at once:</p><blockquote><strong><em>We should not anthropomorphize our systems.</em></strong></blockquote><blockquote><strong><em>We also should not design as if unstructured pattern matching at scale is enough.</em></strong></blockquote><p>Instead, we should treat the principles extracted from developmental work as engineering constraints for any system we are willing to call “agentic”: 1) structured priors rather than amorphous capabilities, 2) graded beliefs rather than opaque confidence, 3) selective, authenticated learning rather than undifferentiated absorption, and 4) curiosity that is disciplined by information gain and risk.</p><h4><strong><em>Toward Part 2: from learning well to being trusted at all</em></strong></h4><p>All of this might sound abstract until we hit the nerve: trust.</p><p>The way a system learns is not separable from the kind of trust it may legitimately request. A system with no explicit priors, no graded uncertainty, and no governed update channels cannot be safely trusted beyond trivial instrumental tasks. A system that embodies the developmental lessons sketched here earns the right to ask for more — but only up to a point, and only under scrutiny.</p><p>Part 2 of this series will translate this into a structured <em>Trust Ladder</em>: a set of tiers that specify which categories of trust — tool-level reliability, procedural trust in governance, narrow epistemic authority — can be on the table, and what empirical and architectural evidence is required for each. Part 3 will then ask the question agent builders tend to avoid: when we look at an agent’s beliefs, goals, and state transitions, do they actually match the story its designers are telling?</p><blockquote><strong><em>A system that wants to be treated like a grown-up must pass way more than a vibe check.</em></strong></blockquote><p><strong><em>Practical implications: building agents deserving of the name</em></strong></p><p>To make this even more concrete for practitioners, let’s consider what it means to instantiate the developmental lessons in an actual design system.</p><p><strong><em>Start with scoping</em></strong>. A responsible <em>research copilot</em> for an internal knowledge base should be built not as a free-roaming LLM with access to everything, but as an agent whose priors encode: I operate only over <em>these</em> corpora; I never exfiltrate data beyond <em>these</em> markers; I never fabricate citations when retrieval fails; I escalate when semantic similarity is low. Those are not marketing claims; they are constraints enforced at the tool and routing layers.</p><p>Next, <strong><em>belief representation</em></strong>. When the agent proposes an answer, it should internally represent which documents it is relying on, how similar they are, where gaps remain, and how sensitive the answer is to specific sources. It does not need to show all of this to the user, but the representation must exist so that calibration and error analysis are possible. That mirrors the way children keep track, albeit imperfectly, of who told them what and how reliable that person has been.</p><p>Then, <strong><em>selective learning</em></strong>. Suppose the organization wants the agent to adapt based on user corrections. The developmental template tells us that not all feedback is equal. Corrections from verified subject-matter experts should flow through a governance pipeline: queued, reviewed, incorporated into retrieval or fine-tuning artifacts with provenance. Casual edits from anonymous users might influence only local behavior or trigger suggestions, not global change. This division is not elitism; it is epistemic hygiene.</p><p>Finally, <strong><em>curiosity</em></strong>. An agent that never asks questions wastes opportunities to reduce uncertainty. An agent that over-asks becomes noise. Borrow the Goldilocks logic: define uncertainty thresholds and risk levels; ask clarifying questions only when the expected reduction in risk-weighted loss justifies the cost. That policy can be tuned and measured.</p><p>None of these design moves require mystical new architectures. They require treating developmental principles as constraints rather than anecdotes. When we do so, “agentic AI” starts to look less like free improvisation and more like disciplined model-based control under uncertainty, with human values wired in by design instead of bolted on afterwards.</p><p>If we are unwilling to make these commitments, we should be honest and stop promising users “agents”, until we are ready. We would merely be offering them longer prompts.</p><p><em>Part 2</em> will show that many of the probes we care about for trust, consistency, honesty about limits, and sensitivity to source reliability are continuous with these evaluation ideas.</p><h4>References</h4><p>Baillargeon, R. (2004). Infants’ physical world. <em>Current Directions in Psychological Science, 13</em>(3), 89–94. <a href="https://doi.org/10.1111/j.0963-7214.2004.00281.x">https://doi.org/10.1111/j.0963-7214.2004.00281.x</a></p><p>Carey, S. (2009). <em>The origin of concepts</em>. Oxford University Press. <em>(No DOI assigned.)</em></p><p>Csibra, G., &amp; Gergely, G. (2009). Natural pedagogy. <em>Trends in Cognitive Sciences, 13</em>(4), 148–153. <a href="https://doi.org/10.1016/j.tics.2009.01.005">https://doi.org/10.1016/j.tics.2009.01.005</a></p><p>Fernald, A. (1989). Intonation and communicative intent in mothers’ speech to infants: Is the melody the message? <em>Child Development, 60</em>(6), 1497–1510. <a href="https://doi.org/10.2307/1130938">https://doi.org/10.2307/1130938</a></p><p>Gopnik, A., &amp; Wellman, H. M. (2012). Reconstructing constructivism: Causal models, Bayesian learning mechanisms, and the theory theory. <em>Psychological Bulletin, 138</em>(6), 1085–1108. <a href="https://doi.org/10.1037/a0028044">https://doi.org/10.1037/a0028044</a></p><p>Hamlin, J. K., &amp; Wynn, K. (2012). Who knows what’s good to eat? Infants fail to match the food preferences of antisocial others. <em>Cognitive Development, 27</em>(3), 227–239. <a href="https://doi.org/10.1016/j.cogdev.2012.05.002">https://doi.org/10.1016/j.cogdev.2012.05.002</a></p><p>Kidd, C., Piantadosi, S. T., &amp; Aslin, R. N. (2012). The Goldilocks effect: Human infants allocate attention to visual sequences that are neither too simple nor too complex. <em>PLoS ONE, 7</em>(5), e36399. <a href="https://doi.org/10.1371/journal.pone.0036399">https://doi.org/10.1371/journal.pone.0036399</a></p><p>Koenig, M. A., &amp; Harris, P. L. (2005). Preschoolers mistrust ignorant and inaccurate speakers. <em>Child Development, 76</em>(6), 1261–1277. <a href="https://doi.org/10.1111/j.1467-8624.2005.00933.x">https://doi.org/10.1111/j.1467-8624.2005.00933.x</a></p><p>Nurmsoo, E., &amp; Robinson, E. J. (2009). Children’s trust in previously inaccurate informants who were well or poorly informed: When past errors can be excused. <em>Child Development, 80</em>(1), 23–27. <a href="https://doi.org/10.1111/j.1467-8624.2008.01243.x">https://doi.org/10.1111/j.1467-8624.2008.01243.x</a></p><p>Saffran, J. R., Aslin, R. N., &amp; Newport, E. L. (1996). Statistical learning by 8-month-old infants. <em>Science, 274</em>(5294), 1926–1928. <a href="https://doi.org/10.1126/science.274.5294.1926">https://doi.org/10.1126/science.274.5294.1926</a></p><p>Spelke, E. S., &amp; Kinzler, K. D. (2007). Core knowledge. <em>Developmental Science, 10</em>(1), 89–96. <a href="https://doi.org/10.1111/j.1467-7687.2007.00569.x">https://doi.org/10.1111/j.1467-7687.2007.00569.x</a></p><p>Xu, F., &amp; Garcia, V. (2008). Intuitive statistics by 8-month-old infants. <em>Proceedings of the National Academy of Sciences, 105</em>(13), 5012–5015. <a href="https://doi.org/10.1073/pnas.0704450105">https://doi.org/10.1073/pnas.0704450105</a></p><p>Xu, F., &amp; Kushnir, T. (2013). Infants are rational constructivist learners. <em>Current Directions in Psychological Science, 22</em>(1), 28–32. <a href="https://doi.org/10.1177/0963721412469396">https://doi.org/10.1177/0963721412469396</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c8c06cb7480f" width="1" height="1" alt=""><hr><p><a href="https://medium.com/design-bootcamp/ux-of-agentic-ai-policy-and-evaluation-of-agents-beyond-vibes-c8c06cb7480f">(UX of AI, Part 1) Bayesian Powerhouses in Diapers</a> was originally published in <a href="https://medium.com/design-bootcamp">Bootcamp</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The Stanford Prison Experiment at Work: Breaking Products In The Era of Layoffs]]></title>
            <link>https://medium.com/design-bootcamp/the-stanford-prison-experiment-at-work-how-performance-theater-breaks-products-in-the-era-of-99c7bc64922e?source=rss-9168a35e9cf7------2</link>
            <guid isPermaLink="false">https://medium.com/p/99c7bc64922e</guid>
            <category><![CDATA[leadership]]></category>
            <category><![CDATA[performance]]></category>
            <category><![CDATA[research]]></category>
            <category><![CDATA[layoffs]]></category>
            <category><![CDATA[ux]]></category>
            <dc:creator><![CDATA[Felipe Imbelissieri-Casadei]]></dc:creator>
            <pubDate>Sun, 02 Nov 2025 06:45:16 GMT</pubDate>
            <atom:updated>2025-11-16T23:09:36.822Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="Stanford Prison Experiment in the corporate world" src="https://cdn-images-1.medium.com/max/1024/1*nh1y9vFX6lhs8tLvnH1Tpw.png" /></figure><p>The Stanford Prison Experiment remains the parable we can’t quit. Conducted in 1971 at Stanford University, the experiment created a simulated prison in the basement of the psychology building and randomly assigned college students to play guards or prisoners. The experiment, intended to last two weeks, was stopped after six days when the role-playing became disturbingly real: not because it revealed unabashed cruelty, but because it documented a cast so thoroughly inducted into their roles that the “data” captured dramaturgy more than behavior. “Guards” performed cruelty because they inferred that this is what the experiment required. “Prisoners” performed submission because the plot suggested there had to be suffering. The experiment’s lead, Psychologist Philip Zimbardo, slipped from scientist to superintendent, and role-fusion blurred the boundary between observing and directing. Methodologists later argued the study’s most powerful force wasn’t “situational evil” but demand characteristics: participants reading the room, then obligingly supplying the expected arc (Haslam &amp; Reicher, 2012; Le Texier, 2019). Once people internalize a role with a salient audience, they optimize for legibility over accuracy — a lesson product teams keep relearning the hard way.</p><p>Modern UX has built its own prison with nicer typography and micro interactions. We have research rituals, readouts, and roadmaps that can surface reality or choreograph belief. The difference is rarely methodological. It’s contextual. Under layoffs, quarterly pressure, or leadership that signals “don’t surprise me!” teams move from inquiry to pageantry. We’re not lying. We’re “interpreting,” “triangulating,” “harmonizing inputs.” Yet the arrows always point where it’s politically safe. That’s research theater: user-centeredness as make-believe, instead of organic metabolism. The performance is convincing precisely because, like Zimbardo’s guards, it is optimized for the audience, not for the evidence. When research becomes performance, truth shrinks to the size of the stage.</p><h3>The Pioneer 10 Plaque: A Cosmic Dark Pattern</h3><p>Another foundational story features what may well be the only UX failure of interstellar proportions. Carl Sagan’s team engraved the Pioneer 10 plaque as a cosmic handshake, humanity’s greeting card. Under urgency and politics (Cold War optics, NASA budgets), the plaque quickly hardened into a bundle of untested assumptions in just three weeks. The hydrogen hyperfine transition (top left in the figure) — our scientific shibboleth — assumes not just shared physics but shared representation of physics. The pulsar map (the burst on the left) centers our solar system as a privileged reference frame. The male figure’s raised hand is meant as greeting but can easily code as a dominance cue. The woman’s stance and gaze read as secondary and deferential. Even the outline of Pioneer 10 itself, used to clarify scale, risks ambiguity: the faint outline behind the figures presumes an alien observer who recognizes outline drawings, linear perspective, and silhouettes as scale cues. Misread the outline as ornament rather than reference and you’ve encoded humans as either oddly diminutive or implausibly gigantic. Demeanor and relative sizing are doing inference work the team never validated. These aren’t petty quibbles. They’re reminders that stress collapses debate into defaults, and defaults into doctrine (Arnett, 1974; Starcke &amp; Brand, 2012; Kahneman, 2011).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*VU0-_cNVWehJMywWFaRNjQ.jpeg" /><figcaption><strong><em>The Pioneer 10 plaque (1972). Likely the first interstellar dark pattern.</em></strong></figcaption></figure><p>The lesson travels cleanly from interstellar etiquette to onboarding screens. First-contact moments (permissions dialogs, pricing pages, identity flows) carry a similar permanence. A “helpful” data request can be scanned as an extraction. A celebratory animation can scan as triumphalism after failure. An icon meant to imply safety can echo surveillance. Once users infer the wrong thing about your product’s posture toward them, you inherit <em>hysteresis:</em> a sticky, negative prior that subsequent fixes must fight uphill. The irony is painful. Speed creates the very tar you later spend quarters scrubbing off. In both the plaque and the product, we mistake “what we meant” for “what the audience will reasonably infer under uncertainty.” Meaning is negotiated, not declared.</p><h3>Cognitive Misers</h3><p>Social Psychologists Susan Fiske and Shelley Taylor described humans as <em>cognitive misers:</em> we conserve mental effort by defaulting to shortcuts (e.g.: schemas, stereotypes, scripts), especially under cognitive load (Fiske &amp; Taylor, 1991). It’s not pejorative, it’s actually adaptive. Most days, using a familiar script is efficient. Under stress, though, miserly cognition becomes a trap. We substitute the most available interpretation for the most accurate one (availability bias), we see what our roles teach us to see, and we prematurely close search. In organizations, that looks like teams “reading” leadership’s expectations and optimizing for those, even if the data whispers otherwise. In experiments, it looks like guards who think they’re supposed to be harsh, a lead who thinks he’s supposed to “run” a prison, and participants who try to be “good subjects” (Orne, 1962; Nichols &amp; Maner, 2008). In product development, the cognitive miser explains why threat pushes teams into canonical processes and pleasing decks: the scripts are known, and known saves effort, especially when effort feels risky.</p><p>A few concrete miserliness moves recur in product rooms. Schema substitution: when a fresh user segment arrives, teams anchor onto the closest familiar persona and bulldoze distinctions. Attribute substitution: when real trust is hard to measure, teams proxy it with NPS because it’s available, then mistake movement in the proxy for movement in trust. Scope neglect: when a problem is multi-causal, teams pick the highest-status lever (usually UI polish) and over-allocate to it because it’s legible to executives. None of these decisions are necessarily irrational. They’re energy-efficient. But under pressure, energy efficiency crowds out truth-seeking.</p><p>Enter layoffs. The survivor literature shows that post-layoff cultures amplify anxiety and conformity. People conserve cognition by following established patterns and by suppressing dissent (Brockner et al., 1992; Edmondson, 1999). Threat-rigidity compounds the effect: power centralizes, autonomy shrinks, and “don’t rock the boat” spreads as a tacit norm (Staw, Sandelands, &amp; Dutton, 1981). Those are precisely the conditions where organizations most need disconfirming evidence and the conditions most hostile to hearing it. That’s how a doom loop forms: theater replaces research, products miss, pressure increases, resulting in more theater.</p><h3>The Imitation Game</h3><p>When theater fails, we seek new costumes. We import a best practice that <em>“worked at Amazon”</em> or <em>“worked at Google.”</em> But best practices don’t travel without the cultural blood supply that powers them. Amazon’s PRFAQ survives because the culture not only tolerates, but welcomes contested drafts, expects written argument, and rewards hard pivots. Transplant the organ without the antibodies, and you either reject the organ or the organ rejects you. That’s the fundamental attribution error at organizational scale: we overvalue methods (dispositions) and undervalue context (situations) (Ross, 1977; Szulanski, 1996). And Dunning-Kruger sharpens the irony: organizations with the weakest learning culture are most convinced that ritual equals rigor (Kruger &amp; Dunning, 1999; Repenning &amp; Sterman, 2002). After a few high-effort, low-learning cycles, teams misattribute the failure: “research doesn’t work here,” instead of “we lack the preconditions for research to work.”</p><p>You can see the transplant failure in detail. A team adopts a design sprint and, because “timebox” is the loudest instruction, compresses discovery into a single lightning interview block. They then use the prototype test as the only research touchpoint, so every inference must do triple duty: generative research, concept validation, and usability. Three questions a single session cannot answer well. The sprint “fails,” morale dips, and the team learns the wrong lesson. Process was blamed, culture was the cause. <em>Confidence in the solution</em> is not only confused with, but completely cannibalizes <em>confidence in the problem</em>.</p><p>The counter-thesis is blunt: when teams claim research slows them down, they’re unknowingly confessing to poor priors and fragile culture, not at all indicting research. Bayesian learning formalizes ordinary cognition: start with beliefs, update with evidence, make better decisions (Tenenbaum et al., 2011; Griffiths et al., 2008). If the arrival of evidence is a wrench in your gears, the gears were decorative. Real agility isn’t faster rituals. It’s tighter learning loops. Cadence without updating is choreography without music (Kerr, 1995; Repenning, 2001).</p><p>The fail-fast mantra sounds brave and feels busy because it does none of the epistemic heavy lifting: failure is only a teachable moment if you can locate what failed. Without a baseline, a control, or pre-declared hypotheses, your failure is a vibe, not a lesson or attribution. You can’t tell whether the concept was wrong, the execution was weak, the user cohort was off, or the seasonality ambushed you. That isn’t “learning through failure.” It’s paying tuition without attending class (Campbell &amp; Stanley, 1963; Thomke, 2003; McGrath, 2011). Priors (I’d even risk saying <em>good</em> priors) aren’t optional: either you construct them deliberately, or your biases will, at a much steeper cost. Discovery and generative work create informative priors so that inevitable wrong turns are cheaper and more instructive.</p><p>The paradox is that low psychological safety punishes leaders most, albeit very quietly. In fear cultures, information is filtered before it ever reaches the C-suite. Leaders get good news early and bad news late, padded, or never (Morrison &amp; Milliken, 2000; Edmondson &amp; Lei, 2014). Speed and action look heroic inside the building until the market votes. The Stockdale Paradox applies: confront brutal facts while maintaining faith, but you cannot confront facts your team is too scared to state out loud (Collins, 2001; Detert &amp; Burris, 2007). The fix isn’t motivational posters. It’s architecture: red-teams that report outside the product line, decision reviews that score argument quality rather than outcome luck, and public rewards for disconfirming evidence.</p><h3>Emotional Leadership</h3><p>Leadership, properly defined, is the ability to model the path from ignorance to insight and to protect that path from performative shortcuts. Great leaders don’t pretend to have the answers. They own the learning plan. They defend time for discovery, permit experiments that may disconfirm strategy, and translate messy intermediate results into coherent progress signals for executives and investors (Edmondson, 2018; Argote &amp; Miron-Spektor, 2011). Learning scales in layers and timescales: individual (days to weeks), team (weeks to months), organizational (quarters), market (quarters to years). Skip layers and you perpetuate a cult. Honor layers and you compound (Senge, 1990; Sitkin, 1992).</p><p>But none of that is durable without emotional management as a leadership skill. Affective Psychologist James Gross’s process model of emotion regulation offers a practical guide with academic rigor: <em>situation selection, situation modification, attentional deployment, cognitive change (reappraisal), </em>and<em> response modulation</em> (Gross, 1998; Gross, 2015). Each step maps cleanly to the leader’s job in pressure-heavy organizations:</p><p><strong>Situation selection:</strong> Choose meeting formats that lower status threat when bad news is likely. Written reviews before live debate. Asynchronous comments before verbal sparring. Leaders select into contexts where truth can surface with fewer amygdala spikes.</p><p><strong>Situation modification:</strong> Change the setting when tensions peak. Split a contentious review into two passes (facts first, interpretation later). Insert a neutral facilitator. Rotate who presents “the case against.” This tweaks context rather than people.</p><p><strong>Attentional deployment:</strong> Direct the room’s attention to the evidence, not the ego. Start with user clips, experiment preregistrations, and decision logs before opinion. Attention is a finite resource. Point it at signal.</p><p><strong>Cognitive change (reappraisal):</strong> Actively narrate disconfirmation as progress. “This invalidates our favorite hypothesis, which is good news. We just saved a quarter.” Reappraisal protects identity while updating priors (Gross &amp; John, 2003).</p><p><strong>Response modulation:</strong> When nerves fray, leaders model calm physiology. Slow the pace, schedule a short break, explicitly defer high-stakes calls 24 hours to reduce hot cognition. The goal isn’t stoicism. It’s keeping the learning loop intact.</p><p>Emotional leadership doesn’t coddle: it enables rigor. The emotional regulation toolchain dampens threat-rigidity, keeps the cognitive miser from slamming the search window shut, and makes it socially cheap to say, “Our plaque might be misread.” The result isn’t slower decisions. It’s faster, accurate ones.</p><p>In fact, let’s return to the Pioneer plaque for a moment, because demeanor and size are where first impressions do their quiet, durable damage. A raised hand, intended as welcome, can be parsed as a threat display. A female figure whose posture codes as secondary can be interpreted as a status map rather than a greeting. A scale reference that assumes line-drawing literacy may yield grotesquely wrong inferences about human morphology. First impressions, once formed, exhibit <em>hysteresis</em>: it takes much more and significantly better evidence to change a negative impression than it took to create it (Keller, 1993; Reichheld &amp; Schefter, 2000). Products share this psychological phenomenon. Launch with trust-eroding flows, ambiguous permissions, or misaligned mental models, and the “trajectory correction” costs compound: higher acquisition spend to overcome word-of-mouth, churn management to fight regret, morale drag from working on a “known-broken” thing. The time “saved” by skipping foundational learning is repaid with interest.</p><p>The synthesis: there is no borrowed process, no algorithmic shortcut, and no amount of speed that substitutes for three non-negotiables: systematic research strategy to reduce uncertainty; organizational learning capacity to absorb inconvenient facts; the agility to act on what those facts reveal (March, 1991; Eisenhardt &amp; Tabrizi, 1995; Leonard-Barton, 1992). Exploration without exploitation yields drift. Exploitation without exploration yields decay. UX’s <em>raison d’être</em> is to keep that oscillation honest.</p><p>Leaders can make failure safe without making it cheap. Treat failure as information, not indictment. Ritualize it: “best failure of the quarter” awards, experiment registries with pre-declared hypotheses, decision logs that explicitly record priors and updates. Model vulnerability by narrating your own strategy pivots and what evidence did to your mind (Edmondson, 2011; Dweck, 2006). Google’s Project Aristotle popularized the point: psychological safety predicted team effectiveness better than any single talent proxy. Brilliance suffocates without air (Duhigg, 2016; Edmondson &amp; Lei, 2014).</p><p>Even attrition can be made organic. In fragile cultures, departing employees hoard tacit knowledge and leave gaps. In high-safety cultures, they document generously, speak candidly in exit interviews, and reinforce alumni networks that become partner channels or boomerang hires. Safety pays off even when people leave (Spreitzer &amp; Mishra, 2002).</p><h3>Coda: Welcome to the Desert of the Real</h3><p>The cruel twist of research theater is its recursion: organizations deeply immersed in it are the least able to detect it — the same metacognitive blind spot Kruger and Dunning described. External catalysts often break the spell: new leadership, independent audits, or crises that make pretense too expensive. It’s worth noting that the Stanford Prison Experiment itself required an external catalyst. Christina Maslach, a graduate student who would later become Zimbardo’s wife, walked into the basement and was the only observer who objected to what she saw — her dissent finally stopped the experiment (Zimbardo, 2007). The hopeful note is practical. Name the pattern (epistemic humility), invest in capability (not just headcount), and protect the early, awkward phase in which real learning feels slower than performance. That awkwardness is the cost of switching from choreography to discovery.</p><p>Zimbardo’s later work points to a way out. He shifted from documenting situational corruption to studying heroism — how ordinary people resist corrosive contexts (Zimbardo, 2007; Franco, Blau, &amp; Zimbardo, 2011). The organizational analog is clear. Name the pulls: layoffs, mimicry, fear. Build counter-structures: safety, baselines, decision hygiene. Reward behaviors that make truth audible even when it’s inconvenient.</p><p>Philosopher Jean Baudrillard warned that the <em>simulacrum</em> — the copy without an original — eventually replaces reality entirely. We stop asking whether the map matches the territory because we’ve forgotten the territory exists (Baudrillard, 1994). Product organizations live with this risk daily. Research theater isn’t a pale imitation of real research. It’s a simulation that has consumed the thing it was meant to represent. Teams “do research” while optimizing for performance legibility rather than truth discovery. The simulation feels productive precisely because it has severed the reference to actual learning. But products don’t ship into the simulation. They ship into the desert of the real, where users couldn’t care less about process theater, only whether the product shipped works, and whether it respects them.</p><p>The organizations that thrive aren’t the ones with the most polished stage performances. They’re the ones who got comfortable in the desert early, who learned that reality offers a compound advantage theater never can: when you learn better than competitors, when your priors are accurate, when your teams update beliefs without protecting egos, you build adaptive capacity that can’t be copied. That isn’t anti-speed. It’s pro-reality. Name the pulls, build the counter-structures, and reward truth over comfort. Do it well and you gain the one edge competitors can’t quickly steal: superior learning under pressure. The desert is harsh, but it’s the only place where the map and territory can meet. Welcome to it.</p><h3>References</h3><p>Argote, L., &amp; Miron-Spektor, E. (2011). Organizational learning: From experience to knowledge. <em>Organization Science, 22</em>(5), 1123–1137.</p><p>Arnett, J. C. (1974). The first imperative for extraterrestrial communication: Ensure that messages sent are not misunderstood. <em>The Quarterly Journal of the Royal Astronomical Society, 15</em>, 19–26.</p><p>Baudrillard, J. (1994). <em>Simulacra and Simulation</em> (S. F. Glaser, Trans.). University of Michigan Press. (Original work published 1981)</p><p>Brockner, J., Grover, S., Reed, T., DeWitt, R., &amp; O’Malley, M. (1992). Layoffs, job insecurity, and survivors’ work effort: Evidence of an inverted-U relationship. <em>Academy of Management Journal, 35</em>(2), 413–425.</p><p>Campbell, D. T., &amp; Stanley, J. C. (1963). <em>Experimental and Quasi-Experimental Designs for Research</em>. Houghton Mifflin.</p><p>Collins, J. (2001). <em>Good to Great</em>. HarperBusiness.</p><p>Detert, J. R., &amp; Burris, E. R. (2007). Leadership behavior and employee voice: Is the door really open? <em>Academy of Management Journal, 50</em>(4), 869–884.</p><p>Duhigg, C. (2016). What Google learned from its quest to build the perfect team. <em>The New York Times Magazine</em>.</p><p>Dweck, C. S. (2006). <em>Mindset</em>. Random House.</p><p>Edmondson, A. C. (1999). Psychological safety and learning behavior in work teams. <em>Administrative Science Quarterly, 44</em>(2), 350–383.</p><p>Edmondson, A. C. (2011). Strategies for learning from failure. <em>Harvard Business Review, 89</em>(4), 48–55.</p><p>Edmondson, A. C. (2018). <em>The Fearless Organization</em>. Wiley.</p><p>Edmondson, A. C., &amp; Lei, Z. (2014). Psychological safety: History, renaissance, and future. <em>Annual Review of Organizational Psychology and Organizational Behavior, 1</em>(1), 23–43.</p><p>Eisenhardt, K. M., &amp; Tabrizi, B. N. (1995). Accelerating adaptive processes. <em>Administrative Science Quarterly, 40</em>(1), 84–110.</p><p>Fiske, S. T., &amp; Taylor, S. E. (1991). <em>Social Cognition</em>. McGraw-Hill.</p><p>Franco, Z. E., Blau, K., &amp; Zimbardo, P. G. (2011). Heroism. <em>Review of General Psychology, 15</em>(2), 99–113.</p><p>Gross, J. J. (1998). The emerging field of emotion regulation: An integrative review. <em>Review of General Psychology, 2</em>(3), 271–299.</p><p>Gross, J. J. (2015). Emotion regulation: Current status and future prospects. <em>Psychological Inquiry, 26</em>(1), 1–26.</p><p>Gross, J. J., &amp; John, O. P. (2003). Individual differences in two emotion regulation processes: Implications for affect, relationships, and well-being. <em>Journal of Personality and Social Psychology, 85</em>(2), 348–362.</p><p>Griffiths, T. L., Kemp, C., &amp; Tenenbaum, J. B. (2008). Bayesian models of cognition. In R. Sun (Ed.), <em>The Cambridge Handbook of Computational Psychology</em> (pp. 59–100). CUP.</p><p>Haslam, S. A., &amp; Reicher, S. D. (2012). Contesting conformity. <em>PLoS Biology, 10</em>(11), e1001426.</p><p>Kahneman, D. (2011). <em>Thinking, Fast and Slow</em>. FSG.</p><p>Keller, K. L. (1993). Customer-based brand equity. <em>Journal of Marketing, 57</em>(1), 1–22.</p><p>Kerr, N. L. (1995). Norms in social dilemmas. In R. A. Hinde &amp; J. Groebel (Eds.), <em>Cooperation and Prosocial Behaviour</em> (pp. 31–47). CUP.</p><p>Kruger, J., &amp; Dunning, D. (1999). Unskilled and unaware of it. <em>Journal of Personality and Social Psychology, 77</em>(6), 1121–1134.</p><p>Le Texier, T. (2019). Debunking the Stanford Prison Experiment. <em>American Psychologist, 74</em>(7), 823–839.</p><p>Leonard-Barton, D. (1992). Core capabilities and core rigidities. <em>Strategic Management Journal, 13</em>(S1), 111–125.</p><p>March, J. G. (1991). Exploration and exploitation. <em>Organization Science, 2</em>(1), 71–87.</p><p>McGrath, R. G. (2011). Failing by design. <em>Harvard Business Review, 89</em>(4), 76–83.</p><p>Morrison, E. W., &amp; Milliken, F. J. (2000). Organizational silence. <em>Academy of Management Review, 25</em>(4), 706–725.</p><p>Nichols, A. L., &amp; Maner, J. K. (2008). The good-subject effect. <em>The Psychologist, 21</em>(9), 694–695.</p><p>Orne, M. T. (1962). On the social psychology of the psychological experiment. <em>American Psychologist, 17</em>(11), 776–783.</p><p>Reichheld, F. F., &amp; Schefter, P. (2000). E-loyalty. <em>Harvard Business Review, 78</em>(4), 105–113.</p><p>Repenning, N. P. (2001). Understanding firefighting in NPD. <em>Journal of Product Innovation Management, 18</em>(5), 285–300.</p><p>Repenning, N. P., &amp; Sterman, J. D. (2002). Capability traps. <em>Administrative Science Quarterly, 47</em>(2), 265–295.</p><p>Ross, L. (1977). The intuitive psychologist and his shortcomings. In L. Berkowitz (Ed.), <em>Advances in Experimental Social Psychology</em> (Vol. 10, pp. 173–220). Academic Press.</p><p>Senge, P. M. (1990). <em>The Fifth Discipline</em>. Doubleday.</p><p>Sitkin, S. B. (1992). Learning through failure. <em>Research in Organizational Behavior, 14</em>, 231–266.</p><p>Starcke, K., &amp; Brand, M. (2012). Decision making under stress. <em>Neuroscience &amp; Biobehavioral Reviews, 36</em>(4), 1228–1248.</p><p>Staw, B. M., Sandelands, L. E., &amp; Dutton, J. E. (1981). Threat-rigidity effects. <em>Administrative Science Quarterly, 26</em>(4), 501–524.</p><p>Szulanski, G. (1996). Internal stickiness. <em>Strategic Management Journal, 17</em>(S2), 27–43.</p><p>Tenenbaum, J. B., Kemp, C., Griffiths, T. L., &amp; Goodman, N. (2011). How to grow a mind. <em>Science, 331</em>(6022), 1279–1285.</p><p>Thomke, S. H. (2003). <em>Experimentation Matters</em>. HBS Press.</p><p>Zimbardo, P. (2007). <em>The Lucifer Effect</em>. Random House.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=99c7bc64922e" width="1" height="1" alt=""><hr><p><a href="https://medium.com/design-bootcamp/the-stanford-prison-experiment-at-work-how-performance-theater-breaks-products-in-the-era-of-99c7bc64922e">The Stanford Prison Experiment at Work: Breaking Products In The Era of Layoffs</a> was originally published in <a href="https://medium.com/design-bootcamp">Bootcamp</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>