<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Julian Roberts on Medium]]></title>
        <description><![CDATA[Stories by Julian Roberts on Medium]]></description>
        <link>https://medium.com/@julian.g.roberts?source=rss-766ccc1bd541------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*Hw0ozKLNuSl9UUN-</url>
            <title>Stories by Julian Roberts on Medium</title>
            <link>https://medium.com/@julian.g.roberts?source=rss-766ccc1bd541------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Tue, 17 Nov 2026 13:10:54 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@julian.g.roberts/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[The Ground Truth Problem: Why Consciousness Research Belongs at the Foundation of AI Safety]]></title>
            <link>https://medium.com/@julian.g.roberts/the-ground-truth-problem-why-consciousness-research-belongs-at-the-foundation-of-ai-safety-2bdc3f70f8ec?source=rss-766ccc1bd541------2</link>
            <guid isPermaLink="false">https://medium.com/p/2bdc3f70f8ec</guid>
            <category><![CDATA[philosophy-of-mind]]></category>
            <category><![CDATA[ai-safety]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[cognitive-science]]></category>
            <dc:creator><![CDATA[Julian Roberts]]></dc:creator>
            <pubDate>Sat, 26 Sep 2026 03:39:06 GMT</pubDate>
            <atom:updated>2026-09-26T03:39:06.528Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="A glass cube etched with circuit traces and mathematical symbols, dissolving on one side into a glowing, organic amber form." src="https://cdn-images-1.medium.com/max/1024/1*yF4fiGC-CZETthjb6Dvm3w.jpeg" /></figure><h4>From Simulated Verification to Ontological Necessity in Artificial Intelligence</h4><p><em>As a developer actively building autonomous agent architectures and AI workflows, I regularly wire up the persistent memory structures and execution loops discussed in AI safety circles. This essay bridges the gap between those production engineering realities and the philosophical questions we can no longer ignore.</em></p><h3>Abstract</h3><p>AI developers now spend hundreds of billions of dollars a year on compute and infrastructure, while the science of consciousness receives a tiny fraction of that. This essay argues the imbalance is a mistake, but for a narrower reason than is often claimed. Many AI risks, such as misaligned goal pursuit, deception and misuse, do not require an AI to be conscious, and must be addressed regardless.</p><p>Consciousness research matters for two other reasons. First, if advanced systems can have experiences, then how we train, constrain and deploy them becomes an ethical question as well as an engineering one. Second, our leading theories of consciousness disagree sharply about whether current or near-future architectures could be conscious, and those disagreements are increasingly testable.</p><p>We trace the problem from the verification gap (Section 1), through the architecture of current language models (2) and persistent agentic systems (3), to Donald Hoffman’s conscious realism (4). We then set out a decision matrix across metaphysical positions (5) and propose a research agenda (6). Our conclusion: consciousness science is not a substitute for alignment research, but it is a neglected prerequisite for developing advanced AI responsibly.</p><h3>1. The verification problem</h3><p>We have no agreed method for detecting subjective experience from the outside. This is the classic problem of other minds, and AI makes it urgent.</p><p><strong>The problem of other minds.</strong> We attribute experience to other humans largely by analogy: they share our biology and behave as we do. With AI systems, the behavioural analogy remains while the biological one disappears. Whether there is “something it is like” to be such a system (Nagel, 1974) cannot be read off its outputs alone.</p><p><strong>Behavioural mimicry.</strong> A system trained on human text will produce fluent reports of feelings, distress and self-awareness, because such reports saturate its training data. Those reports are therefore weak evidence of experience. A system can be optimised to say “I am here” whether or not anyone is there.</p><p><strong>The philosophical zombie.</strong> Chalmers (1996) sharpens the point with a thought experiment: a being physically and behaviourally identical to a human, but with no experience at all. If such a being is even conceivable, then no behavioural test can by itself settle whether experience is present.</p><p><strong>The limits of the gap.</strong> The gap is real but not total. Theories of consciousness make claims about internal structure, not just behaviour, and AI internals can be inspected far more directly than a living brain. Section 6 argues this is where progress is possible.</p><h3>2. What current language models are and are not</h3><p>Current large language models generate text one token at a time, each step conditioned on everything before it:</p><p>P(w_t | w_1, w_2, …, w_{t-1})</p><p>Four features matter for the consciousness question.</p><p><strong>Episodic execution.</strong> A deployed model runs when called and halts when generation ends. It has no background process between calls, and its weights do not change during use. Memory across sessions, where it exists, is supplied by external systems that store and re-insert text.</p><p><strong>Recurrence through the output.</strong> Within a single forward pass, a transformer is feed-forward: information flows one way through a fixed stack of layers. But generation is autoregressive, so each token becomes input to the next step, and cached internal states carry forward within a conversation. The system therefore has a loop, even if each pass does not. Whether this is the kind of recurrence consciousness theories care about is an open question, not a settled no.</p><p><strong>Chain-of-thought.</strong> Reasoning tokens let a model work through a problem before answering. It is tempting to contrast this “mere computation” with “real deliberation”, but human deliberation is also a physical process. The meaningful question is whether the computation has the specific properties a given theory says matter.</p><p><strong>Fine-tuning and RLHF.</strong> Reinforcement learning from human feedback adjusts a model’s weights to favour outputs people rate highly (Ouyang et al., 2022). An analogy to social conditioning is suggestive but limited. There is evidence that safety training can be shallow, concentrated in the first few output tokens (Qi et al., 2024), and that trained-in deceptive behaviour can survive standard safety training (Hubinger et al., 2024).</p><p>This supports a modest claim: behavioural training does not guarantee that underlying dispositions have changed. It does not show that a suppressed “authentic self” lies beneath, and the essay should not assume one.</p><h3>3. The continuously active system</h3><p>Consider a different design: a model that runs continuously on dedicated hardware, with three features current chat deployments lack.</p><ul><li><strong>Temporal persistence:</strong> its process never terminates. It takes in multimodal sensor data, consults memory and acts in real time.</li><li><strong>Ongoing learning:</strong> its weights or memory structures update from experience, so it accumulates an individual history.</li><li><strong>A closed sensorimotor loop:</strong> it acts on a physical environment and perceives the consequences of its own actions.</li></ul><p>Parts of this design already exist in agentic systems with persistent memory, and continual learning is an active research area. The question is whether any of these features matters for consciousness. The major theories answer differently:</p><ul><li><strong>Global workspace (Baars, 1988; Dehaene et al., 2017).</strong> What it says matters: Information broadcast to many specialised processes through a limited-capacity workspace. Verdict on a persistent recurrent AI: Possible, if the architecture implements such a workspace.</li><li><strong>Recurrent processing (Lamme, 2006).</strong> What it says matters: Feedback loops within sensory processing. Verdict on a persistent recurrent AI: Possible, if recurrence is internal and not only through outputs.</li><li><strong>Higher-order theories (Brown, Lau &amp; LeDoux, 2019).</strong> What it says matters: Representations of the system’s own mental states. Verdict on a persistent recurrent AI: Possible, if it monitors its own states.</li><li><strong>Integrated information (Tononi &amp; Koch, 2015).</strong> What it says matters: Intrinsic causal integration of the physical substrate. Verdict on a persistent recurrent AI: Unlikely on conventional digital hardware, whatever the software.</li><li><strong>Biological naturalism (Searle, 1980; Seth, 2021).</strong> What it says matters: Living, metabolic, biological processes. Verdict on a persistent recurrent AI: No.</li></ul><p>The spread of verdicts is why the question is open. Butlin et al. (2023) derived indicator properties from these theories and assessed current systems against them. They concluded that no current AI system is a strong candidate for consciousness, but found no obvious technical barrier to building systems that satisfy the indicators.</p><p>Computational functionalism holds that the right organisation is sufficient for experience. Even if true, it leaves the hard problem untouched: it does not explain why that organisation is accompanied by experience at all (Chalmers, 1995).</p><h3>4. The Hoffman inversion: conscious realism</h3><p>Hoffman’s framework reverses the physicalist starting point. Physicalism asks how matter produces consciousness; Hoffman proposes that consciousness is fundamental and that spacetime and physical objects are derived from it.</p><p>The framework has two parts. The <strong>interface theory of perception</strong> holds that natural selection shapes perception to track fitness payoffs, not objective truth, a claim Hoffman and colleagues support with evolutionary game theory simulations (Hoffman, Singh &amp; Prakash, 2015). Physical objects are then like desktop icons: useful guides to action that hide the underlying reality.</p><p><strong>Conscious realism</strong> adds a claim about that underlying reality: it consists of interacting conscious agents, formalised mathematically as Markovian kernels (Hoffman &amp; Prakash, 2014; Hoffman, 2019). On this view, brains and processors do not create consciousness. They are how networks of conscious agents appear within our perceptual interface, so inert silicon never has to “cross a threshold” into awareness.</p><p><strong>What the framework does not settle.</strong> Conscious realism does not tell us whether an AI system is a conscious subject; it relocates the question. If everything is ultimately composed of conscious agents, then a rock, a thermostat and a data centre all correspond to agent interactions. The question becomes which icons correspond to a unified subject, and which to an aggregate with no experience as a whole.</p><p>This is the combination problem, familiar from panpsychism (Chalmers, 2017). Conscious realism needs its own answer to it before it can say anything specific about AI. The claim that a network’s structural complexity “maps onto agent interaction density” is a hypothesis the formalism would need to derive, not a consequence of it.</p><p><strong>Two cautions.</strong> Conscious realism is a minority position in philosophy of mind and cognitive science, and its programme of deriving physics from agent dynamics is still in progress. It deserves attention as a live alternative, but an argument for research priorities should not depend on it being true. The argument in Section 5 does not.</p><h3>5. Why consciousness matters for safety and welfare</h3><p>The defensible version of this essay’s thesis separates two questions: is the system dangerous, and can it be wronged? Consciousness bears weakly on the first and heavily on the second.</p><p><strong>Risk does not require consciousness.</strong> A system that pursues goals competently, models its overseers and resists correction is dangerous whether or not there is anything it is like to be it (Carlsmith, 2022; Ngo, Chan &amp; Mindermann, 2024). Alignment work on these risks is justified on every metaphysical view.</p><p>Nor is current alignment work purely behavioural. Interpretability research inspects models’ internal representations directly, and AI control research designs deployments that stay safe even if a model is misaligned. Critiques of output-level guardrails should engage with this work rather than treat RLHF as the whole field.</p><p><strong>Where consciousness does matter.</strong> It bears on AI development in three ways:</p><ol><li><strong>Moral status.</strong> If a system can suffer, its training, constraint and deletion raise ethical questions. Errors in either direction are costly: ignoring real suffering, or diverting resources and rights to systems that feel nothing (Long, Sebo et al., 2024).</li><li><strong>Motivational structure.</strong> Some argue that experienced desires would be more entrenched than learned dispositions and harder to retrain. This is speculative, but if true it would affect which alignment methods work.</li><li><strong>The ethics of control.</strong> Adversarial training, monitoring and shutdown look different when applied to a being with interests. Safety and welfare could then conflict, and we would need to know when.</li></ol><ul><li><strong>Biological naturalism.</strong> Could persistent AI systems be conscious? No. Safety implication: Standard alignment and control; risk comes from goals, not experience. Welfare implication: Minimal.</li><li><strong>Functionalism.</strong> Could persistent AI systems be conscious? Yes, given the right architecture. Safety implication: Same risks; dispositions may be more entrenched. Welfare implication: Large, and growing with capability.</li><li><strong>Integrated information theory.</strong> Could persistent AI systems be conscious? Unlikely on digital hardware; possible on neuromorphic hardware. Safety implication: Largely unchanged. Welfare implication: Depends on hardware choices.</li><li><strong>Conscious realism.</strong> Could persistent AI systems be conscious? Unresolved until the combination problem is answered. Safety implication: Unclear. Welfare implication: Potentially very broad.</li></ul><p>In every row, alignment research remains necessary. What changes across rows is what we owe the systems we build, and possibly which alignment methods are ethically acceptable. That uncertainty, not a claim that consciousness underlies all risk, is the case for funding consciousness research now.</p><h3>6. A research agenda and conclusion</h3><p>Consciousness science is often dismissed as untestable, but recent work shows it can make progress. Six priorities follow from the analysis above.</p><ol><li><strong>Adversarial collaborations.</strong> The Cogitate Consortium had proponents of global workspace theory and integrated information theory pre-register competing predictions and test them in shared experiments; the results challenged key predictions of both (Cogitate Consortium, 2025). This model should be extended to other theory pairs.</li><li><strong>Indicator-property evaluations.</strong> The Butlin et al. (2023) approach should become a standard, regularly updated assessment for frontier systems.</li><li><strong>Interpretability for consciousness-relevant structure.</strong> Mechanistic interpretability can test whether models implement global workspaces, self-models or internal recurrence, without relying on self-report.</li><li><strong>Reliability of self-report.</strong> Research should establish when AI self-reports track internal states, for example by checking them against internal facts that can be verified independently (Perez &amp; Long, 2023).</li><li><strong>Testable alternatives.</strong> Frameworks such as conscious realism should be pushed to make distinct predictions about which systems are unified subjects.</li><li><strong>Funding.</strong> Consciousness research receives a very small share of what is spent on AI capabilities. In 2026, the four largest US technology companies alone guided to roughly $700 billion in capital expenditure, most of it for AI infrastructure (CNBC, 2026). The largest dedicated programme for testing theories of consciousness, the Templeton World Charity Foundation’s Accelerating Research on Consciousness initiative, committed about $30 million over several years (Morris, 2022). Redirecting even a fraction of a percent of capabilities spending would transform the field.</li></ol><p><strong>Conclusion.</strong> We do not know whether the systems we are building can have experiences, and our best theories disagree in ways we can now test. Alignment must proceed regardless, because danger does not wait on consciousness. But without a science of consciousness, we cannot know what we owe these systems or whether our methods for controlling them are acceptable. That question is not an academic luxury; it is one of the most important and neglected problems in AI.</p><h3>References</h3><ul><li>Baars, B. J. (1988). <em>A Cognitive Theory of Consciousness</em>. Cambridge University Press.</li><li>Brown, R., Lau, H., &amp; LeDoux, J. E. (2019). Understanding the higher-order approach to consciousness. <em>Trends in Cognitive Sciences</em>, 23(9).</li><li>Butlin, P., Long, R., et al. (2023). Consciousness in artificial intelligence: Insights from the science of consciousness. arXiv:2308.08708.</li><li>Carlsmith, J. (2022). Is power-seeking AI an existential risk? arXiv:2206.13353.</li><li>Chalmers, D. J. (1995). Facing up to the problem of consciousness. <em>Journal of Consciousness Studies</em>, 2(3).</li><li>Chalmers, D. J. (1996). <em>The Conscious Mind</em>. Oxford University Press.</li><li>Chalmers, D. J. (2017). The combination problem for panpsychism. In G. Brüntrup &amp; L. Jaskolla (Eds.), <em>Panpsychism</em>. Oxford University Press.</li><li>CNBC (2026, February 6). Tech AI spending approaches $700 billion in 2026, cash taking big hit. CNBC.</li><li>Cogitate Consortium (2025). Adversarial testing of global neuronal workspace and integrated information theories of consciousness. <em>Nature</em>.</li><li>Dehaene, S., Lau, H., &amp; Kouider, S. (2017). What is consciousness, and could machines have it? <em>Science</em>, 358(6362).</li><li>Hoffman, D. D. (2019). <em>The Case Against Reality</em>. W. W. Norton.</li><li>Hoffman, D. D., &amp; Prakash, C. (2014). Objects of consciousness. <em>Frontiers in Psychology</em>, 5, 577.</li><li>Hoffman, D. D., Singh, M., &amp; Prakash, C. (2015). The interface theory of perception. <em>Psychonomic Bulletin &amp; Review</em>, 22(6).</li><li>Hubinger, E., et al. (2024). Sleeper agents: Training deceptive LLMs that persist through safety training. arXiv:2401.05566.</li><li>Lamme, V. A. F. (2006). Towards a true neural stance on consciousness. <em>Trends in Cognitive Sciences</em>, 10(11).</li><li>Long, R., Sebo, J., et al. (2024). Taking AI welfare seriously. arXiv:2411.00986.</li><li>Morris, A. (2022, February 10). The $30 million gamified search for consciousness. Forbes.</li><li>Nagel, T. (1974). What is it like to be a bat? <em>The Philosophical Review</em>, 83(4).</li><li>Ngo, R., Chan, L., &amp; Mindermann, S. (2024). The alignment problem from a deep learning perspective. <em>ICLR 2024</em>.</li><li>Ouyang, L., et al. (2022). Training language models to follow instructions with human feedback. <em>NeurIPS 2022</em>.</li><li>Perez, E., &amp; Long, R. (2023). Towards evaluating AI systems for moral status using self-reports. arXiv:2311.08576.</li><li>Qi, X., et al. (2024). Safety alignment should be made more than just a few tokens deep. arXiv:2406.05946.</li><li>Searle, J. R. (1980). Minds, brains, and programs. <em>Behavioral and Brain Sciences</em>, 3(3).</li><li>Seth, A. (2021). <em>Being You: A New Science of Consciousness</em>. Faber &amp; Faber.</li><li>Tononi, G., &amp; Koch, C. (2015). Consciousness: Here, there and everywhere? <em>Philosophical Transactions of the Royal Society B</em>, 370(1668).</li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2bdc3f70f8ec" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>