<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Anders Lie on Medium]]></title>
        <description><![CDATA[Stories by Anders Lie on Medium]]></description>
        <link>https://medium.com/@anerli?source=rss-aedd8d333a4c------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*C5GSX0MT-Cb4j3VJqoXYTg.jpeg</url>
            <title>Stories by Anders Lie on Medium</title>
            <link>https://medium.com/@anerli?source=rss-aedd8d333a4c------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Wed, 18 Nov 2026 06:51:56 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@anerli/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[GISA: A Framework for Understanding AI Risk]]></title>
            <link>https://medium.com/@anerli/gisa-a-framework-for-understanding-ai-risk-bec934eb8e0a?source=rss-aedd8d333a4c------2</link>
            <guid isPermaLink="false">https://medium.com/p/bec934eb8e0a</guid>
            <category><![CDATA[superintelligence]]></category>
            <category><![CDATA[alignment]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[openai]]></category>
            <category><![CDATA[agi]]></category>
            <dc:creator><![CDATA[Anders Lie]]></dc:creator>
            <pubDate>Wed, 20 Dec 2023 20:31:00 GMT</pubDate>
            <atom:updated>2023-12-20T20:31:00.037Z</atom:updated>
            <content:encoded><![CDATA[<h3>Introduction</h3><p>After the ousting of Sam Altman driven by supporters of Effective Altruism, and his and Microsoft’s counterattack re-establishing Altman as CEO of OpenAI, it’s clearer than ever that the lines of a ideological battleground are being drawn. There are those, like Altman, that favor rapidly creating highly-intelligent AGI systems in the hopes that it will benefit humanity greatly. Others, such as supporters of the Effective Altruism movement and other notable figures such as Elon Musk, argue that we must treat advanced AI development with a great deal of caution, or perhaps even stop entirely until we can be more sure that it is safe to continue.</p><p>However, very little attention seems to be given to the specifics of the potential danger itself. What exactly is it that those concerned about AI are worried about? How might those worries be realized? Is there really a risk, and if so, when might this risk potentially be realized into a real-world catastrophe?</p><p>While these questions are inherently ambiguous and difficult to answer with our limited understanding of what our current and future AI systems might be capable of, I will attempt to categorize the different types of AI that we might build, and which might pose the greatest potential risk to humanity.</p><h3>TL;DR</h3><p><strong>GISA </strong>is a classification framework for advanced AI systems to assess four aspects which I believe are key to determined what level of risk an AI system poses to humanity:</p><ul><li><strong>Growth</strong>: How quickly the intelligence of the AI improves over time.</li><li><strong>Intelligence</strong>: The actual intellectual capability of the AI, from below human-level, to inconceivably high intelligence.</li><li><strong>Scope</strong>: The AI’s ability to interact with the world.</li><li><strong>Alignment</strong>: How well the AI’s actions align with human values.</li></ul><p>Based on this classification system, I believe that there is very little immediate risk to humanity from existing AI like GPT-4. However, as GPT and other advanced AI systems are developed, even just in the short-term, we will begin to see super-human intelligence with more dangerous GISA traits and which begin to pose real risk.</p><h3>The GISA Model</h3><p>In order to better understand the potential risks of various types of AI, I’ve created a classification system to try and better understand the various dimensions of an advanced AI system.</p><p>GISA is an acronym representing four key aspects of an AI system which I believe would contribute to its risk factor: <strong>Growth</strong>, <strong>Intelligence</strong>, <strong>Scope</strong>, and <strong>Alignment</strong>. I’ll refer to these from here on as the GISA “traits”.</p><blockquote>A brief note on terminology before I continue: The terms AGI and superintelligence are used somewhat carelessly these days. In a <a href="https://www.linkedin.com/pulse/what-agi-anders-lie-bgpde%3FtrackingId=Fxe51ChEQV6nQ2tnL7DgPw%253D%253D/?trackingId=Fxe51ChEQV6nQ2tnL7DgPw%3D%3D">previous article</a> I went over what I thought AGI was — in summary a spectrum of generally intelligence systems that we are currently building. Superintelligence is sometimes used to refer to super-human level intelligence, but other times to refer to a system capable of extreme self-improvement. The GISA classification system will address those differences. Every AI in the GISA umbrella is an AGI (by my definition, which is just a generalizable intelligence system), and some variants are “superintelligence” by either of the definitions I mentioned above. To avoid confusion I will stick to the definitions of GISA traits instead.</blockquote><p>Each of the GISA traits is a classification of these specific aspects of the AI system. If you’re mathematically inclined, we can represent these traits of an AI system as a set of functions over time { G, I, M, A }. Also, the growth G is simply the derivative of intelligence I, i.e. G(t) = dI/dt.</p><h3>Growth</h3><p>The growth trait of an AI system represents its potential to grow in intelligence over time. The growth capability is represented as one of the following:</p><p><strong>G0 (Manual Growth)</strong>: The AI can only grow at a rate proportional to the human effort put into increasing its intelligence. For example, a team of AI researchers is building the AI system — you can improve the system faster with more researchers, and perhaps those researchers can write code to improve it, feed it more data, etc., but the improvement is always proportional to the amount of human effort contributed.</p><p><strong>G1 (Linearly Accelerating Recursive Growth):</strong> The AI is capable of self-improvement, i.e., “reflecting” on itself (whether that be the parameters of a neural network, its own source code, or whatever it needs to modify) and making changes that end up improving its own intelligence. For G1, the AI is capable of <em>consistent </em>self-improvement (never getting stuck in a pattern of degrading intelligence), with a growth rate that increases linearly over time.</p><blockquote>Note that this assumption of linear growth over time excludes any less-capable self-improving AI systems, for example a self-improving AI system with a constant growth rate — as that would be effectively equivalent to manual growth with a higher improvement factor.</blockquote><p><strong>G2 (Intelligence-Proportional Recursive Growth): </strong>The AI is capable of consistent self-improvement at a rate proportional to the current intelligence of the system. This equates to exponential intelligence growth.</p><h4>Mathematical Model For Growth</h4><blockquote>Feel free to skip if you don’t like math.</blockquote><p>We can model these growth behaviors in terms of intelligence over time I(t) and growth over time G(t) of the AI system. Growth over time is just the derivative of intelligence, G(t) = dI/dt.</p><p>For G0, we can model the growth as G(t) = h, where h is a constant representing the human effort being contributed to improving the AI. We can see how the intelligence would change over time by taking the integral, I(t) = ht + C. This is of course a simplification as the human effort over time may change, but generally the intelligence over time is thus a function of the human effort contributed and the time passed.</p><p>For G1, we can model the growth as G(t) = kt + h. The growth is now a function of time in addition to the constant human effort. Here, k is just a constant representing the linear scalar of how quickly the AI improves its own growth. If we once again integrate we get I(t) = (1/2)kt² + ht + C. Now the intelligence is polynomial over time, and the human contribution will become increasingly insignificant as time passes and the t² term overpowers the t term, regardless of the initial constants k and h.</p><p>For G2, we can model the growth as G(t) = kI(t) + h. We are going to drop the human contribution constant h here as it will become insignificant. If we solve G(t) = kI(t) for I(t), we get I(t) = I₀∙ e^(kt) where I₀ is the initial value of intelligence I(t) at t = 0. In other words, the intelligence grows exponentially over time!</p><p>In reality of course, the growth of an AI system could most likely not be modeled by these simple growth functions, however these classifications serve as some boundaries for the approximate growth rate of an AI system.</p><h3>Intelligence</h3><p>The intelligence of an AI system is an abstraction for the general “intelligence capabilities” of the AI at a given point in time. I’ve already described in the Growth trait how this intelligence might change over time, but there is also value in recognizing when an AI system reaches certain recognizable points in intelligence capabilities. I define these as follows:</p><p><strong>I0 (Sub-human-level Intelligence):</strong> Since all AI systems under the GISA umbrella are variants of AGI, this is still a generally intelligent / capable system, however overall it is not as capable as a human.</p><p><strong>I1 (Human-level Intelligence):</strong> An AI system with this level intelligence is approximately as capable as humans in a wide variety of tasks.</p><p><strong>I2 (Superhuman-level Intelligence):</strong> An AI system which is considerably more capable than even the smartest humans in a wide variety of general tasks.</p><p><strong>I3 (Unbounded / Incomprehensibly High Intelligence):</strong></p><p>Before I try to define this level of intelligence, allow me take a step back. I’m using “intelligence” here to try and capture our ambiguous understanding of how smart, capable, dynamic, creative, intuitive, adaptable, etc. a given thing is — intelligence is not something we fully understand and so I will not try to fully define it.</p><p>However, if we assume there is such a thing as “intelligence” that is approximately higher in some entities than others, then either (1) there is a practical limit to how “intelligent” something can be, or (2) intelligence has no bounds, and in theory an entity could become more and more intelligent with no limit.</p><p>I define this level of intelligence I3 as either, in case (1), with the highest level of bounded intelligence possible, or, in case (2), with an arbitrarily large level of intelligence approaching infinity.</p><blockquote>Also, assuming intelligence is bounded by the physical resources available (for example, a malicious I2 that takes over all computation resources on earth, and then starts using the earths resources to build more computational hardware), we consider an I2 that fully utilizes the physical resources at its disposal to become more intelligent OR decides that it is already sufficiently intelligence to achieve all necessary goals, objectives, or other actionable choices, an I3.</blockquote><h3>Scope</h3><p>This category is fairly simple, containing only two classifications:</p><p><strong>S0 (Internal-Only)</strong>: The AI system has no capabilities aside from internal mechanisms, and only primitive, non-networked, non-executable based I/O. Basically, a system with no access to the “real-world”. Technically if it has any I/O at all it has access to the real-world via interacting with anyone who communicates with the system even if accessing it locally, but we have to draw the line somewhere.</p><p><strong>S1 (Any-External): </strong>The AI system has ANY capabilities at all which interact with the real-world in any way. An extreme case is open access to the internet, but it can be as small as a connection to the local area network, the ability to execute arbitrary code on its own hardware, or any mechanisms allowing it to communicate with humans.</p><p>Scope only has two categories, because if an AI system has any external access at all, if the system is sufficiently intelligent, it will be able to extrapolate on any external access it is given to increase its capabilities, influence, and understanding of the world beyond its immediate environment. Thus an AI system with any external scope at all ends up having arbitrarily extensive external scope.</p><h3>Alignment</h3><p>Alignment represents the degree to which an AI system follows “human values”. I am not going to try and define “human values”, since there are of course many interpretations, but you can imagine this might include principles such as understanding what should not happen in an ideal world like murder and theft, and what humans generally appreciate in a satisfying and happy life and how to accommodate those needs.</p><p><strong>A0 (No Alignment):</strong> The AI system contains no mechanisms which attempt to guide the AI to align with human values.</p><p><strong>A1 (Human-driven Alignment):</strong> The AI system includes a mechanism designed and maintained by humans which aims to align the AI to human values.</p><p><strong>A2 (Autonomous Self-Alignment): </strong>The AI system attempts to align itself to better meet human values autonomously, without the need of human intervention. At this point, it may not really be fully clear what this means as it’s not obvious that something like this would be possible or not — whether a system could adjust itself to human values without human guidance.</p><p><strong>A3 (Provably Guaranteed Auto Self-Alignment): </strong>The AI system aligns itself to human values, and does so in such a way that we (humans) can be 100% confident that it will not deviate from these “human values”.</p><h3>Risk Factor</h3><p>I need to cover one more definition before we get into the “good stuff”. That is, what does it mean for an AI system to have risk? I will define risk factor of an AI system at a given point in time as the % chance that the AI system will cause a <strong>global catastrophe to humanity</strong>. It is entirely possible that less-capable AI systems can still be abused and misused <strong>by humans </strong>to cause harm, for example political manipulation or disinformation. However, when refer to risk with respect to GISA variants, I’m referring to the chance that they <strong>autonomously </strong>decide to do something completely catastrophic for humanity — which could be one of an infinite number of things. Take your pick — nuclear apocalypse, Skynet, oppressive AI-world-government, etc.</p><h3>Near-Future Risk</h3><p>Now that we have defined all these traits, we can begin to utilize them to try and understand the risk factors of different variants of advanced AI systems.</p><h4>GPT-4</h4><p>Let’s take a look at GPT-4, which most would argue is currently the most intelligent system we have created so far. If we use GISA to classify GPT-4, I would say that it is <strong>G0 / I1 / S1 / A1</strong>. Its growth is purely determined by the efforts of the OpenAI team and the data they choose to train it with. Its intelligence is generalizable and pretty similar to human performance by many criteria. It does have access to external scopes in a variety of ways depending on the context in which it’s used — for example on ChatGPT it is able to communicate with arbitrary users which already makes it S1, and additionally it has access to various tools like browser interaction and a code interpreter. Finally, via RLHF (Reinforcement Learning from Human Feedback) used by OpenAI when building ChatGPT, GPT has begun to align with behaviors preferable to humans, which could be seen as starting to align with “human values”.</p><p>I believe that GPT-4 with its <strong>G0/I1/S1/A1</strong> classification has an <strong>extremely low</strong> risk-factor. I would say any system with G0 and I1 is not capable of enough growth or enough intelligence capacity to autonomously cause a catastrophic level of harm.</p><h4>GPT-N</h4><p>Even if GPT-4 does not seem to pose a particular threat, would a future iteration of GPT, say GPT-5, GPT-6, or further pose a real risk?</p><p>Let’s say a hypothetical GPT-5 pretty clearly possesses super-human intelligence, but with no self-improvement capabilities, and no change in the overall alignment paradigm. Then it would be <strong>G0/I2/S1/A1</strong>. For this case though still with no self-improvement mechanisms and assuming the intelligence is not massively better than human-level, I would consider the risk relatively <strong>low.</strong></p><p>However as soon as we reach a super-human level of intelligence, we encounter a new problem. Our modern AI systems are already a “black box” in that we cannot really comprehend their internal mechanisms very well. If we build AI with super-human intelligence, this black box problem gets worse. Consider the fact that we cannot even comprehend the human mind well yet, even though we understand many of the specific chemical and biological interactions. If we cannot understand ourselves with human-level intelligence, I find it unlikely that we could adequately comprehend the intelligence mechanisms of a super-human AI.</p><p>Therefore, once we reach that super-human intelligence, I think trying to classify what that system is capable of is going to be much harder. For example: We might at first think, this is a system we built, with no self-improvement mechanisms, that’s a bit superior to humans in most intelligence tasks, and classify it as <strong>G0/I2/S1/A1</strong>. However if it is sufficiently intelligent and cleverly uses its external scope to find a way to perform self-improvement, this model could actually be <strong>G1/I2/S1/A1</strong>, and quickly build up its I2 intelligence far past human-level<strong>.</strong> All the sudden the AI we thought to be relatively risk-free doesn’t look as safe anymore, and our rudimentary alignment may be completely insufficient for the now-highly-adaptable system.</p><p>In other words, I believe it’s entirely possible that future iterations of GPT, LLMs, or other advanced AI architectures we come up with in the near future will have a very real risk factor.</p><h3>Other Hypothetical Risk Examples</h3><p>Now for a couple other examples, let’s pick the <strong>best </strong>and <strong>worst possible </strong>GISA variants, best meaning highest potential benefit to humanity, and worst meaning highest potential risk to cause a global AI catastrophe.</p><p>Best possible GISA variant: <strong>G2/I3/S1/A3</strong>. Any variant with A3, basically by definition, has <strong>0%</strong> risk factor. Thus we would like the system to be maximally intelligent to increase its benefit to humanity. With exponential intelligence growth G2 and an arbitrarily high level intelligence I3, the contributions this AI could make to the benefit of humanity are incomprehensibly large.</p><p>Worst possible GISA variant: <strong>G2/I3/S1/A0. </strong>With no alignment at all and incomprehensibly high intelligence and intelligence growth, there is no telling the extreme consequences such an AI would have on humanity. If I were to take a wild guess, with no alignment at all, maybe there is a <strong>99% </strong>risk factor — after all, how likely is it, that by complete chance, an unaligned system would decide to copy our values?</p><p>Notice how the line between the best possible and worst possible GISA variant is drawn by a single trait, <strong>Alignment</strong>. This is really what makes the alignment problem so incredibly important to focus on going forward in AI development.</p><h3>Risk Estimations of GISA Variants</h3><p>I’ve broken down a comprehensive list with estimations for every GISA variant in this <a href="https://docs.google.com/spreadsheets/d/1fRVFm2OY0r6VnK-N4xmhSF6WPm6BGqPXQOkuxoUyhDE/edit?usp=sharing">google sheets table</a>. Obviously, many of the specific estimates I’ve made are <strong>highly speculative</strong>, but I think the relative risks that I’ve captured here are likely generally accurate — for example the general trend that risk increases with higher growth capabilities and intelligence levels, and decreases with better alignment.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*1_I32ms-tO7q2ZFzAQyDIw.png" /></figure><p>If anyone else is inspired to create your own risk estimates based on this framework, feel free to copy the <a href="https://docs.google.com/spreadsheets/d/1fRVFm2OY0r6VnK-N4xmhSF6WPm6BGqPXQOkuxoUyhDE/edit?usp=sharing">spreadsheet </a>and fill out the table at the bottom which is empty.</p><h3>Conclusion</h3><p>I’m writing this article not because I want to create a framework that big-tech can use to justify a reckless pursuit of higher intelligence, but rather as an attempt to establish a somewhat scientific methodology for trying to identity risk in the AI systems we build and as an approximation for where and when we need to be the most cautious. I fully expect that this model is a vast oversimplification for how the advanced AI we build might actually behave and what risks they might present. However, I thought it was suitable to try and take a step forward in this unexplored domain of theoretical higher intelligence, even if the step is not 100% in the right direction.</p><p>I hope someone finds this classification system useful, or at least mildly interesting. The fact is that we are marching towards a non-zero chance of AI catastrophe, and we need to very carefully weigh this possibility with the potential benefits highly advanced AI could have for humanity. As we do so, we should seriously consider everything in our capacity to minimize this risk via alignment techniques, and have a system in place to pump the brakes if it becomes clear that alignment is not keeping up.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=bec934eb8e0a" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>