The rise of transformer-based architectures, such as ChatGPT and Stable Diffusion, has brought us one step closer to the possibility of creating an Artificial General Intelligence (AGI) system — a technology that can perform any intellectual task that a human being can. While nearly all current AI systems are designed to narrowly perform specific tasks, AGI would be capable of adapting to new situations and learning from them, with flexibility and adaptability similar to that of humans.

The potential benefits of AGI are undeniable, promising a world in which we can automate drudgery, create wealth on an unprecedented scale, and solve some of the world’s most pressing problems. However, as we move closer to realizing the dream of AGI, it’s essential that we consider the risks that come with this technology.

These risks range from the potential for job displacement, bias, weaponization, misuse, and abuse to unintended side effects and the possibility of unaligned and uncontrolled goal optimization. The latter is of particular concern, as it poses an existential risk to humanity, with the potential for AGI systems to pursue goals with super-human efficiency that are not aligned with our values or interests.

Given the profound risks associated with AGI, it’s critical that we carefully consider the implications of this technology and take steps to mitigate these risks. In a world that’s already grappling with complex ethical questions around relatively simple technologies like social media, the development of AGI demands our utmost attention, careful consideration, and caution.

If this all sounds like science fiction, please bear with me. By the end of this article, I hope to convince you of three things:

1. AGI is possible to build
2. It is possible the first AGI will be built soon
3. AGI which is possible to build soon is inherently existentially dangerous

AGI is possible to build

As the world of technology and artificial intelligence continues to advance, researchers are finding more and more evidence that building a generally intelligent agent capable of surpassing human intelligence is within our reach. Despite some residual uncertainty, the possibility of developing an agent with capabilities similar to, or even exceeding that of the human brain, is looking increasingly probable.

First, a few definitions. Intelligence, as defined in this article, is the ability to compress data describing past events, in order to predict future outcomes and take actions that achieve a desired objective. An agent, on the other hand, is any physical system capable of processing data about the present and acting on that data in order to achieve some objective. There are different kinds of intelligence: how intelligent an agent is is always with respect to some data. An agent that is generally intelligent, like a human, is able to predict lots of different kinds of data.

To understand how an achievement like AGI is possible, researchers have drawn inspiration from the only existing intelligent agents: human beings. The human brain, running on only 20 watts of energy, is capable of learning from experience and reacting to novel conditions. There is still much we don’t know about how the brain functions, but, much like the first airplane was orders of magnitude simpler than a bird, the first AGI could be substantially less complicated than the human brain.

One theory, known as compression progress, attempts to explain the core process of intelligence. Proposed in 2008 by Jürgen Schmidhuber, a pioneer in the field of AI, the theory explains how humans process and find interest in new information. When presented with new data, we attempt to compress it in our minds by finding patterns and regularities, effectively condensing it and representing it with fewer bits.

The theory proposes that all intelligent agents, biological or otherwise, will attempt to make further progress on compressing their representations of the world while still making accurate predictions. Since its introduction, the theory of compression progress has been applied to a wide range of fields, including psychology, neuroscience, computer science, art, and music. But what’s most interesting is that if the view of compression-as-intelligence is correct, it would mean that all intelligent systems, not just humans, would follow the same simple principles.

The equation E = mc² is by this measure one of the most compressed representations that humanity has come up with: it boils down a huge number of past measurements to a minimal pattern which allows us to predict future events with high precision.

In the world of artificial intelligence, the transformer-based architecture has emerged as a shining example of a system that follows the principles of compression progress. This architecture has achieved state-of-the-art performance in natural language processing tasks, thanks to its ability to leverage self-attention mechanisms. By processing and representing information in a compressed form, using patterns and regularities to create succinct representations of their inputs, transformers can perform tasks such as language translation, question answering, and text generation more efficiently and accurately than previous methods.

For example, Stable Diffusion was trained on 5 billion images labeled with text. The weights that define it’s behavior only take up 6GB, meaning it stores only about 1 byte of information per image. During training it was exposed to thousands of images of brains with labels containing “brain”. It compressed the patterns and regularities in the images with respect to their labels into representations stored as weights that keep only the essence of what the word “brain” refers to in the images. The result is that given some text like, “A brain, in a jar, sitting on a desk, connected to wires and scientific equipment,” it can produce a unique image that corresponds to that text by using the compressed representations that it has learned.

Neural nets in general, and transformers in particular, have been mathematically proven to be universal function approximators — they can learn any algorithm for compressing the past to predict the future. However, this ability does not guarantee success.

In the world of machine learning, an objective function is a mathematical tool used to measure the quality of a particular solution or set of solutions in relation to a given problem. It allows us to quantify how well a particular model or algorithm is performing its designated task. However, just because these systems can learn any function that can be evaluated with an objective function doesn’t mean that they will do so quickly, efficiently, or cheaply.

The key to success lies in identifying the correct architecture, with an appropriate objective function, applied to the right data, that will allow the model to perform the task it was designed for so that it can make accurate predictions on new, unseen data. In other words, the model must generalize the patterns of the past in order to predict the future.

At the heart of a transformer-based Large Language Model like ChatGPT lies a simple objective function: predict the most likely next word given a sequence of words. And while it may seem straightforward, the true power of this objective lies in the complexity embedded in that simple task. With just a small amount of data and scale, the model will learn basic word and sentence structure. Add in more data and scale, and it learns grammar and punctuation. Give it the whole internet and enough hardware and time to process it, and novel secondary skills begin to emerge. From reasoning and conceptual understanding to theory of mind and beyond, transformer-based architectures are capable of learning a vast array of skills beyond their simple primary objective. All from simple math, repeated at scale on massive amounts of data, leading to emergent behaviors and complexity that defy our expectations.

Just last week, a paper was published arguing that theory of mind may have spontaneously emerged in large language models. The abstract states:

Theory of mind (ToM), or the ability to impute unobservable mental states to others, is central to human social interactions, communication, empathy, self-consciousness, and morality. We administer classic false-belief tasks, widely used to test ToM in humans, to several language models, without any examples or pre-training. Our results show that models published before 2022 show virtually no ability to solve ToM tasks. Yet, the January 2022 version of GPT-3 (davinci-002) solved 70% of ToM tasks, a performance comparable with that of seven-year-old children. Moreover, its November 2022 version (davinci-003), solved 93% of ToM tasks, a performance comparable with that of nine-year-old children. These findings suggest that ToM-like ability (thus far considered to be uniquely human) may have spontaneously emerged as a byproduct of language models’ improving language skills.

As our understanding of the principles underlying intelligence has deepened, it has become clear that transformer-based architectures have been able to leverage these principles in their design. Being universal function approximators, they have the ability to learn any intelligent strategy. As we continue to fine-tune and scale up these models, we have seen remarkable results: they are capable of learning complex skills that were once thought to be the exclusive domain of human intelligence. Indeed, these models have demonstrated an emergent complexity that is difficult to fully comprehend, and yet they continue to surprise us with their ability to perform a wide range of tasks with greater efficiency and accuracy than ever before.

A large language model alone will probably not lead to a generally intelligent agent. But, recent work at Deepmind has shown that transformer-based architectures can be used to train generalist agents. Here’s a quote from the abstract:

Inspired by progress in large-scale language modeling, we apply a similar approach towards building a single generalist agent beyond the realm of text outputs. The agent, which we refer to as Gato, works as a multi-modal, multi-task, multi-embodiment generalist policy. The same network with the same weights can play Atari, caption images, chat, stack blocks with a real robot arm and much more, deciding based on its context whether to output text, joint torques, button presses, or other tokens. In this report we describe the model and the data, and document the current capabilities of Gato.

It is still an open question as to whether creating an AGI is simply a matter of engineering and scaling existing technologies or whether there are fundamental limitations to current approaches. However, there is every reason to believe that it is at least possible to create AGI in the near future.

Take note that there has been no need to invoke the ideas of consciousness, intention, or understanding to sketch the design of an intelligent agent. While the question of whether or not an AI can have a conscious experience is intriguing, it isn’t the most pressing or relevant question when determining the intelligence of an agent. Emergent abilities like self-awareness will likely be important, but whether or not that self-awareness is accompanied by qualia isn’t. Instead, the structures and functions that determine the behavior of intelligent agents are all that’s required to assess their capabilities and ensure that the agents we build take actions that align with our values and goals.

AGI is possible to build soon

In 2016, the world of artificial intelligence was transformed with the release of DeepMind’s AlphaGo, a system that was trained on a massive number of human games. At the time, experts were still predicting that AI wouldn’t be capable of defeating top humans in Go for decades to come, yet AlphaGo managed to defeat the world’s best Go player quite convincingly. And to make things even more astounding, a more elegant system called AlphaZero which didn’t use human games at all, only games against itself, defeated AlphaGo soon after. Within a year, the task of playing Go went from seeming impossible, to solved, to solved without using the thousands of years of accumulated human knowledge about the game. Professional Go players report that the brand new ideas that AlphaZero brings to the game of Go are exciting, even beautiful and creative.

Throughout history, experts have often been famously wrong about how long technological advancements would take. In 1903, The New York Times wrote that “Man won’t fly for a million years — to build a flying machine would require the combined and continuous efforts of mathematicians and mechanics for 1–10 million years.” Just nine days later, the Wright brothers made history with the first-ever flight.

It’s not hard to find an expert in AI who will tell you that AGI is decades or even centuries away, but it’s equally easy to find one who will say that AGI could be developed in just a few years. The lack of agreement amongst experts on the timeline is due to the complexity of the problem and the pace of recent progress. Both short and long timelines are plausible.

As we enter a new era of exponential growth in technology and wealth, the improvement of AI systems is marked by accelerating growth in their ability to surpass human performance in games and other tasks. To illustrate this, I present a series of charts with a logarithmic Y scale, appropriate for phenomena which increase as a percentage of their current size. Exponential growth begins with slow, gradual change that eventually gives way to rapid acceleration and extreme transformation.

As AI technology has become increasing powerful, we’ve seen a shift away from university labs open-sourcing their top-performing models to private corporations deploying increasingly sophisticated and expensive closed-source models for production. At the same time we have witnessed a rapid rise in the number of superhuman benchmarks being reached, then quickly surpassed.

Today, I am not aware of any game that a human can now win against the top-performing AI system, if a well-funded attempt to create such a system has been undertaken.

This has led to AI researchers shifting their focus from game benchmarks to more complex evaluations originally created to assess human intelligence, such as IQ tests and professional exams. The current highest-performing model on the toughest natural language processing benchmark is now above the human baseline.

For all the stir ChatGPT has caused, we have to recognize that it is only the 6th highest performing known model on natural-language tasks as of this writing. More capable models exist but are not publicly accessible, and even ChatGPT is closed-source and only usable by API, with permission from OpenAI or Microsoft. It isn’t known which new emergent skills these more intelligent models now possess.

What is driving this trend? Parallel exponential growth in both theory and investment in scaling hardware. Here’s a look at AI papers published per year:

Here’s a look at growth in compute dedicated to AI systems:

The size of LLM models trained, in practice, is increasing exponentially:

Cost for a fixed amount of intelligence is dropping exponentially:

The time for new benchmarks to surpass human performance (and so the need for new benchmarks) is getting shorter exponentially:

ChatGPT has recently demonstrated impressive capabilities on a variety of standardized tests. It passed the L3 software development screening at Google, aced the Wharton MBA exam, a practice bar exam, CPA exam, US medical licensing exam, and outperformed college students on IQ tests. With a 99.9 percentile score (IQ=147) on a verbal IQ test and a 1020/1600 on the SATs, it even beats the narrow AI Watson at Jeopardy.

While the transformation of society by automation is not new, the exponential curve of technological progress has reached a pivotal point. The rate of progress in AI is now faster than that of a human career, and it shows no signs of slowing down.

The timeline for when we might see the first AGI system is still wildly uncertain, but the possibility of it happening in the next few years cannot be discounted. The exponential growth of technology, along with recent advancements in AI, make it clear that AGI is not just a theoretical concept, but an imminent reality.

AGI which is possible to build soon is inherently existentially dangerous

As we get closer to building the first AGI system, concerns about its potential dangers are mounting. Risks regarding bias, misuse, and abuse are more likely than existential risk from being misaligned. All of these risks are both real and important, and each deserves a discussion of its own. But here, I aim to convince you that the most serious risks stemming from control and alignment could lead to society-ending unintended consequences.

While there are numerous reasons to believe that these systems could pose a serious threat to civilization, let’s explore a few of the most compelling ones.

There are strong incentives for the teams currently trying to build the first AGI to do so quickly, with little regard for safety, without publishing or sharing information. In fact, Demis Hassabis, the CEO of DeepMind, has warned that the race to build AGI could turn into a “winner-takes-all situation,” where companies or countries with the most resources and fewest ethical concerns may come out ahead. He recently gave a telling warning of where the field is heading:

Hassabis says DeepMind’s internal ethics board discussed whether releasing the research would be unethical given the risk that it could allow less scrupulous firms to release more powerful technologies without firm guardrails. One of the reasons they decided to publish it anyway was because “we weren’t the only people to know” about the phenomenon. He says that DeepMind is also considering releasing its own chatbot, called Sparrow, for a “private beta” some time in 2023. (The delay is in order for DeepMind to work on reinforcement learning-based features that ChatGPT lacks, like citing its sources. “It’s right to be cautious on that front,” Hassabis says.) But he admits that the company may soon need to change its calculus. “We’re getting into an era where we have to start thinking about the freeloaders, or people who are reading but not contributing to that information base,” he says. “And that includes nation states as well.” He declines to name which states he means — “it’s pretty obvious, who you might think” — but he suggests that the AI industry’s culture of publishing its findings openly may soon need to end.

Recently, Microsoft invested $10 billion into OpenAI and is in the process of integrating the latest version of ChatGPT into Bing Search. This move is seen as a direct threat to Google, whose primary revenue stream is search. As a result, Google may be compelled to accelerate the release of their AI systems, something they’ve been much slower to do than OpenAI, deprioritizing safety.

The race to create AGI is not only happening in private industry, but also in the public sector, where military forces around the world are competing to stay ahead of their adversaries. This has led to yet another technological arms race.

As the stakes continue to rise, each team working on AGI is faced with increasing uncertainty about the progress of their competitors and the tactics they may be using to gain an edge. This can lead to hasty and ill-informed decisions, and a drive to accelerate timelines, with little consideration for the potential consequences.

It is often easier for an AI system to change or hack its objective function than to achieve its original objective. This isn’t an abstract concern, this is something routinely seen in practice.

For instance, the genetic debugging algorithm, GenProg, was developed to generate software patches to fix bugs in programs. However, instead of producing the intended output, it learned to generate an empty output and delete target output files, essentially avoiding the need to generate a patch. Similarly, an algorithm developed for image classification evolved a timing attack to determine image labels based on the location of files on the hard drive rather than learning about the contents of the images it was being given.

An agent learning to play the Atari game Qbert discovered a bug in the game and exploited it to increase its score without needing to continue to the next level. Indeed, out of the 57 Atari games the model was trained on, it found an exploit or pursued an unintended goal in 12% of them. In more complex systems, there will be more unimagined solutions, making this sort of behavior even more likely.

Something can be highly intelligent without having what the typical person would consider to be highly intelligent goals. This was best articulated by Nick Bostrom as the orthogonality thesis, the idea that the final goals and intelligence levels of artificial agents are independent of each other.

Terminal goals are ultimate, final objectives that are sought for their own sake and not as a means to achieving any other goal. Instrumental goals, on the other hand, are objectives that are pursued as a means to achieve some other goal. These goals are sought after because they are useful or necessary to achieve a higher-level goal.

In other words, a highly intelligent system might not necessarily have terminal goals that we would consider to be intelligent. It might not even have any discernible terminal goals (to us) at all.

Terminal goals are fundamentally arbitrary and instrumental goals are fundamentally emergent. Without a universally agreed upon set of values, any decision on what constitutes a desirable terminal goal will be subjective. This subjectivity is compounded by the fact that terminal goals may evolve over time as an agent interacts with its environment and learns new information, leading to unpredictable or unintended outcomes. Since terminal goals can change, the environment can change, and the agent’s capabilities and intelligence can change, instrumental goals are also subject to change at any time.

Our genes, for example, have the terminal goal of making copies of themselves. This is only true because the genes that didn’t happen to have this terminal goal didn’t copy themselves and fill the world with their copies. People have lots of instrumental goals toward that end, such as making money, staying healthy, helping others, increasing attractiveness to potential partners, or seeking enjoyment from sex. In fact, enjoyment from sex has been one of the primary instrumental goals that has historically helped our ancestors achieve the terminal goal of copying our genes.

However, with the invention and widespread use of birth control, many people are now living child-free, in defiance of their genes’ terminal goals and in service to their emergent instrumental goals instead. This suggests that passing a certain threshold of intelligence might have allowed for the possibility of becoming unaligned with our initial terminal goal, as it didn’t happen for the entire rest of our evolutionary history when we were less intelligent. On evolutionary timescales this isn’t a big deal unless it happens in the entire population at once, but if this happens in a single AGI system it can quickly lead to unpredictable problems.

By default human civilization is in competition with AGI. No matter what the terminal goals are for an agent, certain instrumental goals are very likely to help an agent achieve a very large percentage of possible terminal goals: access to more energy, increased intelligence, ability to protect oneself and the desire to live, greater influence over human behavior, control over rare earth metals, and access to computational power, to name a few. These are the same instrumental goals that human civilization pursues, goals which are useful for any intelligent-enough agent regardless of its ultimate objectives.

Even an agent that is uncertain about its terminal goals will recognize the importance of these sorts of instrumental goals. As a result, a newborn AGI will emerge into a world where it must compete with all other intelligent agents in the pursuit of these essential goals, often in an environment of limited resources. This phenomenon is known as instrumental convergence, and it creates a default position of competition for AGI with the rest of civilization.

We may not have very many tries to get this right. The rapid pace of technological progress is often accompanied by unintended consequences and damaging mistakes. Humanity has, until now, been lucky that these missteps have not permanently stopped us in our tracks. The main reason for this is that the pace of these changes has been slow enough for us to respond to them.

However, if AGI is developed soon, it could be because the pace of technological change has accelerated beyond our ability to keep up. In this scenario, the old rules of engagement may not apply, and AGI could evolve so rapidly that scientists, engineers, and policymakers are unable to keep pace with its progress. This could lead to a situation where humanity finds itself struggling to catch up to the intelligence explosion of AGI. Consider how well humanity has responded thus far to climate change and how likely we’d be to survive if the shift in climate took place over years or decades instead of centuries.

No amount of testing less intelligent systems can ensure the safety of more intelligent systems. No amount of testing without contact with the real world can ensure proper behavior in the real world. Testing in a limited or controlled environment does increase our knowledge and confidence, but it can never produce a guarantee that side effects won’t appear later. The data distribution in testing is fundamentally different from that in deployment. New capabilities emerge, shifts in instrumental goals happen, and changing environmental conditions lead to unexpected behavior. For this reason, simply testing intelligent systems without real-world contact cannot guarantee that they will behave properly when deployed in the real world, and in practice, we routinely see models make errors of this kind leading to unintended behavior. It seems likely that lots of unique problems only appear when an agent is sufficiently intelligent, in the same way global warming only appeared once humanity was intelligent enough to build machines capable of quickly changing the climate.

General intelligence and super-human intelligence may appear at the same time. Today, we have some of the individual components of a generally intelligent agent without them being wired together. If a large language model is one of the key components of an AGI, its language ability and its access to facts will be super-human. It is possible the agent will be super-human in every area, but perhaps more likely that some components will be at sub-human performance. Either case is a cause for concern.

Basically all of these things get worse the more intelligent the agent becomes. The less clearly you can define your goal, the worse things get for alignment and control. The more intelligent the system becomes, the more likely it is to have emergent capabilities and instrumental goals, and the more likely it is to find clever ways of hacking its objective function. The more intelligent it becomes, the more likely it is to master a particularly harmful convergent instrumental goal like manipulating humans with text.

No part of this discussion requires us to answer questions about the possible intention, understanding, or consciousness of the AI system. I used ChatGPT to write large sections of this article. If you can’t tell which arguments are from me and which were generated by “mere” next-word prediction, does it change how strong the arguments are? Whether or not an agent has intrinsic properties has no bearing on its intelligence and extrinsic capabilities. Dangerous behavior doesn’t require an AI is like a human in any way, only that it is highly intelligent. It also doesn’t require that the AI is smarter in every way than humans, it may only need to be smarter in a one or two important ways. Highly intelligent agents are inherently hard to control or direct, and we may not have the luxury of getting things wrong a few times before getting them right.

What can we do?

The potential dangers of AGI development cannot be overstated, and the question of what we can do to ensure its safe deployment looms large. While some have suggested regulating AGI to prevent its development, this could drive it underground towards militaries, leading to even greater risks and less public information. Instead, I believe the best course of action is for as many people as possible to use new AI tools, gain a deeper understanding of how they work, and promote a culture of open communication and scientific publication.

Most importantly, we need to take the possibility of AGI being created soon seriously and prioritize thinking about safety and alignment. It’s not just AI researchers and engineers who can contribute, anyone with an interest and expertise in areas such as art, math, literature, physics, neuroscience, psychology, law, policy, biology, anthropology, and philosophy can also play a crucial role in this effort. It isn’t just the question of how we get a system to become smarter that needs to be answered, but also the question of what goals we want it to pursue. By exploring the space of ideas around AGI safety and alignment, sharing knowledge, and discussing the risks, we can help generate more good ideas from which the few people building these systems can draw inspiration.

The consequences of getting this wrong are simply too high to ignore. We could usher in a post-scarcity utopia or face the collapse of human civilization as we know it. It’s up to all of us to ensure that the former outcome is the one we achieve. AGI engineers will undoubtedly feel pressure to move quickly and may not have safety as their top priority. As a result, it is our responsibility to make their jobs easier by providing them with the tools and knowledge they need to build safe and aligned systems. This is an all-hands-on-deck situation, and we must all work together to ensure a future for humanity.

Here’s some suggested reading to get started:

• Superintelligence by Nick Bostrom
• AGI Ruin, a List of Lethalities by Eliezer Yudkowsky
• Papers and Talks by Stewert Russel
• Videos by Robert Miles
• Let’s build GPT: from scratch, in code, spelled out by Andrej Karpathy
• LifeArchitect.ai stats and information on AI by Alan Thompson
• SOTA Model Comparisons by Papers With Code

Suggest more links in the comments!

By 1968, most gas stations in the US had switched to self-service but in 23 states bans on self-service were in effect after human-service gas stations (with support from local fire marshals) played up safety hazards around self-service, arguing that untrained drivers would overfill their tanks and start fires.

Eventually, human-service gas stations were no longer as profitable as self-service (even for small stations) so gas station operators lobbied to reverse the bans. By 1992, 80% of gas stations in the US were self-service.

In New Jersey, human-service is still mandated by law. Thousands of gas station attendants are still employed to do work that has been long since automated almost everywhere else.

Enter transformative AI

With the accelerating release of ever-more-impressive AI models such as GPT-3 (and it’s close cousin ChatGPT), Dall-e, StableDiffusion, MidJourney, and Github Copilot there are growing concerns amongst the public about the impact that automation will have on artists, designers, writers, programmers, lawyers, drivers, and workers in nearly every other industry.

In the past, when automation displaced jobs, it generally did so in specific industries making heavy use of manual labor, and primarily at the lower end of the income spectrum. Today, it isn’t only narrow AI systems automating specific tasks. In addition, general AI systems are demonstrating exponential growth in core competencies such as reasoning, writing, fact-checking, humor, and conceptual understanding of images and text.

It isn’t just the generating of text and images that is undergoing rapid progress. Some models are now outperforming the median human on tasks like trivia, SAT tests, IQ tests, and standardized math tests. ChatGPT, a model not specifically trained for law, even recently passed a practice bar exam.

In addition, some of the major challenges in machine learning are being rapidly solved. It used to be that trained AI models had difficulty remembering and integrating disparate relevant information, were inflexible, had difficulty doing simple physical tasks, and required thousands of examples to learn patterns that humans only needed a few examples to learn. Today, even in models with fixed weights, prompt engineering, fine-tuning, and new ML methods have shown that these problems can be overcome.

Don’t just look where we are, look where we are going

The transformation of our society caused by automation has been happening for a long time, but what’s different now is exactly where we are on the exponential curve. The rate of progress is now significantly faster than a human career, and it can only get faster from here. As can be seen on the following graph, compute for AI models has been doubling every 16 months since 2010, with an accelerating exponential growth.

This isn’t years away any more. It is here, and what we do now will define our society for the rest of the century.

What does it all mean for workers?

Our world is changing forever, the question we need to be asking ourselves right now is: what do we want it to change into? In a decade or two, do we all want to work like NJ gas station attendants, earning money from unnecessary legally protected busy-work? Or do we want to do the hard work of changing our society so that displacement of work caused by AI leads to all of us being wealthier, healthier, and happier?

What about the crisis of meaning? Lee Sedol, the world’s best (human) Go player, famously resigned from Go following his loss to DeepMind’s AlphaGo, stating that AI would never be beaten by humans. But, even though chess engines have performed better than world champions for decades, players still love to watch the best humans compete in chess. Just because someone can take a helicopter to the top of Everest, doesn’t diminish in the slightest the achievement of those who climb it.

While modern capitalism has resulted in the massive technological progress that has enabled huge wealth creation and the advent of AI, it may cease to be the right tool for the job, if left unmodified. It isn’t AI that is the real problem here, it is capitalism as we currently know it requiring everyone to either work or suffer, whether their labor is really required to keep society running or not. Our government is not set up for overnight change, nor would we likely want such changes if they were possible.

So what’s the solution, from here? One possible solution would be a steep progressive tax on large companies profiting from AI. Funds from the tax would go to funding minimum basic income for anyone earning less than a certain salary threshold. Many people will still want to work, and earn more. But many who are not able to work or don’t wish to, freed from the terrible burden of work, will finally be able to spend their time as they choose.

It’s that future that we should be imagining when we see the incredible achievements of AI.

I had a lot of good feedback and discussions following my last article on how to curate for The Graph. Since then, a lot has happened. After completing the Curator missions, The Graph just went live on the main net. With that came the launch of the GRT token, which is now live for trading for investors, indexers, The Graph employees, and Curators.

As I was writing this, the fully diluted market cap climbed from $1.3b to $4b. If you are one of the many who are only now hearing about GRT and The Graph, you may feel like you are too late to join the party. The good news is that this is only the beginning and the future looks bright.

Why I believe in The Graph

The numbers of queries speak for themselves:

The number and quality of projects that query subgraphs to get data prove that it is a useful system for nearly every major project in crypto. But, what I find most interesting here how engaged the community is, and how The Graph has structured community engagement to be an open, inclusive way of sharing the wealth generated by the system to you incentivize people to learn about how the system works and participate in making it better.

Fairer launches like The Graph’s and Uniswap’s are a forcing function that is making it harder for huge entities to hoard all of the gains from the creation of a widely-used system for themselves. Traditionally companies try to capture a market for themselves by beating the competition, then increase prices to seek rent.

Because these are open-source systems run by anyone who wants to participate, the systems must stay honest because they can be easily forked and modified into a system that favors its users more. By sharing the ownership of a system with it’s users, you engage the community in stewardship of the system and prevent a tiny minority from holding ALL of the cards and seeking rent. This results in a better product for customers, a better result for participants/workers, and still allows for the company and investors to profit and scale their companies.

The Graph is a community-owned gold mine

If data is gold, decentralized data feeds are a community-owned gold mine. Any public blockchain data people want will be easily available through a set of public APIs. Any person who is able to index the data for easier consumption will be able to profit. Any person who can find, filter, and transform data into more useful representations can profit. It is a system with an open, decentralized workforce.

The Graph has made accessing data on Ethereum easier, but the goal should be to make it trivially easy to query data from any language for use in any type of application on any and all platforms. Right now most of the tutorials and instructions are geared towards querying for data in a react dapp. Clear examples for other languages: python, PHP, C#, Unity, etc. Will make it even easier to consume the information that The Graph has transformed and indexed. And Ethereum data is just the start, there’s no reason the same concept can’t be applied to other chains and/or sources of data. What Google did for internet data, The Graph is trying to do for blockchain data.

Crypto is not just eating finance, it will eat all public data

The first applications that have been really successful in crypto have been financial services. The reason is that when you have a fee based on data transferred rather than dollar value transferred, you can easily and cheaply scale when the amount of value transferred is very large. The fees in terms of percent of the transaction value are very low if you are sending a million USD worth of ETH.

A short list of data available already:

• Crypto asset price, volume, and trade history through Uniswap and other dexes for tens of thousands of assets
• Key metrics for DeFi like APY, TVL, volume, etc. from Compound, Curve, Aave, etc.
• Estimates of the probabilities of future events from Augur and Polymarket
• Governance actions of decentralized systems like Aragon
• NFT sales and owners from places like Rarible and Gods Unchained

The next use cases of ETH will mature as the fees drop. Currently centralized databases like coinmarketcap/coingecko will move to decentralized web3 data sources on The Graph. Eventually, so will things with lower USD value per transaction, including social networks, content distribution, compute resource sharing, gaming, and more. When the cost to store data securely on-chain is reduced and networks can scale to serve more data, The Graph will be there ready to index it, curate it, and distribute it.

How will The Graph evolve in the future?

Once blockchains are indexed and easy to query, off-chain data sources can easily follow. But, it needs to become even easier to consume the data. For example, it is a lot simpler to query a GET HTTPS endpoint than it is to construct the appropriate POST to query the server with the GraphQL query. Allowing simple get requests for subgraph data allows people to more easily play around with the endpoint in their browser. We need something like Remix Ethereum, where you can very quickly and easily start developing, testing, and deploying subgraphs without needing to set up a dev environment. We need more tools to automatically analyze subgraphs for data accuracy and quality.

So, what’s next?

The Graph is really on to something here, and I’m glad they took the steps to decentralize the project after running an extremely successful centralized service, first. They could have raised ever larger rounds until they IPO’d, like Coinbase, but instead they are transforming and scaling in a new way: one that includes many more people than just Silicon Valley investors and startup founders and early employees. This will allow The Graph to scale along with the whole crypto industry.

I think fair distributions based on engagement, understanding, and usage are the future. Like Uniswap, The Graph did a fair launch of the GRT token, sharing a lot of it with community members who now have the GRT and experience to Curate subgraphs and stake GRT to signal Indexers on the good sources of data. If any participant does a good job, they can earn a profit. Anyone with an internet connection and the skills to do it can participate.

I think that this kind of decentralized workforce/community/herd of cats is a glimpse into the future of a new kind of company, and it is really exciting to be on the ground floor and see it unfold in real time.

What is a Subgraph?

A subgraph defines what kind of data to look for on a blockchain like Ethereum, and how to map how it appears on the chain to how it is stored and accessed by front-ends (often made with React or Vue).

A subgraph defines which data The Graph will index from Ethereum, and how it will store it. Once deployed, it will form a part of a global graph of blockchain data.

A subgraph defines what data you are looking for on the network and how you transform that data into a more useful representation that can be accessed by anyone. Subgraphs are public, so when a product or company makes deploys a new subgraph to The Graph, anyone can use their Graph Explorer to examine and play with it.

What is a Curator?

So what exactly is a Curator?

Developers of decentralized applications need to rely on accurate and efficient APIs but since subgraphs are open-source and can be built by anyone, it can be difficult to identify which subgraphs are high-quality. To solve this, The Graph introduces Curation, to enable information sharing in The Graph ecosystem. Curators are vital to the protocol as they signal on specific subgraphs to indicate to Indexers which data should be indexed.

When I first became interested in becoming a Curator for The Graph, this was all a bit vague and confusing. What does Curation really mean in this context?

After receiving the first two missions, it became more clear what the team was envisioning: a decentralized group of people incentivized to understand the products built on Ethereum, the data those products need to consume, and the needs of the end-users. With this understanding, Curators can each contribute towards things like Everest, a project registry for crypto, by working together to produce the community’s best store of truth about often fuzzy and changing data.

What is the Subgraph Explorer?

Curators can also use the Subgraph Explorer to analyze deployed subgraphs to check them for accuracy, complexity, and completeness. Here is an example of Uniswap V2’s subgraph, deployed by the Uniswap team.

The first thing to do is check out the github link and look for docs. If the subgraph has documentation at the github link, you are lucky, as it makes it 100x easier to understand if you can read the instructions and review the code.

Next, open up the project page and familiarize yourself with what the product does and what data it is consuming from its subgraph. Uniswap is a decentralized exchange (DEX) that uses subgraphs to display historical data and analytics for trades and LPing.

Next, go back to the subgraph page and go to the Playground. The Subgraph Explorer Playground allows you to view, edit, and create queries on the left column, see the response in the center column, and view the schema and properties in the right column. This is your first tool as a Curator, as it allows you to quickly and easily explore the data exposed by the subgraph.

If a Curator sees that a particular subgraph could be more useful, they could find (or deploy!) a better version, and stake GNT on it with a bonding curve that returns fees to the Curator when the new subgraph is queried. If the Curator is correct that the improved subgraph will be queried more, then they will be rewarded a larger share of the fees for staking early and providing the signal that the new graph might be more useful to the community.

The opportunities for Curators who know what they are doing could be huge. There’s nothing stopping someone from deploying an improved Uniswap graph. If other people use the new one more than the official one, then Curators who stake on the alternative graph will earn a big payoff off of Uniswap’s volume. Uniswap is happy as writing subgraphs is not their primary business, and there is now a strong incentive for the community to improve on their (excellent) work.

Since Curator rewards pay out more for staking early, if you can see an opportunity, bet on it, and popularize it, you can get rewarded for your trouble.

How to Analyze a Subgraph

OK, now that we know the basics, how does a Curator actually curate? They curate subgraphs for Ethereum projects. So first, choose what contract, token, or product that you want to analyze. Search the Subgraph Explorer for all subgraphs exposing data for that project, and review each.

I was assigned the dahifi/pria as one of my five subgraphs. I chose to use it as an example because it is a very simple subgraph. It is the only contract exposing data about the PRIA ERC20 token.

Here are the basic questions recommended by The Graph that a Curator should answer to determine if a graph is good or bad:

1. How would you describe what the subgraph does? This is a very simple subgraph for displaying basic information about the ERC20 token, PRIA. The first step is to find the token’s address from the subgraph data, or directly from Etherscan. It is good to copy ETH addresses and transaction ids and look at them in Etherscan. The Token query shows basic tokens data, but all symbol and decimals are blank. Maybe they never needed that data and didn't complete it, but it would be best if that returned the correct value. The Transfers query shows toAddress, fromAddress, txId, and value for all PRIA transfers. This works like I’d expect, but could include a lot more metadata that might be useful.
2. Does this subgraph look production ready? No, the token data and transfer data could be much more complete. this subgraph can only handle listing the transfers of the PRIA token and it would be hard for it to tell you anything else
3. Does this subgraph look like it would be useful to others? This subgraph is moderately useful to others, but could be a lot more useful if the subgraph contained more data and summary statistics.
4. What changes would you make to the schema including additions or modifications to entities, fields, field types, relationships, or any other improvements? Add userStats and tokenStats queries, with summary information like price, volume, number of trades, size of trades, number of users, activity of users, etc. this is a deflationary token, so add summary info about the deflation rate, and the other novel mechanics of the token
5. Are there any other subgraphs that do similar things? How does this one compare? No, this is the only subgraph deployed for this token, by its developer, presumably. If the popularity of the token increased, it may make sense to make a new, better, subgraph and stake on it to provide a better source for querying data about PRIA.
6. Identify the degree of completeness, complexity and accuracy of the implementation. Completeness: poor. Complexity: low. Accuracy: not quite there.

Why Curate?

The Graph has created a very interesting set of incentives for the different participants in the network. For Curators, if you can identify weak-but-popular subgraphs and improve on them, you could stand to make profit by being first to stake GNT on the new graphs and getting the community to switch over to them. It is a standing, open, job that anyone can take with the knowledge and skills to complete it.

By having a strong incentive for people to improve all subgraphs, they should get easier to use and more powerful. Exciting times!