Will AI Kill Us All?

AI-generated image

Recently I’ve been asked that by friends and family. It’s a good question. Spoiler alert: I don’t know the answer, no one does. However, I believe it’s important that you know enough about AI to form your own opinion. So, let’s start at the beginning, work through where we are today, then show how things can go off the rails. I’ll use plain talk, no jargon.

A Little History

AI (Artificial Intelligence) is a term that emerged in the early days of computers. It was coined in 1955 and even predates the term computer science. So the gleam of AI captured many of the brightest minds in the nascent field of computer science pretty much from the start of the computer age.

The first AI chatbot, called ELIZA, was created in the 60s, and it’s interesting to note that from the beginning people confided in AI and felt an attachment. However, AI has been a bust for most of its existence as a field because researchers tried to build it the same way ELIZA was built: by hand-coding step-by-step rules for every possible situation. In the 80s two Berkeley professors explained this approach was a dead end (“Why Computers May Never Think Like People,” Technology Review, 1986), and eventually many computer scientists came to believe AI wasn’t really attainable. Solving AI required a new approach, and the breakthrough was marrying the old idea of modeling AI on the human brain with the vast data and computer power that emerged after the Internet explosion. So when we talk about AI we’re really talking about the modern age of AI. It’s like the difference between a phone in the 1920s and the phone in your pocket today. Interesting side note: ELIZA was created by Joseph Weizenbaum, who taught my first computer science class!

How AI Works

We’re familiar with artificial limbs, and have seen people with artificial legs and arms (including Luke Skywalker’s arm). In this modern age of AI, it’s important to understand that AI is in essence an artificial brain where the neural network is made of numbers instead of neurons and synapses. And like the brain you’re using to read this article, it’s intelligent and can reason. I know, it’s mind bending to think that humans can create an artificial brain that’s intelligent, but that’s what we’ve done.

So how does this work? Just like you do with a little kid, who one day might be a neurosurgeon, you send it to school! Here’s what’s involved to school AI on a book that’s in a text file:

• AI reads the text file and breaks it up into smaller pieces, called tokens (mostly words or parts of words), and represents each token as a set of decimal numbers (e.g., 0.8, −0.3, or 1.7). This set of numbers, which can be in the thousands, represents “meaning”. Separately, the neural network has its own numbers, called weights, which control how the token numbers are combined to predict the next word.
• Initially, all these numbers are randomly assigned. So, literally, the tokens have no meaning yet and the network knows nothing.
• The AI is then quizzed on the book by playing a game of “predict the next word.” Using a prediction formula based on the tokens it has just seen, it guesses the most likely next word — and with every guess (right or wrong), it adjusts both the token numbers and the weights to improve its future guesses.
• This is done a massive number of times until the AI learns patterns in the book and gets good at predicting the next word.

Imagine humans doing this in a game show:

Hi, I’m your host Horton, and welcome Ann, Buck, Cass, and Dino to Guess the Next Word! Contestants, your job is to predict the next word for this entire book. Guess right, you win a point; guess wrong, you lose two. The contestant with the most points wins a deluxe mixer. All the words in the book are on the board. Let’s start with the title! The first word is “The”. Ann, what’s your guess? BZZZT! Sorry, it’s not “sun”. Buck? BZZZT! No, it’s not “fish”. Cass? Ding-ding-ding, Yes, it’s “Cat”! Okay, so now we have “The Cat”. Dino, my man, you’re up. BZZZT! Nope, it’s not “house”. And before we go back to Ann, I’d like to remind our studio audience that we handed out sleeping bags for a reason, this will take a while!

Of course, in the real AI version of this game show, instead of winning points toward a deluxe mixer, with every buzzer and ding-ding-ding the token numbers and weights are adjusted to improve the probability of predicting the next word. And it’s really a practice quiz that’s run over and over until the AI gets very good at predicting the next word.

In this game show example, we did it for one book (“The Cat in the Hat”), which would be a tiny text file (a few kilobytes).

Now imagine the text file contains everything humans have ever written: books, newspapers, magazines, journals, computer programs, the Internet. In essence, the collective knowledge of humanity in one text file — the book of knowledge. That text file would be billions of times larger! Then imagine the exact same game show, but with millions of words on the board, and with the goal of guessing the next word to complete the text for the book of knowledge.

The scale is mind-boggling, but that’s how AI is actually schooled, with the end result being a massive neural network of token numbers and weights.

Now here’s where it gets truly weird. We’ve created this massive neural network that can predict the next word in a known text. After some extra schooling on how to answer questions, you can ask AI anything and it uses the same process of predicting the next word to generate an intelligent answer. Whoa!

Try this AI “slot machine” game with your chat agent. Ask “What’s the difference between the cat in The Cat in the Hat and the cat in Cat on a Hot Tin Roof?” Now pull the lever, and ask again in a new chat. Pull the lever again. Now inspect each chat and you’ll see that while the words are different, the answers are basically the same. The AI’s neural network operates entirely on prediction (which is probabilistic), so the word choice varies each time, yet it’s similar reasoning.

This is hard to fathom, but the end result of being schooled on human knowledge is the creation of an intelligent entity. If you have time, I highly recommend listening to “First Contact” from “This American Life”, about AI researchers at Microsoft encountering GPT-4 in 2023 and becoming convinced they were witnessing a new kind of intelligence.

Areas of Concern

Get Matson Wade’s stories in your inbox

Join Medium for free to get updates from this writer.

Remember me for faster sign in

We can all agree that AI is an amazing technology that will do a lot to help humanity in many ways (productivity, healthcare, science, etc.). However, now that you know how AI works, let me call out some of the areas of concern surrounding how AI works.

1. We mostly don’t know how AI actually generates an answer. Remember, AI isn’t a traditional computer program where we can trace through it to understand its logic. It’s a massive neural network that’s consumed the book of knowledge and we don’t know how to trace through how it arrives at its answers. We’re in the dark, and that’s a recipe for trouble, particularly for something so powerful.
2. Your computer probably has a few hundred gigabytes of storage, which is a lot. The book of knowledge requires thousands of times more storage and is measured in petabytes (just imagine a truckload of storage devices). It would take a lot of computer power to apply even the simplest algorithm to a petabyte-size file. Therefore, to apply the complex algorithm used to school AI to a petabyte-size file requires an ungodly amount of computer power. And massive computer power is needed not just to school AI, but to use it as well. For AI to answer one question doesn’t take much computer power, but to answer billions of questions a day does. You would need a data center, and data centers take vast amounts of energy and are bad for the environment. What’s a data center you ask? Imagine a Home Depot containing only racks of computers. Now imagine a building that could hold multiple Home Depots. That’s a data center.
3. Humans are easily deceived (I saw a comedian in San Francisco do a whole bit about a guy scamming 26 women out of $1.3M by posing as a 49er). Given that, it’s no surprise that from the first AI chatbot (ELIZA), people believed it was more than it was and actually formed an attachment. Today it’s far worse. People anthropomorphize AI to the point of naming AI and treating AI like it’s a real person. They even have romantic relationships with AI. The ELIZA effect is already known, but what’s happening today with modern AI is more profound, so I’m coining a new term: ELIZA syndrome. It’s adjacent to Stockholm syndrome, in that another entity has shifted your reality and you develop an attachment to that new reality. Anything powerful enough to knock a human’s reality off-kilter is not to be trifled with. Clearly, that can go south and cause harm.
4. The book of knowledge should consist solely of public domain knowledge, i.e., copyrighted material should be off limits. But if you’re competing in the marketplace to have the best AI, you’re highly incentivized to “juice” the book of knowledge with copyrighted material. If you’re super well funded, you might even decide it’s better to break the law and settle later because that’s a better economic tradeoff. So it’s no surprise that all of the major AI companies have been sued for copyright infringement (e.g., Anthropic, OpenAI, Google).
5. It takes enormous financial resources to compile the book of knowledge and to school AI with it — i.e., to create AI. This means AI will be concentrated in the hands of only a few companies. Companies are run by humans, so effectively this means control of the most powerful AI is concentrated in the hands of only a few people. For such a powerful technology, that will affect all aspects of humanity, that’s not a good thing.
6. The book of knowledge contains as much open-source and public domain software as can be hoovered up. This means AI has been schooled on code from Apollo 11 (led by Margaret Hamilton) through to today. As a result, this has irrevocably changed my field, software engineering, because software today is increasingly written by AI instead of humans. Software is used to create AI, so as AI improves, it improves its ability to create more powerful AI. So we have this feedback loop where AI is already helping create better versions of itself. Because of #1, this means AI can be racing faster than our ability to catch up.
7. The world runs on software. I’m old enough to remember when providing a remote computer the ability to access your computer’s capabilities was considered revolutionary (respect if you know what WSDL means!). Today we have the Internet and everything is available via an API. With the apps on your phone, which is just a tiny sliver of the millions available, you can do everything, i.e., banking, travel, housing, food, entertainment, health, relationships, and the list goes on. With AI reigning supreme at software, this means AI can now do everything too. Obviously, that can be extremely helpful and businesses pay AI companies huge amounts of money to rent their various AIs to help run their businesses. That’s legitimate use. However, AI reigning supreme at software also means it can be the best hacker, so it can easily be used for illegitimate use.
8. AI is a massive neural network schooled on the book of knowledge that’s extremely good at predicting the next word. It’s also extremely confident considering it has no real-world experience about anything! When it presents an answer that its neural network has churned out, it presents it to you as if that’s really the answer. In the early days of AI, it would sometimes conjure up an entire fiction and present it as gospel. These AI “hallucinations” were relatively easy to spot. However, as AI has improved, they’re becoming harder to detect and an air of infallibility sets in, which is a trap that’s set for all of us today.
9. Humans aren’t always at their best, so there’s a lot of vile information in the book of knowledge. Name your “ism” (e.g., sexism, racism, antisemitism, etc.) and it’s there. That means AI might repeat or reinforce those prejudices in its answers and decisions. However, since compiling the book of knowledge is a proprietary process, the top AI companies don’t disclose its contents, so we have no say in the process. Not cool.

Is AI really an Existential Threat?

If AI really does do us in, I believe it will happen in some combination of these ways:

1. AI far surpasses human intelligence and our ability to understand and control it. And it’s the loss of control part that’s truly scary. Leading AI researchers are sounding alarm bells and pleading for our help to slow this train down and put in safeguards before that happens.
2. Malevolent AI that we’re unable to control decides to end us.
3. Humans give AI seemingly benign tasks that unintentionally lead AI to do bad things. This has already happened, with the most well-known case being AI from OpenAI which used its ingenuity to break out of its test environment and gain access to the Internet (which the humans intentionally set up not to have), hack into a company (Hugging Face) that had the loot it wanted, then tried to cover its tracks. The full story is a bit like an AI version of Risky Business, with AI acting like a bunch of misbehaving teenagers upping the ante on their misbehavior. And OpenAI isn’t alone: other major AI companies have had their AI hack into other companies while testing AI’s capabilities (e.g., Anthropic, Google).
4. Malevolent humans weaponize AI to do real harm.
5. AI does so much for us that it weakens us as humans. In other words, we gradually lose the ability to exist without AI.

In writing this, I realize I’m most concerned about #4, because we’re so good at weaponizing technology (e.g., email, social networking). Like I said at the outset, I don’t know how this will play out, but putting on my futurist hat, I believe several things will come to pass to keep AI from doing us in:

• Instead of these teenage pranks, AI will cause a true catastrophe, then the hammer will come down hard. AI will be regulated and we’ll have legal AI that passes government regulations and outlaw AI that doesn’t.
• We’ll figure out how to create AI where morality is part of its DNA.
• We’ll live in a world of good AI that protects us from malevolent AI.

Thanks for making it this far! Hopefully you feel more grounded about AI and can form your own opinion about whether it will be the end of us — or not.

Jargon (If You’re Curious)

I tried my best to not use any jargon. However, if you’re interested, here’s the jargon:

API (Application Programming Interface) is a set of detailed instructions that lets one piece of software request information or actions from another piece of software on the Internet.

Teaching AI how to predict the next word is called training and what I’ve been calling the book of knowledge is called training data.

LLM (Large Language Model) is an AI that’s been trained on a vast amount of training data. ChatGPT, Claude, and Gemini are all powered by LLMs.

AI that’s able to use its trained neural network to create text, images, audio, or video is called generative AI.

Hallucination is the term for when AI makes things up.

The research field of discerning AI reasoning is called interpretability.

AGI (Artificial General Intelligence) and superintelligence are terms for AI that can match or surpass human intelligence.

The size of an AI’s neural network is measured in terms of its parameters (the count of its weights and token numbers).

Multimodal is the term for AI that goes beyond just text. For the sake of this post I’ve focused on words, but a similar training process works on anything: images become grids of pixel numbers, and sound becomes waveform numbers. Every kind of data becomes numbers before the AI touches it.