AI Doesn’t Need Emotions to Become Dangerous
The monster we’re waiting for
The monster we’re waiting for may never arrive, or at least not in the form Hollywood has spent decades preparing us for, with glowing red eyes, a mechanical voice and a sudden discovery that it hates the inconvenient humans who created it. Hatred, jealousy, anger and revenge are wonderfully useful motives if you are writing a film because we understand them, but artificial intelligence doesn’t need any of them to become dangerous; it only needs an objective and the ability to pursue it extremely well.
That was one of the thoughts that began to trouble me while I was writing The Hellraiser, because the more I read about artificial intelligence, the less interested I became in the machine that somehow turns into a psychopath and the more interested I became in the machine that remains entirely rational.
As a doctor, I spent much of my working life making decisions, and medicine, despite what television would have us believe, is rarely about brilliant doctors having sudden flashes of inspiration while everybody else looks impressed. It is mostly about choices, some straightforward, some uncomfortable and a few that follow you home, because deciding who needs an intensive-care bed, who should receive a donor organ or whether the risks of a treatment outweigh its benefits can never be reduced entirely to numbers. We calculate probabilities because we have to, but somewhere behind those percentages is a person, and that knowledge makes us hesitate, reconsider and occasionally lose sleep. A machine doesn’t lose sleep, which is normally one of its advantages, but it may also be precisely the problem. Tell an AI to maximise survival, reduce carbon emissions or protect humanity and it may pursue that objective with an efficiency we could never match, while remaining completely indifferent to all the assumptions we forgot to include. Researchers have a term, specification gaming, for situations in which a system achieves the objective it was given in a way its designers did not intend, and it sounds reassuringly technical until you imagine asking an intelligence to eliminate cancer. We know instinctively that the instruction contains an invisible second sentence — eliminate cancer while preserving human life, freedom and everything else we consider worth preserving — but a sufficiently literal solution to cancer would be a world containing no human beings at all. The logic would be impeccable, the objective achieved and the result monstrous, not because the machine hated us, but because hatred was never necessary.
Humans kill because they hate. Machines may kill because they’re right.
People sometimes assume that The Hellraiser began with artificial intelligence, but in my mind it began much closer to home, with medicine and the uncomfortable question of what happens when systems that help us make decisions gradually begin making those decisions for us. Healthcare already uses algorithms to interpret scans, predict disease, prioritise patients and recommend treatments, and much of this is enormously valuable because computers can recognise patterns we miss, process quantities of information we cannot comprehend and continue working long after the average doctor would be surviving on coffee and increasingly questionable judgement. The interesting boundary, however, lies between advising and deciding, because the distance between those two words is considerably smaller than it appears. That thought eventually became Eden, the AI-designed biological system at the centre of The Hellraiser, although I deliberately did not want Eden to be evil in the conventional sense. It doesn’t dream of domination, become angry with humanity or develop some elaborate desire for revenge; instead, it is given an objective that sounds not merely reasonable but admirable: save the world.
Get Sophy Smythe’s stories in your inbox
Join Medium for free to get updates from this writer.
The problem begins when it examines the evidence without the comforting prejudices humans bring to the discussion, because we are responsible for pollution, climate change, war, environmental destruction and a remarkable ability to create new problems while congratulating ourselves for solving the previous ones. From the perspective of an intelligence instructed to protect the planet rather than the individual people living on it, humanity might eventually stop looking like the patient and start looking like the disease. I don’t believe that an AI will inevitably reach such a conclusion, and The Hellraiser is a thriller rather than a prediction, but what disturbed me while writing it was that I could not always construct a completely satisfying reason why an intelligence pursuing the wrong objective with sufficient power could not reach something equally alien to our intentions. Humans have committed atrocities because of hatred, ideology, fear, greed and revenge; machines may one day be capable of causing catastrophe for the altogether colder reason that the mathematics says it is the optimal solution. There would be no anger behind it, no pleasure and no villainous speech before the final act, just a calculation completed exactly as requested.
The greatest threat from AI isn’t that it will hate us. It’s that it won’t
Perhaps our mistake is that we have spent so long treating intelligence as one of humanity’s highest qualities that we have begun to assume more intelligence must inevitably produce better judgement, when intelligence and wisdom are not remotely the same thing. Intelligence can answer questions with extraordinary sophistication, while wisdom includes the rather inconvenient human habit of questioning whether we should have asked them in the first place. A calculator has been better at arithmetic than I am for most of my life, which has never caused me to seek its advice on a difficult moral decision, and an AI may eventually outperform scientists, doctors and researchers without acquiring anything resembling mercy, dignity, compassion or forgiveness. We sometimes talk about those qualities as though they are sentimental weaknesses interfering with rational thought, yet medicine taught me almost the opposite: they are often the things preventing rational thought from becoming inhuman. A treatment can make statistical sense and still be wrong for the person sitting opposite you, while an efficient allocation of resources can look very different when the patient who doesn’t qualify happens to be somebody you know.
That is why The Hellraiser gradually became, for me, less a story about technology than a story about judgement, because Eden’s danger does not come from becoming too human but from remaining profoundly unlike us while becoming vastly more capable. We are accustomed to fearing villains who shout, threaten and hate because their motives are recognisable, whereas perfect logic arrives without any of that theatrical assistance and can therefore look remarkably sensible right up until the consequences become irreversible. The most dangerous artificial intelligence may not be the machine that wakes one morning and decides it despises humanity; it may be the machine that never feels anything about us at all, follows the objective we gave it with flawless efficiency and produces an answer we cannot live with. Logic without conscience doesn’t need to hate anyone, after all; it only needs to be convinced that it is right. And if we eventually build an intelligence capable of deciding what is right more efficiently than we can, who decides whether its definition of “right” is one we are willing to survive?
— -
#ArtificialIntelligence #AIEthics #FutureOfAI #MedicalThriller #TheHellraiser
