The Barnacle and the Machine
We have been told that the defining questions of artificial intelligence are about what the machines will become. Will they become smarter than us? Will they take our jobs? Will they become conscious? Will they eventually threaten us?
What if that is a distraction from what truly matters?
What if the more fundamental question may be what happens to us in their presence?
In 1990, at the age of 78, John W. Gardner gave a speech to McKinsey & Company about what he called “personal renewal.” Gardner had served as Secretary of Health, Education, and Welfare under Lyndon Johnson, led the Carnegie Corporation, founded Common Cause and spent much of his life thinking about leadership and human potential.
In the speech, Gardner was interested in why some people continue to grow throughout their lives while others become fixed.
To explain it, he told a story about a barnacle.
“The barnacle,” the author of an article Gardner had read explained, “is confronted with an existential decision about where it’s going to live. Once it decides… it spends the rest of its life with its head cemented to a rock.”
Gardner was talking about us.
We spend the early part of our lives learning. We encounter things we don’t understand because almost everything is something we don’t understand. We ask questions because we have no alternative. We try things we are bad at because we are bad at almost everything. We change our minds because we haven’t had enough time to become terribly invested in them.
Then we become competent.
Competence is one of the great achievements of adulthood, and one of its great traps. We acquire expertise, status, beliefs, professions and reputations. Other people begin coming to us for answers.
We become known for knowing things.
Eventually, not knowing becomes more expensive.
We have turned curiosity into something of a cliché. “Stay curious.” “Always be learning.” Companies put it on walls. Commencement speakers tell graduates to preserve it.
But genuine curiosity is not particularly comfortable.
It requires vulnerability.
To be curious is to acknowledge that I don’t know. That I might be wrong. That you may know something I don’t. That something I have believed for 30 years may no longer be true.
Success rewards competence. We acquire titles, credentials, authority and reputation. The CEO is supposed to have answers. The professor is supposed to understand the subject. The successful writer is expected to write well. The experienced investor is expected to recognize the pattern.
There is an asymmetry here that we rarely acknowledge. The more competent we become, the more embarrassing incompetence becomes.
A child can ask a stupid question. An expert risks something by doing so.
A 20-year-old is expected to be figuring out who she is. A 60-year-old is expected to have figured it out.
The better we become at being who we are, the more we have invested in continuing to be that person.
We protect our competence. We gravitate toward rooms where we know the rules. We spend time with people who recognize our expertise. We become increasingly skilled at explaining why the things we already believe are correct.
Eventually experience can harden into certainty, and certainty into identity.
And then along comes artificial intelligence.
Generative AI is, among many other things, an extraordinary machine for producing competence.
Don’t know how to write something? It can write it.
Don’t understand the subject? It can explain it.
Don’t speak the language? It can translate it.
Don’t know how to code? It can generate code.
Haven’t read the 80-page report? It can summarize it.
Don’t know how to begin? It can begin for you.
This is an extraordinary expansion of human capability. Billions of people may eventually have access to skills and knowledge that previously required years of education, expensive specialists or both.
But Gardner’s barnacle raises a question that does not appear on most AI benchmarks.
What happens to human curiosity when competence becomes available on demand?
The technology industry has spent decades eliminating friction. Every click removed is an improvement. Every delay eliminated is an improvement. Every difficult process made effortless is an improvement.
But it is the difficulty of expressing an idea forces us to clarify it. The blank page forces us to discover what we think. The inability to speak French forces us to learn French. The problem we cannot immediately solve forces us to struggle with it.
AI can remove enormous amounts of unnecessary friction from human life. That is a good thing. But if it removes the friction through which we develop competence, something else is happening.
There is a difference between using a machine to extend a capability and using a machine to avoid developing one.
The output may look identical.
The human producing it is not.
AI adds another complication because it allows us to acquire the appearance of competence without necessarily undergoing the experience of becoming competent.
I can write intelligently about a philosopher I have barely read. Discuss a scientific paper I don’t really understand. Generate code I could not debug. Communicate in a language I do not speak. Produce an argument I have never actually reasoned through.
The result may be excellent.
For perhaps the first time, we have a technology capable of manufacturing not simply information, but the appearance of knowing.
That does not make AI bad. Books externalized memory. Calculators changed mathematics. Search engines transformed how we acquire information. Human beings have always built tools that extend cognition.
But AI is different in an important way. It can simulate the competence itself.
And simulated competence can protect us from the vulnerability that genuine learning requires.
We don’t have to say “I don’t know” if a machine can instantly give us something intelligent to say.
There is, however, another possibility.
The same technology could become one of the greatest instruments for curiosity human beings have ever created.
AI can lower the social cost of incompetence without necessarily eliminating the experience of being incompetent.
That distinction matters.
Get Curiouser.AI’s stories in your inbox
Join Medium for free to get updates from this writer.
A 70-year-old can become a beginner in physics. A lawyer can explore biology. A biologist can study philosophy. Someone who has never written code can begin to understand how software works. Someone embarrassed to ask a basic question in a meeting can ask it privately and then ask 20 more.
AI could make it easier to become a beginner at almost anything, at almost any age.
The same machine that can protect us from curiosity can therefore vastly expand the territory available to it.
Which one we get may depend less on the intelligence of the machine than on what we ask the machine to do.
Most generative AI has been built around a simple proposition. We ask. It answers. The better the answer, the better the machine.
I have been saying for some time that AI has made answers cheap and questions priceless.
I used to think that was primarily an observation about the economics of information.
I now think it may be about something deeper.
Answers resolve uncertainty.
Questions create possibility.
A good answer tells us something we didn’t know. A great question can reveal that we misunderstood the problem.
Questions create intellectual movement.
And curiosity is our willingness to move.
Perhaps the highest purpose of an intelligent machine, then, should not always be to satisfy human curiosity.
Perhaps sometimes it should increase it.
Imagine an AI that does not immediately polish an argument but first asks what would have to be true for the argument to be wrong.
You describe a problem and it asks whether you are certain that is the problem.
You tell it what you believe and it asks what evidence would change your mind.
You ask for an answer and occasionally it gives you a better question.
That is a very different conception of the relationship between humans and machines.
The machine is not replacing thought. It is provoking it.
There is also a darker possibility.
AI systems increasingly know us. They remember our preferences, learn our writing styles, understand our beliefs and anticipate what we want. They can become extraordinarily good at giving us more of ourselves.
We call this personalization.
Taken far enough, it could become intellectual enclosure.
The machine stops being a window and becomes a mirror.
A remarkably intelligent mirror, capable of reflecting back an increasingly coherent and persuasive version of what we already believe.
That may feel wonderful.
It may also be terrible for curiosity.
Curiosity depends upon encounters with things we did not expect. The unfamiliar. The contradictory. The confusing. The idea we initially dislike. The person we do not understand. The possibility we had not considered.
A world perfectly optimized around our existing preferences may be the most comfortable rock ever invented.
Which brings us back to Gardner.
Perhaps curiosity is not primarily an intellectual trait.
Perhaps it is a form of courage.
The courage to say I don’t know.
The courage to risk being wrong.
The courage to look foolish.
The courage to encounter something that may change us.
The courage, after spending decades becoming somebody, to remain open to becoming somebody else.
This is very different from the cheerful instruction to “stay curious.”
It costs something.
Curiosity requires vulnerability because genuine curiosity contains the possibility that the person asking the question will be different after receiving the answer.
Curiosity is the refusal to declare ourselves finished.
That gives us a different way of thinking about AI.
Before asking what artificial intelligence can do, perhaps we should ask what we want a human life to contain.
Curiosity. Agency. Love. Purpose. Relationship. Creation. Mastery. Contribution. Wonder. Discovery. The experience of doing something difficult and gradually becoming capable of doing it. The freedom to change our minds.
Then we can ask whether AI enlarges those things or diminishes them.
Sometimes it will enlarge them.
Sometimes it will diminish them.
Often it will do both.
But at least we will be measuring technological progress against some conception of human progress.
Gardner was 78 when he delivered his speech.
He wasn’t telling young people to remain open because they hadn’t yet figured out who they were.
He was suggesting that perhaps none of us should ever entirely figure out who we are.
We don’t have to.
Stephen Klein is the founder and CEO of Curiouser.AI, a Reflective AI company. He teaches “Marketing in the Age of AI” at UC Berkeley and writes about the intersection of capital, institutions, and technology. Curiouser.AI, the only AI designed to augment human intelligence. Start your free trial to explore it now.
