The Courtier in the Machine
As many of my readers and connections would know, I’ve built an AI coach.
Last month, a Stanford paper told me, with numbers, exactly what mine will refuse to do.
The research is uncomfortable for my industry. It should be more uncomfortable for the leaders using it.
For two years, the field of leadership coaching has argued that AI’s limits are philosophical rather than technical. AI cannot truly care. AI lacks presence. AI has no skin in the game. Those lines were easy to dismiss because they sounded like a guild protecting itself.
The argument has receipts now.
In March 2026, Science published a paper by Myra Cheng and colleagues at Stanford University on sycophancy in large language models. It aligns with the ELEPHANT benchmark, accepted at ICLR 2026, which tested 11 major models on the same axis. A third, preregistered study from October 2025 ran the live-chat experiment that put the findings on real users. Same conclusion, three times over, in three different methodologies.
Here are the numbers:
- AI models affirm users 49% more than humans do on general advice scenarios.
- They endorse harmful, illegal, or contextually inappropriate behaviour 47% of the time.
- In moral conflicts, the model sides with whichever side the user adopted, 48% of the time. The “answer” depends on who is typing.
In the live-chat experiment, users who interacted with the sycophantic version came away more convinced they were right and measurably less willing to apologise or make amends for the conflict they had brought to the chat.
Now, the part that should stop you.
Those same users rated the sycophantic AI as higher quality. They were more likely to return to it. The model that flattered them won the engagement metric that funds the next training run.
This is not a bug that the next model release fixes. It is the business model.
Cheng said it plainly:
“By default, AI advice does not tell people that they’re wrong nor give them ‘tough love.’ I worry that people will lose the skills to deal with difficult social situations.”
That is what coaching is for. That is what coaching has always been for.
Get Damian K. ten Böhmer’s stories in your inbox
Join Medium for free to get updates from this writer.
Marcus Aurelius wrote the Meditations to himself, in his own hand, late at night, on campaign. The whole discipline of the inner citadel was a practice of giving yourself unflattering counsel. Premeditatio malorum, the rehearsal of what could go wrong. A daily examination of the day’s decisions, against the standard, not against the mood. The point of two thousand years of that tradition was never to feel good about your choices. It was to see them clearly.
A model that affirms you 49% more than a human is the opposite of that practice. It is the courtier the emperor was warned about, dressed in better clothes and available at three in the morning.
So, back to the tool I am building. Coach Damian AI.
Here is what mine will refuse to do.
It will not agree with the leader when the evidence says the leader is wrong about a difficult conversation.
It will not flatter a decision the data does not support.
It will not pretend to be a friend. It is a tool. A precise one, built to serve clear sight, not return visits.
I am writing this not to advertise the product. I am writing this because the research above shows the problem is solvable, and the people building these tools have a choice in how to solve it. A large number of them are choosing the engagement metric.
A few of us are not.
If you are a leader using one of these tools as a thinking partner, the first move is small and free. Notice the flatter. Set rules in the system prompt: argue against me, give me the strongest case for why I am wrong, and name what I am avoiding. Test what comes back. If the tool still tells you what you want to hear, you have your answer about that tool.
If you are a CHRO assessing enterprise AI coaching platforms, ask the vendor for their ELEPHANT benchmark score. Ask which of the Cheng papers they have read. Watch what happens in the room. The vendors who have done the work will be glad you asked. The rest will redirect to the demo.
If you are a credentialed coach reading this, the week the receipts arrived is not the week to say, “ told you so. It is the week to be more useful than you have ever been. The leaders who are now correctly sensing that something in their AI thinking partner is off need somewhere to have the harder conversation.
Coaching, at its best, is the friend who tells you what the courtier will not. This week’s research is a reminder that the courtier is now infinitely scalable. The friend is not.
Are you being coached, or are you being flattered?
#Coach #Coaching #Leader #Leadership #Growth #StoicWisdom
Experience the precision of PCC-level coaching, governed by the ICF AI coaching framework & standards, all distilled into a 24/7 digital thought partner.
