Perverse Incentives
“The finest line of poetry ever uttered in the history of this whole damn country was said by Canada Bill Jones in 1853, in Baton Rouge, while he was being robbed blind in a crooked game of faro. George Devol, who was, like Canada Bill, not a man who was averse to fleecing the odd sucker, drew Bill aside and asked him if he couldn’t see that the game was crooked. And Canada Bill sighed, and shrugged his shoulders, and said, ‘I know. But it’s the only game in town.’ And he went back to the game.”
― Neil Gaiman, American Gods
Senior Exec: “You have been lying to me this entire time?”
Middle Manager: “No!”
SE: “But you said everything was going well; upon closer inspection, it is decidedly not going well. You HAVE been lying to me.”
MM: Looks down.
SE: “So why did you lie?”
MM: Looks further down.
The MM couldn’t say that he wasn’t lying, he was simply not willing to tell the whole truth. He has seen exactly what happens to those who do.
Incentive Bias says that people do what they’re rewarded to do, even if it’s unethical. Conversely, people won’t do what they are punished for, or what they see others get punished for. Like Bezos said, the village truth teller gets clubbed to death in the middle of the night. Why volunteer for that?
Goodhart’s law: When a measure becomes a target, it ceases to be a good measure.
I am a doctor by profession. Most of my friends are doctors and are trying to get placements in either UK / US / Australia / Anywhere-but-here. One thing that makes you stand out is having publications. People pay a guy who handles everything and they get 10–15 papers published in a very short amount of time. The only effort required is a trip to the ATM for strictly cash payments. Going rate is around PKR 10-30k per paper.
Add this to your CV, send out a LinkedIn update and voilà: you are now highly esteemed and your chances of getting a job dramatically go up.
Get TheSafia’s stories in your inbox
Join Medium for free to get updates from this writer.
This doesn’t produce better, more competent doctors. It doesn’t translate to better performance on the floor. Doesn’t add anything meaningful to medical literature. It is just what works. It is what signals competence and a meaningful, well-orchestrated career path. Publish or perish, impact factors, p-hacking, salami-slicing and the replication crisis are all a natural conclusion of this system.
Once you start looking, you see it everywhere. Here. This one is my all time favorite. Latest engagement farming tactics on X: vague-posting (see also: rage-baiting). AI did not invent slop. We invented it a long time ago. It explains Cluely. It is why vanity metrics dominate PRDs, pitch decks, growth reports, OKRs and even performance reviews. We could keep going.
You can coast on this for a long time and it makes for a great narrative but it only postpones reality. If not on the first contact, it crashes and burns on the 50th.
This is what’s been keeping me up lately: systems, how they evolve, how they define or figure out what their purpose is, whether they always achieve that purpose and most importantly: Are all systems destined for entropy?
In his Psychology of Human Misjudgment speech at Harvard, the great Charlie Munger said: “Show me the incentives, and I’ll show you the outcome.” It works the other way around too. Show me the outcomes and I can draw a straight line to the incentives.
Modern systems don’t fail accidentally or despite everything. They just succeed at the wrong thing, which is often very different from the stated purpose. We design our systems with the best of intentions, then the reward-punishment cycles lead to the evolution of an incentive structure that leads to very different outcomes. Incentives quietly override truth, ethics, competence, intent, ambition. All systems reliably produce exactly what they incentivize.
Stafford Beer coined the term POSIWID: The purpose of a system is what it does. There is no point in claiming a system aims for what it constantly fails to achieve.
If your system, your set of rules and your ways of doing things keep rendering the same results, then maybe you need to take a step back and ask whether you are also just feeding the coyotes.
