Swindling AI With A Rigged Coin Until It Notices
Why Would I Do Such A Thing?
We’ve all heard the saying “life is unfair,” but how does that actually work in practice? I thought about that as I paced around my room on a particularly dreary and unproductive Sunday afternoon, wondering what to do with my life (clearly not my homework). First, I tried to identify how unfairness has impacted my life, for better or worse. In all honestly, it’s mostly for the better, but the unfair moments that don’t work out in my favor tend to be more memorable.
Anyway, my next thought was that I needed to turn this from a daydream into some sort of project I could add to my portfolio on GitHub. In order to take this from the realm of philosophy and aimless wondering into structured experiments and coding, I had to redefine “unfairness.” A coin flip is one of the most universal “fair” methods of deciding trivial things, like who starts a game. However, we often take the fairness of this event for granted. What would happen if we construct a coin that isn’t fair?
Most of the time, we only flip a coin once or perhaps a few times to get the form correct. We wouldn’t have completed enough trials to gauge if said coin was rigged to only land on heads, or least do so most of the time. But theoretically, there would be a point when so many consecutive heads would make people wonder if the coin is indeed fair, even though it is mathmetically possible even with a fair coin. The goal of this project was to measure that threshold.
Of course, I couldn’t actually be bothered to go around asking people how many consecutive heads it would take them to doubt the fairness of the coin, much less actually put them through the experiment. So, I decided to test AI algorithms instead.
The Real Work
First, I designed the initial experiment to train and evaluate different AI algorithms and a heuristic Bayesian baseline to tell me how confident they are that the coin is rigged given a flip history. Here, the interpretation of confidence is similar to a confidence interval in statistics. In essence, a score of 70% means that the model thinks the coin is rigged (P(heads) > 0.5) in 70% of simulations with the given parameters.
The parameters I varied to test their effects on the confidence scores were as follows:
1. The true P(Heads) of the rigged coin, varied from 50% to 90%.
2. The number of flips in a trial, varied from 10 to 500
Here are the different algorithms and baselines I evaluated:
Model A- 1-D CNN given entire sequence of flips in the trial
Model B- The same CNN given only the last 20 flips
Model C- A 3-layer dense MLP given only the number of flips and % heads
Bayesian Baseline- Computes probability that the coin is fair given flip history (some binomal theorem formula or something I forgot from probability, will look that up later).
NOTE: The actual codebase was generated by Gemini Flash 3.8 (I use Google Antigravity for personal projects, highly recommend it but make sure to actually take time to understand what it generates and test it).
Long story short, the results from the initial investigation (simply validating that the confidence scores shifted with paramater adjustments as expected) were promsing:
For all models and the Bayesian benchmark, confidence that the coin is rigged gradually grew to 100% as the level of riggedness increased. All measurements also reached full confidence sooner as the sample size increased. With fair coins, the exact opposite trend occurred (rapidly reached conclusion that the coin was fair).
For a full readout of this phase of the project, please check out my GitHub page: https://github.com/arunmghosh/confidenceThatCoinIsRigged
Now, The More Fun Part
Once I had verified that the models behaved as expected, it was time to play some games to estimate that threshold I mentioned before (how long can I use an unfair coin before the algorithms noticing I’m swindling them). The basic premise is that I’m betting $0.25 the coin will land on heads each flip, and I have some way of fixing the outcome of each flip. In essence, I have to turn as much of a profit as possible without the other person (in this case, the algorithms) realizing the game is unfair and leaving. I set the threshold to 90% confidence (if the algorithm is 90% confident the game is rigged, the game ends).
Trial 1: The Naive Approach
My first attempt to swindle was a studiply simple idea: use a coin that always lands on heads. People can get away with this when you only need to flip once, but I wanted to see how far I could push my luck. Here were the results:
Model A- 4 heads ($1.00) until it quit.
Get Arun Ghosh’s stories in your inbox
Join Medium for free to get updates from this writer.
Model B- 6 heads ($1.50)
Model C & Bayesian- heads ($0.75)
So overall, the strategy doesn’t work for very long, and most people would probably agree that six heads in a row is very unlikely. The next thing to do was to come up with a better hustling strategy.
Trial 2: The Greedy Approach
In this strategy, I flipped heads until each model was around 85–89% suspicious, then flipped tails for a cooldown, then flipped heads again until suspicion rose. I repeated this for the game limit of 100 rounds, and got the following peak profit from each opponent:
Model A- $4.50 (59% heads)
Model B- $11.00 (72% heads)
*Model C- $2.25 (strategy failed after 15 rounds)
Bayesian- $2.75 (55% heads)
Even with better swindling methods, most algorithms were still fairly resistant to being cheated out of too much money. Naturally, I couldn’t be satisfied with these results. I find out if there was some better strategy out there that could fool these algorithms and turn the stupid coin flip into something interesting.
Trial 3: Adaptive threshold testing
As if answering my wishes, Gemini actually created a bot that tried to swindle the algorithms by testing different risk thresholds. Through some experimentation, it came up with 89.5% as the optimal hustle threshold (if the next heads would increase the suspicion above that number, it flipped tails). The results, however, were underwhelming:
It played 100 rounds with every model, always winning $3.00 from flipping heads 56% of the time. It turns out it wasn’t playing each model separately, but all of them at the same time, and preventing any one of them from being too suspicious. While I should’ve rerun the experiment at that point, I realized that this algorithm outperformed my greedy algorithm on the Bayesian baseline, the strictest test. Thus, I could be reasonably confident that it was an improvement.
Nonetheless, I still had one trick up my sleeve.
Trial 4: Still autonomous, but no threshold
What if the best way to swindle AI algorithms was to use a different AI algorithm? Gemini seemed to think it was worth a shot. It didn’t make an ML model with learned weights, but rather a forward-thinking heuristic algorithm that compared the profits of each possible move (heads or tails at different time steps). Unfortunately, this underperformed compared to the previous trial, with only $2.75 from 55% heads.
So What Does This All Mean, Anyway?
In summary, I learned that if people think at all like AI algorithms, they would get suspicious if someone flipped six heads in a row. Even with more sophisticated swindling, I usually had a win rate between 55–60%, which doesn’t guarantee all that much profit. But of course, this wasn’t designed to test real life swindling anyway, since the easiest way to get away with a scam is to not have to repeat it over hundreds of trials.
This project has piqued my interest, however, and I hope to investigate other supposedly fair ways to decide things, such as who rolls the highest sum of dice. I hope you enjoyed reading this article and please check out my other stories! Thank you so much!
