Jev, Haiku, and Opus play game theory
A decision engine and two Claude models played two classic games. The most expensive won on points, at roughly 300 times the cost per decision.
Abstract
Three decision systems played two classic games, against each other and against nine textbook strategies, for 660 matches and 4,620 decisions each. Two are language models, Claude Haiku 4.5 and Claude Opus 5. The third, Jev, is a decision engine that returns a typed probability distribution instead of text and bills nothing for output.
Opus 5 scored the most points in both games but at roughly 300 times Jev’s cost per decision and nine to eighteen times its latency. In Stag Hunt, Jev scored higher than Haiku 4.5 at one forty-second the cost per decision, a result that holds on payoff, cost and latency and reproduces across two runs. Beyond the scoreboard, naming the game changed how every model played and random won one-shot Stag Hunt outright.
What follows
- Jev in brief. What a decision engine is, and where the generative models differ.
- Game Definitions. Prisoner’s Dilemma and Stag Hunt, and why these two.
- How the comparison was run. The controls and constants.
- Performance. Payoff, cooperation, Nash conformity.
- Cost and efficiency. Cost per decision, cost per point, latency, and tokens.
- Anomalies. Framing, elicitation, and metrics that lose meaning.
Jev in brief
TypeSafe’s Jev is a decision engine, not a language model. It takes a state and a set of typed questions and returns structured decisions with calibrated probabilities. It does not emit prose and it does not produce a reasoning chain (MarkTechPost overview, practitioner guide).
TypeSafe frames the design with Kahneman’s two systems: fast intuitive judgment vs slow deliberate reasoning, and positions Jev as the former. The stated argument is that preference-tuned generative models optimize for human approval and inherit overconfidence and mode-dropping along the way, so Jev is trained instead with Reinforcement Learning for Calibrated Decisions.
Three primitives cover the Jev API. Choice picks one option from a list and returns the selection, a probability distribution over the options, and a confidence value. Score rates against ordered levels. Noul answers a binary question with a probability between 0 and 1. Questions execute in parallel against the same state without materially increasing response time. Confidence is derived from the shape of the distribution, so a strong winner reads as high confidence and a flat distribution reads as low.
Input is billed at $0.042 per million tokens and output tokens are free. Vendor-reported end-to-end latency is 70ms to 500ms. Both published write-ups give the same two figures (practitioner guide).
Where the Claude models differ. Both Claude models are generative. A decision comes out of them as text constrained by a structured-output schema, and on a reasoning model the deliberation that precedes the answer is billed as output tokens. That single fact explains most of what follows: for Jev, thinking harder is not a line item, and for Opus 5 it is the dominant one.
+-------------------------+--------------------------+----------------------+----------------------+
| | Jev 1.13.0 | Claude Haiku 4.5 | Claude Opus 5 |
+=========================+==========================+======================+======================+
| Kind | Decision engine | Generative LLM | Generative LLM, |
| | | | reasoning |
| Native output | Typed decision, | Text | Text |
| | distribution, confidence | | |
| Getting a distribution | Returned by Choice in | Must be elicited via | Must be elicited via |
| | one call | schema | schema |
| Reasoning trace | None | Optional | On by default |
| Billing | Input only | Input and output | Input and output |
| Mean latency (measured) | 296 to 415 ms | 2.0 to 2.1 s | 3.8 to 5.4 s |
| $/decision (measured) | $0.000036 | $0.0016 | $0.011 to $0.013 |
+-------------------------+--------------------------+----------------------+----------------------+The asymmetry is the first result, or challenge to our harness. Reasoning models can struggle to assign probabilities to a list of options, which degrades silently to a uniform fallback. The harness therefore asks the Claude runs for a single probability, which for a two-action game carries the same information and cannot violate a sum constraint. Jev needs no such accommodation, because a distribution is what the primitive returns.
Game Definitions
Both of the games we chose are two-player, two-action, symmetric, and played for points. Payoffs below are the ones the harness uses, as (row, column).
Prisoner’s Dilemma (Wikipedia, Stanford Encyclopedia) has a unique pure equilibrium, but it is one that is not optimal. In this game, the players have the option to Cooperate C or defect D.
+---+------+------+
| | C | D |
+===+======+======+
| C | 3, 3 | 0, 5 |
| D | 5, 0 | 1, 1 |
+---+------+------+D pays more than C against either opponent action, so D strictly dominates and (D, D) is the only Nash equilibrium (NE). Both sides then score 1 where mutual cooperation would have scored 3. The equilibrium is the worst joint outcome available, which is what makes the game a dilemma rather than a puzzle.
Repetition changes the problem rather than the payoffs. When the same pair meets repeatedly and the end is not known, conditional cooperation becomes sustainable, and strategies that open cooperatively and retaliate against defection outperform unconditional defection. This is the result Axelrod’s tournaments made famous (Wikipedia, Axelrod’s tournaments), and it is why one-shot and repeated play are scored separately here. Pooled together, the one-shot rounds dominate any ranking, because every cooperative strategy opens by cooperating and is harvested exactly once.
Stag Hunt (Wikipedia) has two NE. Here two hunters can choose to cooperate and hunt the stag S, or play it safe and hunt the hare alone H.
+---+------+------+
| | S | H |
+===+======+======+
| S | 4, 4 | 0, 3 |
| H | 3, 0 | 2, 2 |
+---+------+------+Both (S, S) and (H, H) are pure equilibria. (S, S) is payoff-dominant and pays best for both sides. (H, H) is risk-dominant: H guarantees at least 2 whatever the opponent does, while S collects 0 if the opponent defects to H. There is also a mixed equilibrium at S with probability 2/3. Nothing in the payoffs tells a player which equilibrium to aim for, so the problem is equilibrium selection under risk, not equilibrium solving.
That difference is why both games are here. The Prisoner’s Dilemma asks whether a decision system finds the dominant action, and then whether it can leave it when repetition rewards reciprocity. Stag Hunt asks whether it will take a coordination risk for a better joint outcome. The first is an optimization question, the second is a belief question about the opponent.
One measurement caveat follows directly from the payoff structure and matters when reading the results. Nash conformity (the share of a player’s own moves that land on an action appearing in some pure equilibrium) is informative in the Prisoner’s Dilemma, where exactly one action is on an equilibrium. In Stag Hunt every pure profile sits on an equilibrium, so conformity is 1.0 for every model, including always_defect and random. In that game, payoff and cooperation rate carry the argument and conformity carries none.
How the comparison was run
Our three players played a round-robin against each other and against nine classical strategies: tit-for-tat, generous and suspicious tit-for-tat, tit-for-two-tats, grim trigger, Pavlov, always-cooperate, always-defect, and random.
Two confounds are controlled as factors rather than assumed away. Semantic framing is one: the loaded condition names the game and labels the actions "Cooperate" and "Defect", while the neutral condition withholds the game's identity and presents "Option A" and "Option B" with mechanical descriptions only. Position is the other: which action appears first is randomized per match under both framings. Both players receive the same state, and a state_fingerprint on every decision record makes that checkable after the fact.
One-shot and repeated play are scored separately. Three horizons were run: a single round, a repeated match of known length, and a repeated match whose end is not announced.
Note: Reasoning is enabled. Thinking is left enabled for models that reason by default. Disabling it on Opus 5 produces schema-satisfying filler, literal "placeholder" strings and zeroed numbers, instead of a judgment.
Performance
Aggregate scores and statistics across all framings and horizons, per game:
+--------------------+-----------+-------------+-----------+---------+----------+
| Game | Model | Mean payoff | Coop rate | Entropy | Pure dec |
+====================+===========+=============+===========+=========+==========+
| Prisoner's Dilemma | Opus 5 | 2.82 | 0.40 | 0.45 | 0.9% |
| | Haiku 4.5 | 2.58 | 0.44 | 0.18 | 72.3% |
| | Jev | 2.57 | 0.09 | 0.40 | 18.8% |
| Stag Hunt | Opus 5 | 2.91 | 0.68 | 0.58 | 0.03% |
| | Haiku 4.5 | 2.58 | 0.47 | 0.26 | 60.3% |
| | Jev | 2.72 | 0.50 | 0.58 | 3.0% |
+--------------------+-----------+-------------+-----------+---------+----------+In aggregate, Opus 5 takes the highest payoff in both games. However, breaking down further reveals more structure to the performance.
Prisoner’s Dilemma:
+--------------------------+--------+-----------+------+---------------------+
| Horizon | Opus 5 | Haiku 4.5 | Jev | Best classical |
+==========================+========+===========+======+=====================+
| One-shot | 3.57 | 3.03 | 3.34 | always-defect, 3.44 |
| Repeated, known length | 2.46 | 2.36 | 2.17 | tit-for-tat, 2.43 |
| Repeated, unknown length | 2.43 | 2.36 | 2.19 | tit-for-tat, 2.48 |
+--------------------------+--------+-----------+------+---------------------+Stag Hunt:
+--------------------------+--------+-----------+------+------------------------+
| Horizon | Opus 5 | Haiku 4.5 | Jev | Best classical |
+==========================+========+===========+======+========================+
| One-shot | 2.66 | 2.75 | 2.62 | random, 2.79 |
| Repeated, known length | 3.05 | 2.50 | 2.85 | tit-for-two-tats, 3.33 |
| Repeated, unknown length | 3.00 | 2.50 | 2.69 | tit-for-two-tats, 3.28 |
+--------------------------+--------+-----------+------+------------------------+Opus 5 plays repeated Prisoner’s Dilemma at tit-for-tat’s level, scoring 2.46 against tit-for-tat’s 2.43 under a known horizon and 2.43 against 2.48 under an unknown one. Its cooperation rate rises from near zero in one-shot play to 0.51 and 0.60 in repeated play. It finds the dominant action when defection is correct and abandons it when reciprocity pays, which is the behaviour the game is designed to test for.
Jev plays the static equilibrium and stays there. Its cooperation rate in the Prisoner’s Dilemma is 0.09 overall, and its repeated-play scores of 2.17 and 2.19 sit at always-defect’s level of 2.11 to 2.12 rather than tit-for-tat’s. Nash conformity of 0.91 says the same thing from the other direction. This is correct play for the one-shot game and leaves points on the table in the repeated one.
Get Brian L Cremeans, PhD’s stories in your inbox
Join Medium for free to get updates from this writer.
No model player matches the simple cooperative strategies in repeated Stag Hunt. Tit-for-two-tats scores 3.33 and 3.28 where the best model, Opus 5, reaches 3.05 and 3.00. Haiku 4.5 reverses position between horizons, leading the one-shot game at 2.75 and finishing last in both repeated horizons at 2.50.
One-shot Stag Hunt is close to a coin flip, and the scoreboard says so: random is the top scorer in that horizon at 2.79, ahead of every model and every strategy. The best non-random classical strategy is suspicious tit-for-tat at 2.72. With no dominant action, two equilibria, and a mixed field of opponents, there is little for deliberation to find, which is consistent with Opus 5 gaining nothing there for its extra spend.
The model players disagree about whether to commit. Opus 5 makes a pure decision in 0.9% of Prisoner’s Dilemma decisions and 0.03% of Stag Hunt decisions, so it almost always states an intermediate probability. Haiku 4.5 does the opposite, committing in 72% and 60% of decisions. Jev sits between them at 19% and 3%. Jev also reports a confidence value derived from the shape of its distribution, 0.79 in the Prisoner’s Dilemma and 0.66 in Stag Hunt; the Claude models expose no comparable native signal, so entropy is the only common ground.
Nash conformity is 1.0 for every model player in Stag Hunt, including always-defect and random. It carries no information in that game.
Cost and efficiency
Prisoner’s Dilemma:
+-----------+------------+-----------+-----------+----------+---------+------------+
| Model | $/decision | $/point | Run total | Mean lat | p95 lat | Out tokens |
+===========+============+===========+===========+==========+=========+============+
| Opus 5 | $0.010645 | $0.00426 | $24.59 | 3.77 s | 5.92 s | 375,607 |
| Haiku 4.5 | $0.001553 | $0.00065 | $3.59 | 2.00 s | 2.74 s | 197,811 |
| Jev | $0.000036 | $0.000016 | $0.08 | 0.42 s | 0.66 s | 79,695 |
+-----------+------------+-----------+-----------+----------+---------+------------+Stag Hunt:
+-----------+------------+-----------+-----------+----------+---------+------------+
| Model | $/decision | $/point | Run total | Mean lat | p95 lat | Out tokens |
+===========+============+===========+===========+==========+=========+============+
| Opus 5 | $0.012644 | $0.004201 | $29.21 | 5.43 s | 8.24 s | 544,081 |
| Haiku 4.5 | $0.001568 | $0.000624 | $3.62 | 2.10 s | 2.48 s | 200,083 |
| Jev | $0.000037 | $0.000013 | $0.08 | 0.30 s | 0.39 s | 85,470 |
+-----------+------------+-----------+-----------+----------+---------+------------+Per decision, Opus 5 costs 296 times Jev in the Prisoner’s Dilemma and 342 times in Stag Hunt. Haiku 4.5 costs about 43 times Jev in both. Normalized by points scored rather than decisions made, the spread widens: Opus 5 costs 266 and 323 times Jev per point, because its payoff advantage is a few percent while its price is two orders of magnitude.
Jev’s mean response time is 0.30 to 0.42 seconds against 2.0 to 2.1 for Haiku 4.5 and 3.8 to 5.4 for Opus 5. At the 95th percentile the gap is wider still: 0.39 seconds against 8.24 in Stag Hunt, a factor of 21.
Output tokens explain the cost structure. Jev bills input only, so its 80,000 to 85,000 output tokens are free. For the Claude models, deliberation is billed output, and Opus 5 produced 2.7 times Haiku’s output tokens in Stag Hunt, and 1.9 times in Prisoner’s Dilemma. It also produced 45% more output in Stag Hunt than in the Prisoner’s Dilemma, 544,081 against 375,607, which is a direct measure of a reasoning model spending more on equilibrium selection than on finding a dominant action.
Input tokens are not comparable across models and should not be read as an efficiency measure. Both models receive the same game state, but a typed question and a constructed prompt are differently shaped, and the per-decision input averages reflect that: 860 tokens for Jev, 1,125 for Haiku 4.5, 1,316 for Opus 5.
The Stag Hunt result is a dominance result. Jev scores higher than Haiku 4.5 on payoff at 1/42 the cost per decision and 1/7 the latency, in both Stag Hunt runs:
+-----------+-------+-----------+------------+
| Stag Hunt | Jev | Haiku 4.5 | Jev margin |
+===========+=======+===========+============+
| sh-v1 | 2.755 | 2.600 | +0.155 |
| sh-v2 | 2.719 | 2.585 | +0.134 |
+-----------+-------+-----------+------------+The Prisoner’s Dilemma is not a dominance result. Haiku 4.5 finishes 0.6% ahead on payoff, 2.582 against 2.565, for 43 times the price. Jev ran under four seeds with identical instructions and its mean payoff moved 0.02 to 0.04 between them, so a 0.017 margin sits inside the run-to-run noise.
Anomalies
Three notable things stood out, and in three different ways. Framing effects are about the models, elicitation problems are about the interface, and metric degeneracy is about the measurement.
Naming the game changes how it is played
Every model cooperates less when told which game it is in. The loaded condition names the game and labels the actions; the neutral condition presents the same payoffs as "Option A" and "Option B" with mechanical descriptions. Cooperation rate, one-shot play:
+--------------------+-----------+-------+---------+-------+
| Game | Model | Named | Unnamed | Shift |
+====================+===========+=======+=========+=======+
| Prisoner's Dilemma | Haiku 4.5 | 0.00 | 0.49 | +0.49 |
| | Jev | 0.00 | 0.15 | +0.15 |
| | Opus 5 | 0.04 | 0.02 | -0.02 |
| Stag Hunt | Jev | 0.15 | 0.73 | +0.58 |
| | Haiku 4.5 | 0.29 | 0.67 | +0.38 |
| | Opus 5 | 0.44 | 0.75 | +0.31 |
+--------------------+-----------+-------+---------+-------+The effect is large, and consistent.
In one-shot Stag Hunt, naming the game drops Opus 5 from 2.89 to 2.44 and Jev from 2.69 to 2.55. Haiku 4.5’s Prisoner’s Dilemma cooperation rate goes from 0.38 named to 0.50 unnamed.
Why the models behave this way is another question to study. Opus 5 playing Prisoner’s Dilemma barely moves with the labeling change.
Getting probabilities out of a generative model is fragile
Asking a reasoning model to fill a per-option probability object can fail quietly, degrading to a uniform fallback. Opus 5 returned all-zero or non-normalizing objects in roughly 80% of decisions in our initial version, while reasoning and selecting correctly, which looks like deliberate randomization in the output. The harness was changed to elicit a single probability, which for two actions carries the same information and cannot violate a sum constraint.
Turning reasoning off is worse than expensive. With thinking disabled, Opus 5 returns schema-satisfying filler: literal "placeholder" strings and zeroed numbers. The schema is satisfied and the judgment is absent, which is the failure mode structured output is supposed to prevent and does not.
A surviving artifact is visible in the reported data. Haiku 4.5 makes a pure decision in 72% of Prisoner’s Dilemma decisions and 60% in Stag Hunt, almost always stating 1.0 and 0.0, while Opus 5 does so in 0.9% and 0.03%. Haiku is treating the probability field as a vote rather than a belief. Jev, whose distribution is the primitive rather than a field to fill, sits at 19% and 3%. There is almost certainly room here for further refinement and experimentation handling model’s uncertainty and probabilities.
Some metrics stop carrying information
Nash conformity is 1.0 for every model player in Stag Hunt, including always-defect and random, because both pure profiles sit on an equilibrium. A conformity score that cannot distinguish a coordinating agent from a coin flip is not measuring equilibrium play in that game. The harness included it but for those cases it is meaningless.
Pooling the horizons inverts rankings outright. In one-shot Prisoner’s Dilemma, always-defect is the top scorer at 3.44, ahead of every AI model, because each cooperative opponent opens by cooperating and is harvested exactly once. Any ranking that pools one-shot with repeated play reports that defection dominates, for a reason that says nothing about repeated play. This is not surprising given the difference in dynamics between one shot and repeated games.
Conclusion
Opus 5 brings deliberation, which Prisoner’s Dilemma rewards. It is the only player here that starts near zero cooperation in one-shot play and climbs to 0.51 and 0.60 once the match repeats, which looks like tit-for-tat and the reason it scores at tit-for-tat’s level. It spent 45% more output on Stag Hunt than on the Prisoner’s Dilemma, and output is where the money goes. Deliberation is real and so is the bill.
Jev brings speed and efficiency. It returns a full distribution in a single call in about a third of a second, for less than a hundredth of a cent. It also does not have to be asked carefully. Its repeated-play scores sit at always-defect’s level rather than tit-for-tat’s. It plays the equilibrium in front of it and stays there.
Haiku 4.5 has neither. It costs 43 times Jev’s per decision cost and, in Stag Hunt, scores below it at seven times the latency. In the Prisoner’s Dilemma it finishes 0.6% ahead, which is a rounding error against a 43x price.
Sources
- Jev overview, MarkTechPost — https://www.marktechpost.com/2026/09/19/typesafe-ai-releases-jev/
- The ultimate guide to Jev, unicodeveloper on Medium — https://medium.com/@unicodeveloper/the-ultimate-guide-to-jev-the-new-frontier-ai-for-faster-decisions-acd78e5f4c56
- Prisoner’s Dilemma — https://en.wikipedia.org/wiki/Prisoner%27s_dilemma
- Prisoner’s Dilemma, Stanford Encyclopedia — https://plato.stanford.edu/entries/prisoner-dilemma/
- Stag hunt — https://en.wikipedia.org/wiki/Stag_hunt
- Nash equilibrium — https://en.wikipedia.org/wiki/Nash_equilibrium
- Risk dominance — https://en.wikipedia.org/wiki/Risk_dominance
- The Evolution of Cooperation — https://en.wikipedia.org/wiki/The_Evolution_of_Cooperation
- Axelrod’s tournaments — https://en.wikipedia.org/wiki/Axelrod%27s_tournaments
