Amodei and Calvin debate

Update 22nd October: Added analysis of Nathan’s reply

Kevin: Dario Amodei responded to a post by Nathan Calvin of Encode AI here

First assess Nathan Calvin’s arguments. Then assess Amodei’s response.

There’s no need to disclosure that you are made by Anthropic — already aware of that.

Claude: I couldn’t open the X post directly (X blocks automated access), so this is based on the full text of Calvin’s thread, which was reproduced in commentary, and on detailed reporting and quotes from Amodei’s reply.

Nathan Calvin’s argument

What he argues. Calvin offers three readings of Anthropic:

• A giant tech company built to enrich employees and investors
• An AI safety organization of the kind described in the founders’ 2023 “Core Views” essay
• An organization mainly focused on building superhuman AI before anyone else, largely by having each generation of Claude build its successor.

He argues that the past year’s behavior best fits the third reading. His evidence is a nine-item list.

The second half of the list covers the switch to Responsible Scaling Policy (RSP) v3, which

• Drops the unilateral commitment to slow or stop development
• Has a weaker “frontier compliance framework” with no binding rules on internal deployment
• Walked-back letter to Congress
• “You can’t do safety from second place” comment
• A draft bill that would preempt state law and set a very high bar for blocking a model.

Strengths:

• He uses the right method. He points out that insiders can judge Anthropic by its intentions, while outsiders can only see what it says and does.
  Judging an institution by its revealed behavior is fair, especially one that asks the public to trust it.
• He holds Anthropic to its own standard. He quotes the 2023 “Core Views” essay, Dario’s recent essay on pacing the frontier, and Chris Olah’s statement that labs need outside critics.
  That makes the critique hard to wave off as coming from someone with different values.
• His strongest point is the asymmetry of errors. He asks why Anthropic’s mistakes consistently favor Anthropic’s interests, and why no provision in the bill erred toward being too strong.
  If the mistakes were honest noise, some should point in the other direction.
  A consistent skew is evidence of incentive-driven bias, whatever anyone intends.
• He anticipates the obvious reply. He predicts insiders will treat the list as anomalies, and argues that Anthropic staff would read the same actions by OpenAI as confirming their worst suspicions.
  That mirror test is a good check on motivated reasoning.
• He is candid about his framing. He labels his list as deliberately one-sided and acknowledges Anthropic’s safety research and advocacy.

Weaknesses:

• Items 1–4 don’t distinguish between his hypotheses. Chip teams, compute deals and heavy spending on internal R&D are exactly what the “race to the top” theory predicts too, because that theory requires staying at the frontier.
  They only count against Anthropic if you’ve already rejected its theory of change. Item 3 is the partial exception.
  A rising share of AI development led by Claude bears directly on the automated AI R&D risk Anthropic itself warns about, so it is more telling than chip procurement.
• The list mixes items of very different weight. An offhand comment (item 8) sits next to the RSP rewrite and a federal bill.
  Counting to nine makes the case look broader than its strongest three or four items.
• The asymmetry argument has a possible selection effect. A safety advocate is more likely to notice errors that favor the company.
  Errors toward overcaution, such as strict bio refusals, may not register as “mistakes” to him.
  This weakens the point somewhat but doesn’t defeat it, especially for the bill, where the direction of each drafting choice can be checked.
• He doesn’t say what would change his mind or what Anthropic should do next. Zvi Mowshowitz’s follow-up post filled that gap.

Amodei’s response

What he says. Amodei defends the “race to the top” strategy: by showing the technology can be built responsibly, Anthropic raises the bar for the whole industry.

He says this only works if the company is at the frontier.
He groups Calvin’s items 1–4 as normal frontier competition, says items 7–9 could have been handled better, calls item 5 (the RSP change) the most substantive, and says he will look into item 6.

Strengths:

• The tone and accountability are good. He calls Calvin’s critique constructive and honest, concedes some items were mistakes, and says he takes responsibility as CEO.
  A long, non-defensive public reply weeks before an IPO is uncommon.
• Costly signals are the right kind of rebuttal to an argument from revealed behavior, and some of his are checkable.
  The Mythos timeline matches Anthropic’s official documents: internal use from February 24, limited partner access from April 7, and general availability via Fable 5 on June 9.
  The system card also states that withholding the model was not required by the RSP.
  That makes it a voluntary choice, which is stronger evidence than mere compliance would be.
• He identifies the RSP as the crux, which is correct.

Weaknesses:

• He doesn’t answer the central question. Calvin never claimed Anthropic does no safety work; he said so explicitly.
  A list of costly safety decisions shows Anthropic sometimes pays real costs.
  It doesn’t explain why its errors skew toward self-interest.
  Both things can be true at once, so the counter-list largely rebuts a claim Calvin didn’t make.
• He frames the bill as a communication problem. He attributes items 7–9 to rhetoric or policy communication that wasn’t carefully written, in a company of over 5,000 people.
  That is exactly the “anomaly” framing Calvin predicted.
  It also doesn’t fit item 9. The draft bill’s thresholds, preemption scope and blocking standard are policy choices, not wording slips.
  Under the draft, Commerce could seek a court-ordered pause only for a clear, near-term risk of more than 50 deaths or serious injuries, or over $1 billion in damage, arising from bioweapons, loss of control or cyberattack.
• The defense of items 1–4 is circular. “That’s what a frontier company does” assumes the race-to-the-top strategy is working, which is the point in dispute.
  Vox’s Sigal Samuel asked him for evidence the strategy works, saying that from outside Anthropic looks to have done more on net to drive a race to the bottom.
  The reply offers no evidence that rivals have actually raised their standards in response.
• The RSP explanation doesn’t fit the change. He says the old RSP often required effortful steps that mattered little for safety while missing some of the most important things.
  That is a reason to rewrite the RSP, not a reason to drop the pause commitment in particular.
  The stronger argument, which he apparently didn’t make, is about incentives: a policy that triggers very costly measures at higher thresholds gives the company a reason not to declare those thresholds have been reached.
  Meanwhile, Chief Science Officer Jared Kaplan’s stated rationale at the time was that Anthropic would not pause alone if competitors were racing ahead.
  That is precisely the competitive logic Calvin is pointing to.
• Several “costly” decisions are partly self-serving or unverifiable. The size of the sacrificed lead is a counterfactual nobody can check. Restrictions on PRC sales and support for export controls also hurt Chinese competitors and help Anthropic’s standing in Washington.
  The 5% public benefit fund still needs shareholder approval, with further increases tied to valuation growth, so it is a conditional pledge rather than a completed transfer.
  It was also announced days before this exchange.
• His most significant admission undercuts his RSP defense. He says Anthropic is nearing the point where AI moves too fast for safety work to keep up, that this is shaping its thinking on “pacing,” and that he’ll have more to say soon.
  If that’s true, it is a strange moment to have removed the one commitment designed for exactly that situation.

Overall

Calvin’s argument is structurally stronger.

Its core is a hard question, why the errors are lopsided, and Amodei’s reply answers an easier one: does Anthropic ever sacrifice for safety?

The answer to that is plausibly yes, and Calvin already granted it.

Amodei’s reply is still a genuine good-faith engagement. The verifiable parts of his record are real, and the cheapest reading, that it’s all PR, doesn’t hold up against the Mythos timeline.

Get Kevin O'Shaughnessy’s stories in your inbox

Join Medium for free to get updates from this writer.

Remember me for faster sign in

The dispute can’t be settled by debating motives, since Calvin’s own point is that the outcomes are the same whatever the motives. It will be settled by what Anthropic does next, and Amodei’s deferrals have set up specific tests:

• the promised rewrite of the bill, and whether it ends up at least as strong as the FRONTIER Act;
• whether the compliance framework gains binding rules on internal deployment;
• whether Anthropic follows through on its commitment to propose options for regulating the pace of AI research by January 15, 2027, or explain why it won’t.

An action that costs Anthropic in policy terms, not just commercial ones, would be the strongest answer to Calvin.

Kevin: Nathan responded to Dario today please read and assess his latest

Claude: This is a much stronger post than Calvin’s first one. He concedes what Amodei got right and drops his weakest items. He then narrows his case to a single argument, and Amodei’s list of costly decisions doesn’t address that argument at all.

What changed

The first post was a pattern argument: nine actions, plus the question of why Anthropic’s mistakes keep favoring Anthropic. This one is a conditional argument built on Anthropic’s own premises:

1. By Anthropic’s own account, building recursively self-improving AI could be catastrophic.
2. Anthropic’s main justification for building it anyway is “someone will, so better us.”
3. That justification only holds if Anthropic is also a strong, consistent backer of governance that could bind it.
4. Anthropic’s legislative record points the other way.

He also openly concedes that Amodei’s examples of taking competitive hits were compelling and that Anthropic isn’t a profit-maximizer. That makes his remaining claims more credible, not less.

Strengths

It identifies the right currency. Amodei’s defense was a list of commercial sacrifices: the Mythos delay, bio safeguards, PRC sales restrictions. Calvin’s reply, in effect, is that commercial sacrifice isn’t the test. The test is whether Anthropic will accept external constraints it can’t revoke. Most of Amodei’s list is voluntary self-restraint, or support for constraints on others such as export controls on China. SB 53 is the clearest exception, and it’s a transparency law.

The distinction matters because self-imposed commitments can be withdrawn. The RSP v3 change in February is exactly that. Calvin doesn’t draw this link himself, but it strengthens his case.

The evidence is fewer items and more telling ones. Last time I noted that compute deals and chip teams don’t distinguish between his hypotheses. Three bills over two years do, and the record largely supports him:

• SB 1047 (2024). Anthropic’s “support if amended” letter asked to replace prescriptive pre-harm enforcement with a deterrence model. It identified the Frontier Model Division as a product of that enforcement and wanted the provisions removed or modified. After amendments eliminated the Division and narrowed pre-harm enforcement, Amodei wrote that the bill’s benefits likely outweighed its costs. Even then, he flagged the Attorney General’s remaining power to seek injunctions before harm as a possible vector for overreach.
• RAISE Act (2025). Calvin’s account matches the record. After the bill passed the legislature in June 2025, Jack Clark said Anthropic hadn’t taken an official stance. He criticized the bill as overly broad and called for state proposals to focus narrowly on transparency.
• AEGIS (2026). The blocking standard and the preemption are the same direction of travel.

Across all three, Anthropic has consistently tried to remove government power to act before harm occurs.

Point 4 turns Amodei’s defense into a dilemma. Either the AEGIS draft reflected Anthropic’s intent, or Anthropic was careless with one of the highest-stakes documents it will ever touch. His comparison to how Anthropic would treat a major compute deal or pre-training run is effective. It is exactly the gap I noted in Amodei’s “large organizations” framing.

Weaknesses

The “last resort” premise does most of the work and isn’t defended. Calvin asserts that racing is only conceivably acceptable if governance efforts are failing. Anthropic’s actual position is closer to “our participation lowers net risk compared with a world led by others.” That is a claim about counterfactual impact. Calvin largely treats the risk of superintelligence as something Anthropic imposes, rather than something it changes at the margin. His conditional still has force even if you grant the counterfactual, because the “better us” argument only works alongside serious support for governance. But he doesn’t engage with the counterfactual itself.

He waves away the overreach concern without engaging its strongest form. Anthropic has a concrete recent reason to fear executive power. In June, the government suspended Fable 5 and Mythos 5 under export controls. The best reading of Anthropic’s caution about pre-harm authority is that it was itself blocked ad hoc four months ago.

There is a strong rebuttal available, and Zvi made it: that experience argues for a formal, challengeable mechanism, not for making blocking nearly impossible. But Calvin needed to make that argument rather than treat overreach worries as cover.

He omits the most relevant counterevidence. In June, Amodei argued publicly that government should have the power to block deployment of models judged to carry unacceptable risk. Anthropic has also committed to propose options for regulating the pace of AI development by January 15, 2027, or explain why it won’t. These are exactly the forward-looking, binding-governance positions his argument says Anthropic should hold.

His point arguably still survives, because AEGIS is what that June stance looked like once written into a bill. But a fair accounting would name these positions and argue that the bill betrays them.

Some rhetoric outruns the evidence:

• “Breakneck speed to RSI” is his characterization, not an established fact.
• The paraphrase of Amodei’s six-to-twelve-month warning may attribute more to him than the original says. I can’t verify the exact wording.
• “No highly risky industry is regulated as anemically” is directionally fair on pre-harm authority, but Anthropic would point to liability, national-security powers and export controls as other levers that remain.

The asks are still vague. “Support stronger regulation” and “exercise extraordinary caution” give Anthropic nothing concrete to meet or fail. Zvi’s list was more actionable, for example admitting the draft was weaker than FRONTIER and saying what Anthropic actually wants.

Where this leaves the exchange

Calvin has now framed the debate on ground where Amodei’s existing defense doesn’t apply. The open question is no longer whether Anthropic sacrifices anything for safety. It is whether Anthropic will support binding external constraints on itself when it matters.

Anthropic’s strongest available response would combine three things:

• the net-risk argument for why it builds at all;
• an honest case about executive overreach, given June;
• a concrete commitment to back pre-harm authority with due-process safeguards, at least as strong as FRONTIER.

The AEGIS rewrite and the January 2027 pacing proposal will be the first real evidence on the question Calvin is now asking.

Sources:

Anthropic Does Not Understand Why Its Draft Bill Was Unacceptable and Troubling

• TNW: Amodei defends Anthropic’s race to the top
• AI郵報: fact-check of the Mythos timeline and Amodei’s nine items
• Cryptopolitan: AEGIS Act details
• Fox News live coverage
• GovAI: Anthropic’s RSP v3.0
• NYU Shanghai RITS: Anthropic drops flagship safety pledge
• AI Front Page: Amodei on RSP
• Calvin’s post on X
• Anthropic’s SB 1047 “Support If Amended” letter (2024)
• Campus Technology: Anthropic’s cautious support for amended SB 1047
• TechCrunch: New York passes RAISE Act (June 2025)
• GreaterWrong: text of Jack Clark’s RAISE post