Google’s Elephant-Goldfish Model: The Death of the Million-Token Chat

Why your AI coding agent gets dumber the longer you talk to it, and how Google engineers separated architecture from implementation.

Google’s Elephant-Goldfish Model: separating long-horizon context drafting from stateless, zero-bias code review.

Non-members can read for free here!

1. The dirty secret of long-context coding

You spend 45 minutes chatting with an AI coding agent in a single session.

The first 10 minutes feel like pure sorcery. The model grasps your intent, suggests sensible interfaces, and writes clean boilerplates.

Then you ask for a refactor.

By minute 30, things start to feel sluggish. The model suggests an import that does not exist in your package manager. You point out the error. It apologizes profusely, fixes the import, and silently drops two edge-case checks you specified 20 minutes ago.

By minute 45, the agent is trapped in a downward spiral. It hallucinates database column names, overwrites working unit tests, and nods along with whatever broken architectural idea you propose.

Sound familiar?

The industry spent the last two years celebrating 1-million-token and 2-million-token context windows. The marketing promised that you could dump an entire repository into a prompt and let the…