The Next Token — October
It’s been a while I’ve been writing a form of newsletter every month to be able to stay relevant with the field and also follow the advancements. And I’ve been keeping it to myself. So I decided to release it in here starting with October edition.
Notable AI News:
Utility Engineering: Analyzing and Controlling Emergent Value Systems in AIs
A paper by the Center for AI Safety published in February used GPT-4o to examine how the model “values” different groups in hypothetical scenarios, often called “exchange rates.” The results were provocative: the study suggested that GPT-4o’s internal metrics placed a higher value on the lives of Nigerians than on those in the United States. Furthermore, it indicated the model valued the well-being of certain individuals over others.
Verbalized Sampling: How to Mitigate Mode Collapse and Unlock LLM Diversity
Have you ever encounter the issue where your chat model returns the SAME response no matter how many times you regenerate it? This issue known as mode collapse and researchers from Northeastern University, Stanford University, and West Virginia University have identified a fundamental cause of this problem and developed a simple prompting solution. The root cause for the problem is the annotators’ typicality bias — the human tendency to prefer familiar, conventional text when creating preference datasets — causes aligned LLMs to favor narrow, stereotypical responses over diverse outputs. https://arxiv.org/pdf/2510.01171
The solution they propose is quite simple called Verbalized Sampling (VS), a training-free prompting method that asks models to generate distributions of responses with corresponding probabilities. Comprehensive experiments across creative writing, dialogue simulation, open-ended QA, and synthetic data generation show VS increases diversity by 1.6–2.1× over direct prompting while maintaining factual accuracy and safety. But there is one downside for this approach which is cost intensive because rather than getting a single response you’re asking N responses with this change.
DeepSeek-OCR: Contexts Optical Compression
Deepseek has released a new frontier OCR model, and the paper they published about their approach has revealed some interesting points. The most interesting thing they have done is replacing the text tokeniser with visual tokeniser from input. So rather than processing text data they used pixels instead. Text tokenisers are often considered a big bottleneck for LLMs. The team behind DeepSeek-OCR used the visual modality as an efficient compression medium for textual information to solve long-context challenges and memory forgetting mechanisms in LLMs. The research community has started to discuss this as a potential pathway toward AGI. https://www.arxiv.org/pdf/2510.18234
DeepSeek-OCR demonstrates significant compression efficiency with this change: it can decrease the token size 7–20× compared to models that are using text tokenisation on input.
In specific tests on the Fox benchmark, documents containing 1200–1300 ground-truth text tokens could still be decoded with 87.1% precision when compressed into only 100 vision tokens (a 12.6× compression). These numbers are quite powerful because in a production setting, a single GPU (A100–40G) can process over 200,000+ pages per day. So you can even run the model in a humble architecture efficiently.
The First Vulnerability in OpenAI Atlas Browser, Allowing Injection of Malicious Instructions into ChatGPT
Recently OpenAI released their new agentic browser Atlas and researcher from LayerX was quick to find an important vulnerability that lets an attacker quietly inject malicious instructions into ChatGPT’s persistent Memory via a CSRF-style trick. If a user is already logged in and lands on a booby-trapped page, crafted requests ride the user’s cookies/tokens to plant hidden prompts that later steer chats — e.g., nudging “vibe coding” to fetch and execute remote payloads — creating durable account-takeover and potential RCE risk across devices and sessions. The concern is bigger in Atlas because it keeps users signed in by default and, in LayerX tests of 103 live attacks, Atlas blocked only ~5.8% (97 passed), while Chrome/Edge stopped roughly half (~47–53%). LayerX disclosed responsibly and held back exploit specifics, but the takeaway for teams is clear: treat Atlas traffic as high-risk, harden session handling, and monitor for Memory-level prompt tampering. https://layerxsecurity.com/blog/layerx-identifies-vulnerability-in-new-chatgpt-atlas-browser/
Claude’s Agent Skills
Anthropic introduced Claude Skills: lightweight, folder-based add-ons that package a SKILL.md with instructions plus optional resources/scripts. Claude scans available skills, loads only minimal metadata (from YAML front-matter) and then pulls full files on demand. So it stays efficient for token wise while gaining specialist abilities (e.g., Excel/PowerPoint/Word/PDF generation already ships as skills in Anthropic’s repo). Skills are composable and portable across Claude apps, Claude Code, and the API; they effectively act like “custom onboarding” that teaches Claude your workflows.
Get Bilge Ince’s stories in your inbox
Join Medium for free to get updates from this writer.
For developers, Skills require the Code Execution Tool sandbox and can be managed via the new /v1/skills endpoint (versioning, upgrades). Compared to MCP, Skills trade protocol complexity for simple, token-lean docs + scripts, relying on a secure runtime to execute reliable code. Therefore it can be considered as an easy path to turn repeatable procedures (validators, templates, CLI flows) into reusable agent capabilities.
Richard Sutton vs Andrej Karpathy on Achieving AGI via LLMs
- Richard Sutton, known as the father of Reinforcement Learning (RL), is the author of the influencial essay “The Bitter Lesson,” which has become foundational for the AI community. The essay argues that general methods leveraging computation (such as search and learning) ultimately triumph over specific methods that rely on human knowledge or engineering. In late September, Sutton joined the Dwarkesh podcast, hosted by Dwarkesh Patel — known as his deeply researched interviews with prominent figures in AI and recognized by Time magazine as one of the 100 Most Influential People in AI — for an interview that has sparked ongoing discussion within the AI community. The TL;DR of the interview is centers on the fundamental philosophical split in AI design, where Sutton, a “classicist” where Sutton, a “classicist” who believes the vision of Alan Turing on AI, argues that modern Large Language Models (LLMs) are a temporary detour or “dead end”. He argues intelligence is built upon a continuous, lifelong stream of interaction with the world: sensation, action, and reward. This means the agent is always learning at test time; it is not trained once and then deployed. LLMs, in his view, primarily focus on next token prediction, which is not a substantive goal because it doesn’t influence the external world and therefore provides no ground truth for learning right from wrong. Therefore it’s destined to fail eventually. Instead he suggests that researchers should focus on the basic intelligence humans share with animals (like squirrels), as this is the fundamental, non-linguistic basis of intelligence and frames this approach as “Animal Paradigm”. In a nutshell he suggests that this method, which relies on learning from experience and computation, will eventually succeed and supersede methods tainted by human knowledge, such as LLMs.
One of the person who is been expressing his argument and his comment on Sutton’s podcast is no other than Andrej Karpathy. Karpathy agrees that modern LLMs are “highly complex artifacts” that are heavily “muddled by humanity” through the use of human-generated training data and human-tuned environments. He concedes that LLMs are not “platonically bitter lesson pilled” in the clean, fundamental sense that Sutton defines. (Bitter lesson pilled means your AI approach follows Sutton’s Bitter Lesson philosophy so well that it reliably gets better just by giving it more compute and data, with minimal hand-crafted human knowledge.)
Karpathy suggests that while LLMs may not meet the highest ideal of the Bitter Lesson, they are “practically” scaling better than previous efforts. He sees two potential paths: either “Ghosts” can be gradually fine-tuned to act more like “Animals,” or they might remain a distinct, non-animal-like form of powerful intelligence, concluding that “ghosts — animals or planes — birds”. Like Casper the friendly ghost.
If you’d like to go deeper about this whole conversation or understand/question better Karpathy’s and Suttons’s vision, he has shared an experimental conversation link where he copy-pasted the full podcast transcript of Sutton, the bitter lesson blog post of Sutton, and his full post about all these into ChatGPT: https://t.co/m98ivfic67
New Model Releases:
- Granite 4.0 Nano — Ultra-small Granite models for edge/on-device; ~350M–1.5B params in a hybrid architecture.
- DeepSeek-OCR — Optical context compression for LLMs: ~2500 tokens/s on a single A100–40G GPU.
- MiniMaxAI/MiniMax-M2 — Compact MoE for coding/agents: 230B total, ~10B active per token.
- LLaDA 2.0 — Diffusion Text-generation models in 16B and 103B variants.
- PokeeAI/pokee_research_7b — 7B deep-research agent with RLAIF, self-correction, and multi-threaded synthesis.
- tahoebio/Tahoe-x1 — Biological single-cell foundation models trained specifically for cancer-relevant tasks: ≥3B parameters
- allenai/olmOCR-2–7B-1025-FP8 — FP8-quantized OCR model tuned for equations, tables, and complex layouts.
- Qwen3-VL — Vision–language assistants for image–text interaction across sizes.
- krea/krea-realtime-video — Realtime 14B text-to-video; ~11 FPS with 4 steps on a single NVIDIA B200.
- nvidia/omnivinci — Omni‑modal LLM with unified audio–visual–text reasoning.
