Unlimited-OCR: Baidu Quietly Fixed the Problem Every Document Parser Has Been Hiding
The Problem Every Document Pipeline Quietly Lives With
I keep coming back to the same question whenever a new model paper drops: is this solving a real problem, or just chasing a benchmark? With Unlimited-OCR, it’s clearly the former, and it’s a problem anyone who’s built a document pipeline has run into without ever naming it properly.
Here’s the issue. Feed almost any OCR or document-parsing model a long PDF, and somewhere past page ten it starts quietly falling apart. Tables that span a page break come out as two disconnected fragments. A heading on page eleven loses the paragraphs it’s supposed to govern on page twelve. It’s not a crash. It’s worse, because it looks like it’s working.
Baidu’s Unlimited-OCR is a direct attack on that failure mode, and it does it with an architecture change rather than a bigger model.
Why OCR Breaks on Long Documents
The dirty secret of document parsing is that almost every model processes one page, emits its output, and forgets everything before starting the next page. It’s a for-loop wearing a trench coat, pretending to be comprehension.
That works fine for a single clean scan. It falls apart the moment structure spans a boundary; a table that continues onto the next page, a contract clause that references something three pages back, a multi-column research paper where reading order itself depends on remembering where you were. Baidu’s own paper states it plainly: no existing model could parse even ten pages in a single forward pass before this. Everyone chunks, everyone stitches the output back together afterward, and everyone quietly accepts that as the cost of doing business.
What Baidu Actually Built
Unlimited-OCR is a 3-billion-parameter model, though only around 500 million parameters are active at any point thanks to a mixture-of-experts setup underneath. It isn’t built from scratch. It’s explicitly positioned as a continuation of DeepSeek-OCR, pushing that architecture “one step further,” and it reuses DeepSeek-OCR’s vision stack rather than reinventing image understanding from zero.
The headline capability is the one in the name: it can parse dozens of pages, up to roughly 40, in a single continuous forward pass, under a 32K token ceiling. No restarting per page. No external stitching layer patched on afterward to undo the damage of chunking.
The Actual Fix Nobody Else Tried: R-SWA
Here’s the part I think is easy to gloss over in the surface-level coverage: the fix isn’t scale. It’s a new attention mechanism called Reference Sliding Window Attention, or R-SWA.
Get Karthik’s stories in your inbox
Join Medium for free to get updates from this writer.
The problem with just “generating more tokens” in a transformer is that the KV cache, the model’s running memory of everything it has generated, grows with every new token. Longer output means more memory, means slower generation, until eventually you run out of GPU room entirely. That’s the actual reason long-document OCR was structurally hard, not a data problem, an architecture problem.
R-SWA solves this by treating two kinds of tokens differently. The source document stays permanently visible to the model at all times, it never loses access to what it’s transcribing. But its own output, the text it has already generated, slides through a fixed-size window and gets dropped once it falls far enough behind. The cache stops growing. Memory stays flat regardless of how long the document is.
The Human Reading Analogy, and Why It Actually Holds Up
The inspiration behind R-SWA is almost embarrassingly simple once you see it: it’s just how a person transcribes something long.
You don’t re-read every word you’ve already copied down before writing the next one. What you wrote a page ago fades from active attention, you’re not holding it word for word, you just remember the gist. But you keep glancing back at the source material as often as you need to. That’s the entire design principle transplanted into an attention mechanism: permanent access to the reference, fading access to your own prior output.
It’s a small conceptual move, but it’s the kind of move that tends to age well, because it maps onto how memory actually needs to behave in any long-horizon generation task, not just OCR.
Architecture Deep Dive
Structurally, Unlimited-OCR keeps DeepSeek-OCR’s DeepEncoder, a SAM-ViT-B plus CLIP-L combination, whose job is to compress an entire document image into a very small number of visual tokens before the decoder ever sees it. That aggressive compression is what makes processing dozens of pages computationally feasible in the first place; you’re not feeding raw pixels into a growing context, you’re feeding a compressed representation.
The decoder is where the real change lives. It’s a mixture-of-experts model, and every attention layer inside it has been replaced with R-SWA. The vision side stayed frozen. The generation side got rebuilt around the sliding-window-plus-reference idea.
Do the Numbers Actually Hold Up?
On OmniDocBench v1.5, Unlimited-OCR scores 93.23 overall, about 6.2 points above the DeepSeek-OCR baseline it’s built on. On the newer v1.6 benchmark it reaches 93.92, the top score in that comparison, with gains showing up consistently across text, formula, and table recognition specifically, not just in an averaged headline number.
Speed improves too, which is the part that actually matters for production. In base mode it hits roughly 5,580 tokens per second against DeepSeek-OCR’s 4,951. Some long-output tests reported throughput gains around 35% over the baseline. The interesting bit isn’t just that it got more accurate, it’s that it got faster and more accurate at the same time, which doesn’t usually happen when you’re solving a memory-efficiency problem.
Getting It Running
It’s already out on Hugging Face, ModelScope, and Baidu Cloud, with a demo up on Hugging Face Spaces and support for vLLM inference contributed by the community. If you go the vLLM route, there are two things that will trip you up if nobody warns you first: you need to register a custom no-repeat-ngram logits processor on the server, and your prompt has to begin with a literal image tag before the actual instruction text. Neither is documented loudly, both cost real debugging time if you skip them.
Unlimited-OCR closes a real and specific gap, page-boundary memory loss, while keeping its efficiency intact. Whether it holds up outside clean benchmark conditions is still an open question.
