Evidence at a glance
Long Documents Are More Than a Page-Count Problem
Baidu's Unlimited OCR 2 is an open-source vision-language OCR model designed for long documents. It takes page images in order and produces text, tables, formulas, figures, heading hierarchies, and reading order. Its aim is to process hundreds of pages from books, filings, and reports in one request. The model has 3.34 billion parameters, a maximum position length of 131,072 tokens, and weights released under the MIT license.
Accepting many pages at once is not just a matter of feeding images into a model. The challenge is also how to handle a context that keeps growing during generation. In the supplied multi-page configuration, each page costs 273 image tokens, so 100 pages contribute about 27,300 input tokens before any generated text. If every generated token attends to the full history, each decoding step has more context to scan as the document grows. Unlimited OCR 2 is designed to change that accumulating decoding cost.
A Fixed Reference Set Changes the Decoding Cost
The model uses SR-SWA, or Sparse Reference Sliding Window Attention. Unlike full attention, which consults the entire KV cache at every step, SR-SWA compresses text-visual attention into a fixed set of reference positions while retaining a local window for recent content. The model card gives 512 tokens as an example reference set. The configuration pairs this with a 128-token local window and top-16 selection from 32-token blocks. In effect, the attention range handled at each decoding step no longer grows in proportion to the number of historical tokens.
That does not mean the model looks at only 512 tokens, or that processing a whole document takes a fixed amount of computation. The reference set and local window constrain the attention workload, but pages still need visual encoding and generated output can keep growing. The supplied material explicitly says image encoding and total runtime still increase with page count. So “nearly flat per-step decoding cost” describes the scaling of one important part of the pipeline, not constant end-to-end cost. “Unlimited-horizon” refers to long-range processing within a finite context window, not an infinite-context guarantee.
The Scores Show Long-Document Strengths—and a Structural Weakness
Baidu reports that Unlimited OCR 2 ranked first on both multi-page document benchmarks it tested, against nine other models. On MarathonDocBench, it scored 77.1 overall; the next model listed, Kimi-K3, scored 33.7. This supports a cautious conclusion: in Baidu's reported comparisons, sparse attention's lower long-context decoding burden coincides with strong long-document parsing results.
An overall score can still obscure task-specific weaknesses. Unlimited OCR 2 scored 48.2 on the heading-hierarchy TEDS sub-score on MarathonDocBench, its lowest sub-score. The supplied material also identifies heading hierarchy as a weaker metric across the two benchmarks. For applications that need only continuous text, this may matter less than missed pages. But systems that preserve chapter structure, create searchable archives, or pass parsed documents to downstream processes can be materially affected by hierarchy errors. The leaderboard and scores are self-reported by Baidu, so they should not be treated as independently reproduced results.
Deployment Tests Must Cover the Whole Pipeline
The apparent deployment threshold is modest: the supplied material says the model runs in BF16 on a single CUDA GPU, and that Baidu tested it on an NVIDIA A800 with 80 GB of memory. The BF16 weights total about 6.7 GB. The code example loads the model through Transformers and calls infer_multi with ordered page images. For serving, Baidu recommends a customized SGLang build and illustrates page-window scheduling with 10 pages per window. That scheduling detail is a reminder that supporting long documents does not require a service to treat every page as one indivisible workload.
A sound deployment decision must test model quality against the workflow's actual needs. Beyond aggregate scores, teams should examine page order, tables, formulas, heading hierarchy, and reading order, while tracking end-to-end latency and resource use as page counts rise. The supplied material does not provide independent evaluations, throughput figures across hardware, or evidence that every file can be handled reliably up to the 131,072-token limit. The defensible conclusion is not that long-document OCR is solved, but that SR-SWA offers an engineering approach to limiting the growth of per-token decoding work. Whether it should replace an existing pipeline depends on tests with the target documents, required structural fidelity, and end-to-end cost.