💡 The paper presents MonkeyOCRv2, a visual-text foundation model for document AI. The problem addressed is that mainstream visual encoders are not effective for document images without document-oriented adaptation, as they are pre-trained on natural images and cannot handle dense text and fine-grained character strokes. To solve this, the authors construct MonkeyDocv2, the largest document-image pre-training corpus, comprising 113 million images spanning 17 languages. They propose a pre-training strategy that jointly learns image-to-text generation and pixel-level document reconstruction, aligning visual representations with textual content and preserving character strokes and layout details.

The authors conduct extensive experiments on five representative document analysis tasks, including text recognition, formular recognition, text detection, document tampering detection, and overlapping text segmentation. Replacing the original encoders with MonkeyOCRv2 consistently improves performance across all five tasks. The model is also validated as the vision encoder of multimodal large language models on more challenging tasks of document parsing and document understanding.

The results show that MonkeyOCRv2 yields a 0.7B document parsing model that sets a new open-source state-of-the-art on the MDPBench benchmark, surpassing the previous best by 2.8% absolute with a vision encoder roughly 11 times smaller. The frozen encoder also powers a document understanding model that outperforms counterparts built on CLIP, DINO, and SAM across eight benchmarks under identical training settings. These results suggest that document-oriented visual pre-training can serve as a foundation for document intelligence in its own right.