💡 The paper introduces MiniCPM-V 4.5, a highly efficient 8 billion parameter multimodal large language model that achieves strong performance. The development of multimodal large language models is rapidly advancing, but their training and inference efficiency has become a major obstacle to making them more accessible and scalable. To address this challenge, the authors propose three key improvements: a unified 3D-Resampler architecture for compact encoding of images and videos, a unified learning paradigm for document knowledge and text recognition without requiring extensive data engineering, and a hybrid reinforcement learning strategy for proficiency in both short and long reasoning modes.

The unified 3D-Resampler architecture enables highly compact encoding of visual data, while the unified learning paradigm simplifies the learning process by eliminating the need for heavy data engineering. The hybrid reinforcement learning strategy allows the model to excel in both short and long reasoning modes, making it a versatile and efficient model.

The authors evaluated MiniCPM-V 4.5 using the OpenCompass evaluation framework and found that it outperforms widely used proprietary models such as GPT-4 and larger open-source models like Qwen2.5-VL 72B. Notably, MiniCPM-V 4.5 achieves state-of-the-art performance on the VideoMME benchmark among models under 30 billion parameters, while using significantly less GPU memory and inference time compared to other models. Specifically, it uses 46.7 percent of the GPU memory cost and 8.7 percent of the inference time of Qwen2.5-VL 7B, demonstrating its remarkable efficiency. Overall, the paper presents a significant contribution to the development of efficient and scalable multimodal large language models.