AI & ML Papers
Photo
🔥 MiniCPM-o 4.5: Towards Real-Time Full-Duplex Omni-Modal Interaction
📅 Published on Apr 30
🔗 Links:
• arXiv: https://arxiv.org/abs/2604.27393
• PDF: https://arxiv.org/pdf/2604.27393
• Project Page: https://huggingface.co/openbmb/MiniCPM-o-4_5
• GitHub: https://github.com/OpenBMB/MiniCPM-o ⭐ 24.7k
🤖 Models citing this paper:
• https://huggingface.co/openbmb/MiniCPM-o-4_5
• https://huggingface.co/openbmb/MiniCPM-V-4.6
• https://huggingface.co/openbmb/MiniCPM-V-4.6-Thinking
🚀 Spaces citing this paper:
• https://huggingface.co/spaces/openbmb/MiniCPM-V-4.6-Demo
• https://huggingface.co/spaces/usermma/treadon-MiniCPM-V-4.6-Abliterated-AND-Disinhibited-Q4_K_M-gguf
• https://huggingface.co/spaces/lspatilvs/Medical-Report-OCR
━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://xn--r1a.website/PaperNexus
#MultimodalInteraction #FullDuplexCommunication #OmniModalProcessing #RealTimeLanguageModels #MultimodalLargeLanguageModels
💡 The paper introduces MiniCPM-o 4.5, a model that enables real-time full-duplex multimodal interaction, allowing it to see, listen, and speak simultaneously in real-time. The current state of multimodal large language models has limitations, including separated perception and response phases and reactive behavior, which prevent them from incorporating new inputs for timely adjustments during generation. To address these issues, the authors propose Omni-Flow, a unified streaming framework that aligns omni-modal inputs and outputs along a shared temporal axis, converting conventional turn-based interaction into a full-duplex, time-aligned process. This enables simultaneous perception and response and allows proactive behavior to arise within the same framework. MiniCPM-o 4.5 has 9B parameters and achieves state-of-the-art open-source performance, surpassing other models in omni-modal understanding and speech generation while delivering better computation efficiency. The model can perform real-time full-duplex omni-modal interaction on edge devices with less than 12GB RAM cost, making it a significant step towards human-like multimodal interaction. The key contributions of the paper are the introduction of Omni-Flow and the development of MiniCPM-o 4.5, which mitigates the gaps in current multimodal interaction models and enables real-time full-duplex omni-modal interaction. The results show that MiniCPM-o 4.5 approaches the performance of other models, such as Gemini 2.5 Flash, and surpasses Qwen3-Omni-30B-A3B in omni-modal understanding and speech generation, demonstrating its effectiveness and efficiency.
📅 Published on Apr 30
🔗 Links:
• arXiv: https://arxiv.org/abs/2604.27393
• PDF: https://arxiv.org/pdf/2604.27393
• Project Page: https://huggingface.co/openbmb/MiniCPM-o-4_5
• GitHub: https://github.com/OpenBMB/MiniCPM-o ⭐ 24.7k
🤖 Models citing this paper:
• https://huggingface.co/openbmb/MiniCPM-o-4_5
• https://huggingface.co/openbmb/MiniCPM-V-4.6
• https://huggingface.co/openbmb/MiniCPM-V-4.6-Thinking
🚀 Spaces citing this paper:
• https://huggingface.co/spaces/openbmb/MiniCPM-V-4.6-Demo
• https://huggingface.co/spaces/usermma/treadon-MiniCPM-V-4.6-Abliterated-AND-Disinhibited-Q4_K_M-gguf
• https://huggingface.co/spaces/lspatilvs/Medical-Report-OCR
━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://xn--r1a.website/PaperNexus
#MultimodalInteraction #FullDuplexCommunication #OmniModalProcessing #RealTimeLanguageModels #MultimodalLargeLanguageModels
arXiv.org
MiniCPM-o 4.5: Towards Real-Time Full-Duplex Omni-Modal Interaction
Recent progress in multimodal large language models (MLLMs) has brought AI capabilities from static offline data processing to real-time streaming interaction, yet they still remain far from...
❤1