AI & ML Papers
33K subscribers
7.11K photos
533 videos
24 files
7.78K links
Advancing research in Machine Learning – practical insights, tools, and techniques for researchers.

Admin: @HusseinSheikho || @Hussein_Sheikho
Download Telegram
AI & ML Papers
Photo
🔥 MiniCPM-o 4.5: Towards Real-Time Full-Duplex Omni-Modal Interaction

💡 The paper introduces MiniCPM-o 4.5, a model that enables real-time full-duplex multimodal interaction, allowing it to see, listen, and speak simultaneously in real-time. The current state of multimodal large language models has limitations, including separated perception and response phases and reactive behavior, which prevent them from incorporating new inputs for timely adjustments during generation. To address these issues, the authors propose Omni-Flow, a unified streaming framework that aligns omni-modal inputs and outputs along a shared temporal axis, converting conventional turn-based interaction into a full-duplex, time-aligned process. This enables simultaneous perception and response and allows proactive behavior to arise within the same framework. MiniCPM-o 4.5 has 9B parameters and achieves state-of-the-art open-source performance, surpassing other models in omni-modal understanding and speech generation while delivering better computation efficiency. The model can perform real-time full-duplex omni-modal interaction on edge devices with less than 12GB RAM cost, making it a significant step towards human-like multimodal interaction. The key contributions of the paper are the introduction of Omni-Flow and the development of MiniCPM-o 4.5, which mitigates the gaps in current multimodal interaction models and enables real-time full-duplex omni-modal interaction. The results show that MiniCPM-o 4.5 approaches the performance of other models, such as Gemini 2.5 Flash, and surpasses Qwen3-Omni-30B-A3B in omni-modal understanding and speech generation, demonstrating its effectiveness and efficiency.


📅 Published on Apr 30

🔗 Links:
• arXiv: https://arxiv.org/abs/2604.27393
• PDF: https://arxiv.org/pdf/2604.27393
• Project Page: https://huggingface.co/openbmb/MiniCPM-o-4_5
• GitHub: https://github.com/OpenBMB/MiniCPM-o 24.7k

🤖 Models citing this paper:
https://huggingface.co/openbmb/MiniCPM-o-4_5
https://huggingface.co/openbmb/MiniCPM-V-4.6
https://huggingface.co/openbmb/MiniCPM-V-4.6-Thinking

🚀 Spaces citing this paper:
https://huggingface.co/spaces/openbmb/MiniCPM-V-4.6-Demo
https://huggingface.co/spaces/usermma/treadon-MiniCPM-V-4.6-Abliterated-AND-Disinhibited-Q4_K_M-gguf
https://huggingface.co/spaces/lspatilvs/Medical-Report-OCR

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://xn--r1a.website/PaperNexus

#MultimodalInteraction #FullDuplexCommunication #OmniModalProcessing #RealTimeLanguageModels #MultimodalLargeLanguageModels
1