💡 The paper introduces MiniCPM-V 4.5, a highly efficient 8 billion parameter multimodal large language model that achieves strong performance. The development of multimodal large language models is rapidly advancing, but their training and inference efficiency has become a major obstacle to making them more accessible and scalable. To address this challenge, the authors propose three key improvements: a unified 3D-Resampler architecture for compact encoding of images and videos, a unified learning paradigm for document knowledge and text recognition without requiring extensive data engineering, and a hybrid reinforcement learning strategy for proficiency in both short and long reasoning modes.

The unified 3D-Resampler architecture enables highly compact encoding of visual data, while the unified learning paradigm simplifies the learning process by eliminating the need for heavy data engineering. The hybrid reinforcement learning strategy allows the model to excel in both short and long reasoning modes, making it a versatile and efficient model.

The authors evaluated MiniCPM-V 4.5 using the OpenCompass evaluation framework and found that it outperforms widely used proprietary models such as GPT-4 and larger open-source models like Qwen2.5-VL 72B. Notably, MiniCPM-V 4.5 achieves state-of-the-art performance on the VideoMME benchmark among models under 30 billion parameters, while using significantly less GPU memory and inference time compared to other models. Specifically, it uses 46.7 percent of the GPU memory cost and 8.7 percent of the inference time of Qwen2.5-VL 7B, demonstrating its remarkable efficiency. Overall, the paper presents a significant contribution to the development of efficient and scalable multimodal large language models.

💡 The paper introduces MiniCPM-o 4.5, a model that enables real-time full-duplex multimodal interaction, allowing it to see, listen, and speak simultaneously in real-time. The current state of multimodal large language models has limitations, including separated perception and response phases and reactive behavior, which prevent them from incorporating new inputs for timely adjustments during generation. To address these issues, the authors propose Omni-Flow, a unified streaming framework that aligns omni-modal inputs and outputs along a shared temporal axis, converting conventional turn-based interaction into a full-duplex, time-aligned process. This enables simultaneous perception and response and allows proactive behavior to arise within the same framework. MiniCPM-o 4.5 has 9B parameters and achieves state-of-the-art open-source performance, surpassing other models in omni-modal understanding and speech generation while delivering better computation efficiency. The model can perform real-time full-duplex omni-modal interaction on edge devices with less than 12GB RAM cost, making it a significant step towards human-like multimodal interaction. The key contributions of the paper are the introduction of Omni-Flow and the development of MiniCPM-o 4.5, which mitigates the gaps in current multimodal interaction models and enables real-time full-duplex omni-modal interaction. The results show that MiniCPM-o 4.5 approaches the performance of other models, such as Gemini 2.5 Flash, and surpasses Qwen3-Omni-30B-A3B in omni-modal understanding and speech generation, demonstrating its effectiveness and efficiency.

💡 The paper presents LoomVideo, a unified architecture for video generation and editing that efficiently handles multimodal inputs. The problem addressed is that existing unified frameworks for video generation and editing rely on large models with high computational overhead, typically requiring 13 billion parameters or more. These models often incorporate source video conditions for editing by concatenating sequence tokens, which doubles the sequence length and quadruples the computational complexity of the self-attention mechanism.

To address this issue, LoomVideo introduces a novel 5 billion parameter architecture that replaces the standard text encoder with a Multimodal Large Language Model and employs a Deepstack injection mechanism to align multi-layer features with the Diffusion Transformer. The key contribution is the introduction of a zero-overhead Scale-and-Add conditioning approach for video editing, which eliminates the need for token concatenation and reduces computational cost. This approach scales and directly adds the clean source video latent to the noised target latent, allowing for complex and non-rigid edits.

Additionally, the model incorporates a Negative Temporal RoPE strategy to handle multiple reference images. The results demonstrate that LoomVideo achieves state-of-the-art or highly competitive performance across comprehensive benchmarks, with exceptional performance in e-commerce and fashion generation scenarios. The zero-overhead conditioning mechanism enables LoomVideo to achieve at least a 5.41x acceleration in inference speed compared to models of similar capabilities, making it a highly practical and efficient video foundation model. Overall, LoomVideo presents a significant advancement in unified video generation and editing models, offering a more efficient and effective solution for handling multimodal inputs.

💡 The paper introduces a new benchmark suite called RNG-Bench to evaluate the ability of multimodal large language models to reconstruct past observations and use them for decision-making in multi-step interactions. The problem addressed is that existing benchmarks do not adequately test a model's ability to recall and act on past observations, which is a crucial skill for deploying these models in real-world applications. The RNG-Bench suite consists of two games, Matching Pairs and 3D Maze, which are designed to test a model's ability to reconstruct past observations and use them to make decisions. The games have controlled difficulty parameters, including grid size, visual pattern, and observation modality, which allow for a thorough evaluation of a model's skills. The benchmark also introduces a head-to-head duel protocol to control for instance-level variance and a Memory Gap metric to distinguish between forgetting and poor decision-making. The results show that most residual errors in the models' performance are due to forgetting earlier observations rather than suboptimal decision-making. The paper also demonstrates that fine-tuning a model on optimal-policy rollouts and filtered model demonstrations can improve its performance on RNG-Bench and transfer to existing benchmarks without degrading its general multimodal capability. Overall, the paper provides a new benchmark suite and evaluation methodology for multimodal large language models, and demonstrates the importance of testing these models' ability to recall and act on past observations.

💡 The paper introduces a new benchmark and dataset for photography assistance, focusing on capture-time guidance for both camera framing and subject pose. Existing models primarily evaluate post-hoc crop prediction and overlook subject-side recommendations, leaving a gap in the capabilities of multimodal large language models. To address this, the researchers developed CaptureGuide-Bench, a benchmark with two tasks: photographer-side composition decision and refinement, and subject-side scene-conditioned pose recommendation. They also constructed CaptureGuide-Dataset, comprising 130K samples with textual rationales and visual annotations.

The researchers then developed ShutterMuse, a unified multimodal large language model trained with supervised and reinforcement fine-tuning. ShutterMuse provides both composition guidance and pose recommendations during image capture. The evaluation reveals that general-purpose models can make composition decisions but lack precise refinement localization, while specialized aesthetic cropping models localize crops effectively but are limited to refinement and do not provide pose guidance.

The experiments on CaptureGuide-Bench show that ShutterMuse achieves the best overall photographer-side performance among evaluated baselines and competitive subject-side pose recommendation with lower inference cost. This demonstrates the potential of multimodal large language models as interactive assistants for photography during image capture, addressing the need for capture-time guidance in real-world photography. The paper contributes to the development of models that can provide effective guidance for both camera framing and subject pose, making it a significant step forward in the field of photography assistance.