💡 The paper introduces RLDX-1, a general-purpose robotic policy for dexterous manipulation that addresses the limitations of existing vision-language-action models. These models have shown progress in human-like generalist robotic policies but struggle with complex real-world tasks that require broader functional capabilities such as motion awareness, memory-aware decision making, and physical sensing. To overcome this, RLDX-1 uses a Multi-Stream Action Transformer architecture that integrates heterogeneous modalities through modality-specific streams with cross-modal joint self-attention. This architecture is combined with system-level design choices including synthesizing training data for rare manipulation scenarios, learning procedures specialized for human-like manipulation, and inference optimizations for real-time deployment. The results show that RLDX-1 outperforms recent frontier vision-language-action models across both simulation benchmarks and real-world tasks, achieving success rates of 86.8 percent in ALLEX humanoid tasks compared to around 40 percent for other models. This positions RLDX-1 as a promising step toward reliable vision-language-action models for complex and dynamic real-world dexterous manipulation. The method and results demonstrate the ability of RLDX-1 to control a high-degree-of-freedom humanoid robot under diverse functional demands, highlighting its potential for complex real-world tasks.

💡 The paper introduces D-OPSD, a new training approach for diffusion models that enables efficient supervised fine-tuning while preserving few-step inference capabilities. The current landscape of high-performance image generation models is shifting from inefficient multi-step models to efficient few-step models, but these models are challenging to fine-tune using traditional techniques. The problem with traditional fine-tuning methods is that they compromise the model's inherent few-step inference capability.

To address this issue, the authors propose D-OPSD, which leverages on-policy self-distillation with text and multimodal features. The method works by making the model act as both the teacher and the student, where the student is conditioned only on the text feature, and the teacher is conditioned on the multimodal feature of both the text prompt and the target image. The training process minimizes the difference between the predicted distributions over the student's own roll-outs, allowing the model to learn new concepts and styles without sacrificing its original few-step capacity.

The key contribution of D-OPSD is that it enables on-policy learning during supervised fine-tuning, which allows the model to learn from its own trajectory and under its own supervision. This approach enables the model to inherit the in-context capabilities of its encoder, making it possible to fine-tune the model continuously without compromising its few-step inference capability. The results show that D-OPSD enables efficient supervised fine-tuning for diffusion models, making it a promising approach for high-performance image generation models.

💡 This paper proposes a novel approach called CapVector to improve the performance of vision-language-action models. The problem addressed is that pre-trained models often fail to improve performance and reduce adaptation costs during standard supervised finetuning. Advanced finetuning methods with auxiliary training objectives can improve performance but incur significant computational overhead.

The proposed method decouples the auxiliary training objectives from standard supervised finetuning to enhance model capabilities while reducing computational overhead. This is achieved by training the model to converge on a small-scale task set using two distinct training strategies, resulting in two finetuned models. The parameters difference between the two models is interpreted as capability vectors provided by auxiliary objectives. These vectors are then merged with pre-trained parameters to form a capability-enhanced meta model.

The method also uses a lightweight orthogonal regularization loss to augment standard supervised finetuning, which reduces computational overhead. The results show that the capability vectors are effective and versatile across diverse models, and can generalize to novel environments and embodiments without additional training. The proposed approach achieves performance comparable to auxiliary finetuned baselines with reduced computational overhead, making it a promising solution for improving vision-language-action models.

💡 The paper addresses the issue of unified multimodal models where visual understanding and generation are not well aligned due to separate training objectives. The prevailing approach of optimizing understanding through text signals and generation through pixel objectives leads to isolated representation spaces. To bridge this gap, the authors propose a novel approach called Semantic Generative Tuning, which uses semantic segmentation as a generative proxy to align and synergize multimodal capabilities.

The method involves formulating hierarchical visual tasks as generative proxies, with a focus on high-level semantic tasks like image segmentation. The authors find that segmentation provides structural semantics that enhance both vision-centric perception and generative layout fidelity. Unlike low-level tasks, segmentation does not distract models with texture details, making it an optimal proxy.

The results show that Semantic Generative Tuning fundamentally improves feature linear separability and optimizes visual-textual attention allocation patterns. Extensive evaluations demonstrate that this approach consistently improves both multimodal comprehension and generative fidelity across mainstream benchmarks. The authors provide a systematic investigation into generative post-training and introduce a new paradigm that leverages segmentation to align multimodal capabilities. The code for the proposed method is made available for further research and development. Overall, the paper presents a significant contribution to the field of unified multimodal models by introducing a novel approach that enhances multimodal alignment and performance.

💡 The paper introduces LLaVA-OneVision-2, a vision-language model that achieves superior performance across various multimodal benchmarks. The problem addressed is the need for a more capable model that can efficiently process and understand video content. The method used to achieve this is codec-stream tokenization, which treats compressed video as a continuous bit-cost stream and allocates a limited token budget to event-bearing content. This approach enables more stable long-video token compression than fixed groups of pictures. The model also incorporates windowed attention for efficient local computation and a shared 3D RoPE to place codec canvases, sampled frames, and images in a unified spatiotemporal coordinate system.

The model was trained using large-scale open supervision, with approximately 8 million re-captioned video samples for pretraining and a 4 million sample spatial corpus for fine-tuning. The paper also introduces JumpScore, a temporal-localization benchmark that targets fine-grained grounding in high-frequency, densely repeated motion. The results show that LLaVA-OneVision-2 outperforms existing models, including Qwen3-VL-8B, by a significant margin. On the JumpScore benchmark, LLaVA-OneVision-2-8B reaches 74.9 JumpScore mAP, surpassing Qwen3-VL-8B by 44.8 points. The model also outperforms Qwen3-VL-8B by 4.3 average points on video tasks, 5.3 on spatial tasks, and 15.6 average J&F on tracking tasks.

The key contributions of the paper are the introduction of codec-stream tokenization, windowed attention, and large-scale open supervision, which enable the model to achieve superior performance across a broad range of multimodal benchmarks. The paper also highlights the importance of unified perception across video understanding, temporal grounding, spatial grounding, and manipulation-trace reasoning. Overall, the paper demonstrates the effectiveness of LLaVA-OneVision-2 in achieving next-generation perceptual intelligence.

💡 The paper introduces WebWatcher, a multimodal agent designed to improve visual-language reasoning in deep research tasks. The problem addressed is that most existing research agents are text-centric and overlook visual information, making multimodal deep research challenging. To solve this, WebWatcher is equipped with enhanced visual-language reasoning capabilities, leveraging synthetic multimodal trajectories for efficient training, utilizing various tools for deep reasoning, and enhancing generalization through reinforcement learning.

The method involves using high-quality synthetic multimodal trajectories for cold start training, which allows the agent to learn from both visual and textual information. The agent is also designed to work with various tools to improve its reasoning abilities. Additionally, the paper proposes a new benchmark called BrowseComp-VL, which is used to evaluate the capabilities of multimodal agents in complex information retrieval tasks involving both visual and textual information.

The results show that WebWatcher significantly outperforms existing baseline agents, including proprietary and open-source agents, in four challenging visual question answering benchmarks. This demonstrates the effectiveness of WebWatcher in solving complex multimodal information-seeking tasks and paves the way for further research in this area. Overall, the paper contributes to the development of multimodal agents with stronger reasoning abilities, which can handle both visual and textual information, and provides a new benchmark for evaluating the performance of such agents.

💡 The paper introduces M3Eval, a comprehensive evaluation framework and benchmark for assessing the memory capabilities of multi-modal models in video understanding systems. The problem addressed is that current multi-modal models have significant limitations in their memory capabilities, particularly in maintaining disentangled representations and demonstrating human-like interference patterns. To address this gap, the authors designed M3Eval, which is grounded in cognitive psychology and features carefully constructed tasks that isolate key aspects of memory.

The method involves conducting extensive experiments across representative multi-modal models using the M3Eval framework, which evaluates different memory dimensions such as what models retain, how faithfully information is preserved, and how robust memory remains under interference. The framework includes tasks that test the models' ability to maintain disentangled representations, exhibit human-like interference patterns, and demonstrate symbolic memory.

The results of the experiments reveal consistent weaknesses and distinctive behaviors in the models. The models struggle to maintain disentangled representations when processing parallel video streams, exhibit interference patterns that differ substantially from those observed in human memory, and ground memory sources more reliably in the spatial domain than the temporal domain. Additionally, the models demonstrate limited symbolic memory.

The paper's contributions include providing a valuable resource for future research in the form of the M3Eval benchmark and highlighting memory as a fundamental yet underexplored capability in multi-modal models. The findings offer insights for designing more effective memory mechanisms in multi-modal models, which can advance the field of video understanding systems. The code and dataset are made available to facilitate future research.

💡 The paper introduces UniCorn, a self-improvement framework for unified multimodal models that addresses the generation gap in these models. The generation gap refers to the discrepancy between a model's ability to understand multimodal inputs and its ability to generate high-quality outputs. This gap is formalized as Conduction Aphasia, where models can accurately interpret multimodal inputs but struggle to translate that understanding into faithful and controllable synthesis.

To address this issue, UniCorn proposes a simple yet elegant self-improvement framework that eliminates the need for external data or teacher supervision. The framework partitions a single unified multimodal model into three collaborative roles: Proposer, Solver, and Judge. The Proposer generates initial outputs, the Solver refines these outputs, and the Judge evaluates the quality of the refined outputs. Through self-play and cognitive pattern reconstruction, UniCorn generates high-quality interactions and distills latent understanding into explicit generative signals.

The authors introduce UniCycle, a cycle-consistency benchmark based on a Text to Image to Text reconstruction loop, to validate the restoration of multimodal coherence. The results demonstrate that UniCorn achieves comprehensive and substantial improvements over the base model across six general image generation benchmarks. Notably, it achieves state-of-the-art performance on several benchmarks, including TIIF, DPG, CompBench, and UniCycle, and delivers substantial gains on WISE and OneIG.

The contributions of the paper are significant, as UniCorn enhances text-to-image generation while maintaining robust comprehension, demonstrating the scalability of fully self-supervised refinement for unified multimodal intelligence. The results highlight the effectiveness of the self-improvement framework in addressing the generation gap in unified multimodal models, and the potential of UniCorn to improve the performance of these models in various applications. Overall, the paper presents a novel approach to self-improving unified multimodal models, with significant implications for the development of more advanced and effective multimodal models.

💡 The paper proposes a novel framework called Robust-U1 to enhance the robustness of multimodal large language models against visual corruptions. The problem addressed is that existing models perform poorly when faced with real-world visual corruptions such as noise or blur. Current approaches to improve robustness have limitations, either lacking interpretability or being unable to restore lost pixel-level details.

The Robust-U1 framework is designed to equip models with explicit visual self-recovery capability, allowing them to recover corrupted visual content by themselves. The approach consists of three stages: supervised fine-tuning for initial reconstruction, reinforcement learning with dual rewards to align high visual quality, and multimodal reasoning that considers both the corrupted input and the recovered image.

The results show that Robust-U1 achieves state-of-the-art robustness on a real-world corruption benchmark and maintains superior performance under adversarial corruptions on general visual question answering benchmarks. The analysis confirms that high-quality visual recovery directly enhances reasoning performance, establishing self-recovery as a critical mechanism for robust visual understanding. Overall, the paper demonstrates that multimodal large language models can self-recover corrupted visual content, leading to improved robustness and performance in visual understanding tasks.

💡 The paper introduces a new dataset and method for improving audio-visual question answering systems. Current systems typically process videos in short clips and generate separate descriptions for audio and visual modalities, which can lead to inconsistent descriptions and a lack of cross-modal reasoning. To address this, the authors propose a two-part approach: entity-anchored video scripting, which transforms videos into structured scripts with summaries, main entity lists, and segment-wise audio-visual descriptions, and clue-guided QA generation, which prompts models to mine cross-segment clues from the script and generate QA pairs based on these clues.

The entity-anchored video scripting mechanism ensures cross-segment referential consistency and reconstructs audio-visual associations, while the clue-guided QA generation mechanism encourages models to generate questions that require long-term temporal connections and deep cross-modal reasoning. The authors use this pipeline to construct a new dataset called OmniVideo-100K, which consists of structured scripts and QA pairs, as well as a human-verified test set called OmniVideo-Test.

The results show that fine-tuning models on OmniVideo-100K yields significant performance gains, with improvements of up to 20.59% on the OmniVideo-Test set. The models also demonstrate strong generalization, with improvements of up to 12.64% on established benchmarks such as Daily-Omni and JointAVBench. Overall, the paper contributes a new dataset and method for improving audio-visual question answering systems, with a focus on cross-modal reasoning and temporal consistency.

💡 The paper presents Orchestra-o1, an omnimodal agent orchestration framework that enables efficient collaboration across multiple modalities such as text, image, audio, and video. The existing agent orchestration frameworks are limited to a narrow set of modalities and struggle to generalize to complex settings where heterogeneous modalities coexist and interact. To address this limitation, Orchestra-o1 introduces a unified orchestration mechanism that enables modality-aware task decomposition, online sub-agent specialization, and parallel sub-task execution. This allows agent systems to effectively tackle complex real-world tasks involving heterogeneous information sources. The framework is trained using decision-aligned group relative policy optimization, an efficient agentic reinforcement learning approach. The results show that Orchestra-o1 achieves superior performance on complex multimodal benchmarks, surpassing the second-best approach by 10.3 percent accuracy on the OmniGAIA benchmark. Additionally, the trained Orchestra-o1-8B model achieves state-of-the-art performance against all existing open-source omnimodal agents, demonstrating the effectiveness of the proposed framework. Overall, the paper contributes to the development of omnimodal agent orchestration frameworks that can efficiently collaborate across multiple modalities, enabling the creation of more complex and powerful agent systems.

💡 The paper introduces UniverSat, a new approach to applying Vision Transformers to Earth Observation data. The problem with current Vision Transformers is that they rely on rigid patch projectors, which makes it difficult to transfer them to Earth Observation tasks where the input data can vary widely in terms of modality, scale, and resolution. To address this issue, the authors propose a Universal Patch Encoder that can map patches from arbitrary spatial, spectral, and temporal resolutions, and from both optical and non-optical sensors, into a shared embedding space using a shared set of weights. This allows a single model to be trained on heterogeneous multimodal data using self-supervision, resulting in robust and sensor-agnostic spatial features. The authors validate their approach by achieving strong results on classification and segmentation tasks using standard Earth Observation benchmarks. The key contribution of UniverSat is its ability to enable resolution- and modality-agnostic spatial feature extraction, making it a versatile and effective tool for Earth Observation tasks. The authors make their code and models available for further research and development.

💡 The paper addresses the challenges of training large scale high quality video foundation models that can generate high quality videos. Video foundation models have been used to simulate the real world and develop creative visual experiences but training them is difficult due to the complexity and size of video datasets. To overcome this the authors present a scalable open source pipeline using NVIDIA NeMo for training and inference of video foundation models. The pipeline provides accelerated video dataset curation multimodal data loading and parallelized video diffusion model training and inference. The authors also provide a comprehensive performance analysis highlighting best practices for efficient video foundation model training and inference. The pipeline is designed to address the challenges of training large scale video foundation models and provides a scalable and efficient solution for generating high quality videos. The results of the paper demonstrate the effectiveness of the pipeline in training video foundation models and provide insights into the best practices for efficient training and inference. Overall the paper contributes to the development of video foundation models by providing a scalable and efficient pipeline for training and inference.

💡 The paper introduces a unified multimodal model that formulates computer vision tasks as generation problems using natural language and visual prompts. This approach allows for a single model to perform a wide range of vision tasks without requiring task-specific architectures. The model, called SenseNova-Vision, uses natural-language instructions and optional visual prompts to specify tasks and generates responses as text, images, or mixed text-and-image outputs. To support large-scale training, the authors created the SenseNova-Vision Corpus, a computer-vision instruction-response corpus that spans text, image, and mixed targets. The model is trained on this corpus, along with auxiliary multimodal data, and achieves performance comparable to specialized systems across diverse vision tasks, including detection, OCR, keypoint estimation, segmentation, and camera pose estimation. The results demonstrate that a single unified model can match leading task-specialized systems, suggesting that unified multimodal generation is a scalable route for integrating computer vision capabilities into general-purpose foundation models. The model and corpus are publicly available, providing a valuable resource for the research community. Overall, the paper presents a significant contribution to the field of computer vision, offering a unified and flexible approach to tackling a wide range of vision tasks.

💡 The paper introduces Orca, a general world foundation model that learns a unified world latent space from multimodal data, including videos and language descriptions. The model is trained using a next-state-prediction approach, which involves predicting the next state of the world given the current state, rather than just predicting the next token, frame, or action. This approach allows the model to capture both dense natural state transitions from continuous videos and sparse meaningful state transitions from language-described events.

The model is trained on a large-scale dataset consisting of 125K hours of video data and 160M event annotations. The training process involves two complementary paradigms: unconscious learning, which captures state transitions from videos, and conscious learning, which models state transitions from language descriptions and visual question answering supervision.

The results show that Orca outperforms similar-sized specialized baselines in downstream tasks, including text generation, image prediction, and embodied action generation. The model's backbone is frozen, and only the lightweight modality-specific decoders are trainable, demonstrating the scalability of the proposed paradigm. The experiments verify that a stronger world latent space enables stronger downstream readouts, indicating that Orca presents a promising approach to understanding, predicting, and acting upon the world.

The paper's contributions include the introduction of a unified world latent space, a next-state-prediction modeling approach, and a large-scale world-learning inventory dataset. The results demonstrate the effectiveness of Orca in learning a general world foundation model that can be applied to various downstream tasks, and the paper discusses the current limitations and potential future directions for the model. Overall, the paper presents a novel approach to learning a unified representation of the world and demonstrates its potential for a wide range of applications.

💡 The paper introduces VideoChat3, a fully open, efficient, and generalist video-centric multimodal large language model for video understanding. Current open-source models are limited in several ways, struggling to generalize across diverse video types and being computationally demanding, which restricts their efficiency and scalability. Most models are also only partially open, with key components such as training code, strategy, or datasets unavailable, hindering reproducibility and slowing community-driven development.

To address these issues, VideoChat3 advances video understanding through two complementary designs. For efficiency, it introduces the Inflated 3D Vision Transformer and Adaptive Frame Resolution for Streaming Video Perception, enabling efficient spatiotemporal representation and reducing the cost of processing video inputs during training and inference. For effectiveness, it develops a scalable video data synthesis pipeline that curates three diverse, high-quality training datasets, covering general, long-form, and streaming video scenarios, which improves the model's generalization across domains.

By integrating these designs, VideoChat3 achieves a rare balance of broad generalization and computational efficiency. Experiments across general, long-form, and streaming benchmarks demonstrate that VideoChat3 surpasses prior open-source models with equal or larger parameter counts, using only 4B parameters and achieving higher efficiency. The paper's contributions include a fully open and efficient video understanding model, a scalable video data synthesis pipeline, and state-of-the-art results on various video understanding benchmarks.

💡 The paper presents RESOURCE2SKILL, a framework that extracts executable skills for software agents from human-created multimodal resources such as tutorial videos, repositories, articles, and reference artifacts. Existing skill libraries are mostly hand-written, text-centric, or derived from agent traces, leaving multimodal human resources largely underused. RESOURCE2SKILL addresses this issue by distilling these resources into executable skills, which are organized as a hierarchical multimodal Skill Wiki. Each entry in the wiki combines structured text, code, visual examples, metadata, and provenance, preserving complementary signals from different sources.

The framework allows agents to retrieve and compose relevant skills from the wiki at inference time. When coverage is insufficient, the same construction operator can acquire new skills online. The authors evaluate RESOURCE2SKILL across seven practical authoring domains and demonstrate that it improves the average overall score by 11.9 percentage points over no-skill agents and outperforms strong harness baselines in 26 of 28 main-aggregate model-domain cells.

The results confirm the value of the multimodal skill format, hierarchical organization, source diversity, selection strategy, and online acquisition. The paper's contributions include a novel framework for extracting executable skills from multimodal resources, a hierarchical multimodal Skill Wiki, and a method for online skill acquisition. Overall, RESOURCE2SKILL provides a new approach to creating skill libraries for software agents, leveraging the wealth of information available in human-created multimodal resources.

💡 The paper introduces Mage-Flow, a compact 4B-scale generative stack for efficient text-to-image generation and instruction-based image editing. The stack is built from two co-designed components: Mage-VAE, a lightweight high-fidelity latent tokenizer, and a Native-Resolution Multimodal Diffusion Transformer trained with rectified flow matching. Mage-VAE uses one-step diffusion-style encoding and decoding with anchor-latent regularization, preserving the reconstruction quality of strong public VAEs while reducing tokenization cost by more than an order of magnitude.

The stack supports flexible-resolution training and improves end-to-end training throughput by about 2.5 times, thanks to native-resolution packing and stack-level CUDA kernel fusion. Built on this foundation, the authors develop a complete model family with Base, RL-aligned, and Turbo variants for both generation and editing. The results show that Mage-Flow and Mage-Flow-Edit achieve competitive performance across standard generation and editing benchmarks, despite their compact scale.

The Turbo variants make high-resolution generation and editing practical for interactive use, generating an image in 0.59 seconds and editing an image in 1.02 seconds at 1024^2 resolution on a single NVIDIA A100 GPU, while maintaining a small memory footprint. The results demonstrate that careful tokenizer-backbone-system co-design can deliver strong high-resolution generation and editing within an efficient 4B model family. The paper addresses the problem of large-scale visual generators being costly to train, fine-tune, and deploy, and provides a solution with Mage-Flow, which is a more efficient and compact alternative.

💡 The paper introduces OmniVAE, a joint audio-video variational autoencoder that learns fine-grained semantic alignment between audio and video latent representations. Recent generative models have moved beyond silent video or standalone audio synthesis towards joint generation of synchronized audio and video, but this remains challenging due to the fundamental structural differences between the two modalities. Most existing methods use audio and video VAEs trained separately, resulting in a lack of cross-modal alignment, which leaves the downstream generative model to learn cross-modal synchronization from scratch.

OmniVAE addresses this issue by jointly training an audio-video VAE that captures temporal-semantic correspondence and aligns the two latent spaces. It uses a segment-level audio-video contrastive objective to achieve this alignment and also distills features from pre-trained modality-specific semantic encoders into each modality, improving the downstream learnability of both latent spaces.

The results of extensive experiments show that OmniVAE consistently improves the learnability of the latent spaces, translating into higher generation quality and more accurate cross-modal synchronization in downstream text-to-audio-video generation tasks. The findings underscore the importance of learning unified representations as a foundation for omnimodal modeling. Overall, OmniVAE provides a novel approach to joint audio-video generation, enabling more effective and synchronized generation of audio and video content.

💡 The paper introduces MODUS, a novel any-to-any modeling approach that enables the prediction of any modality from any combination of others within a single network. Existing any-to-any models are typically trained from scratch using encoder-decoder or diffusion architectures, which can impact their performance and prevent them from leveraging strong pre-trained decoder-only models. In contrast, MODUS treats all modalities symmetrically and supports arbitrary modalities as inputs and outputs without modality-specific heads, losses, or task pipelines. This is achieved through a decoder-only architecture that allows every modality to be both an input and an output of the same model. The resulting model, MODUS, can support a range of applications, such as chained generation through intermediate modalities or cross-modal self-verification by scoring the model's own outputs with another generated modality. The authors demonstrate that MODUS exhibits strong out-of-the-box performance and is competitive with specialist and multi-task baseline models using a single model across various benchmarks. Overall, MODUS provides a flexible and effective approach to any-to-any modeling, and all materials are open-sourced for further research and development.