💡 The paper introduces Agent S2, a novel compositional framework designed to improve the performance of computer use agents that automate digital tasks by interacting with graphical user interfaces. Current agents face challenges such as imprecise grounding of GUI elements, difficulties with long-horizon task planning, and performance bottlenecks due to relying on single generalist models. To address these challenges, Agent S2 delegates cognitive responsibilities across various generalist and specialist models. The framework uses a Mixture-of-Grounding technique to achieve precise GUI localization and Proactive Hierarchical Planning to dynamically refine action plans in response to evolving observations. The evaluations demonstrate that Agent S2 achieves state-of-the-art performance on three prominent computer use benchmarks, with relative improvements of 18.9% and 32.7% over leading baseline agents on the OSWorld 15-step and 50-step evaluation. Additionally, Agent S2 generalizes effectively to other operating systems and applications, surpassing previous best methods by 52.8% on WindowsAgentArena and by 16.52% on AndroidWorld. The code for Agent S2 is available, making it possible for others to build upon and further improve the framework. Overall, the paper contributes a novel approach to improving the performance of computer use agents, with significant implications for enhancing human productivity by automating digital tasks.

💡 The paper introduces DFlash, a speculative decoding framework designed to improve the speed of large language models while maintaining their quality. The problem with current large language models is that they require sequential decoding, which leads to high latency and poor GPU utilization. Speculative decoding has been proposed as a solution, where a fast draft model generates outputs that are then verified in parallel by the target model. However, existing speculative decoding methods still rely on sequential drafting, which limits their speedup.

To address this, the authors propose using a lightweight block diffusion model for parallel drafting. This model generates draft tokens in a single forward pass and conditions the draft model on context features extracted from the target model. The result is a framework that enables efficient drafting with high-quality outputs and higher acceptance rates.

The experiments show that DFlash achieves significant speedup over existing autoregressive methods, with over 6x lossless acceleration across a range of models and tasks. This is up to 2.5x higher speedup than the state-of-the-art speculative decoding method. The method contributes to improving the efficiency of large language models, making them more suitable for practical applications. Overall, DFlash offers a promising solution for speeding up large language models without sacrificing their performance.

💡 The paper introduces ARIS, an open source research harness that enables autonomous research through adversarial multi agent collaboration. The problem addressed is the reliability of long term research outcomes, particularly in cases where AI generated claims may be unsupported or misreported. The central failure mode in long horizon research workflows is not a visible breakdown but rather a plausible unsupported success, where a long running agent can produce claims with incomplete or misreported evidential support.

To address this problem, ARIS uses a cross model adversarial collaboration approach, where an executor model drives forward progress while a reviewer from a different model family critiques intermediate artifacts and requests revisions. The ARIS architecture consists of three layers: the execution layer, which provides reusable skills and model integrations, the orchestration layer, which coordinates end to end workflows, and the assurance layer, which checks the integrity of experimental claims and ensures that they are supported by evidence.

The assurance layer includes a three stage process for checking claims, as well as a five pass scientific editing pipeline, mathematical proof checks, and visual inspection of rendered PDFs. The system also includes a prototype self improvement loop that records research traces and proposes harness improvements, which are adopted only after reviewer approval.

The contributions of the paper are the introduction of the ARIS research harness, which provides a reliable and autonomous way to conduct research, and the demonstration of its effectiveness in ensuring the integrity of research outcomes. The paper also highlights the importance of adversarial collaboration in ensuring the reliability of AI generated research claims. Overall, the paper presents a significant contribution to the field of autonomous research, providing a framework for ensuring the reliability and integrity of research outcomes.

💡 The paper proposes a novel framework called PersonaLive for real-time portrait animation in live streaming scenarios. Current diffusion-based portrait animation models focus on visual quality and expression realism but neglect generation latency and real-time performance, limiting their application in live streaming. To address this, PersonaLive uses a hybrid approach with implicit signals, including implicit facial representations and 3D implicit keypoints, to control image-level motion. The framework also employs a fewer-step appearance distillation strategy to reduce appearance redundancy and improve inference efficiency. Additionally, PersonaLive introduces an autoregressive micro-chunk streaming generation paradigm with a sliding training strategy and a historical keyframe mechanism to enable low-latency and stable long-term video generation. The results show that PersonaLive achieves state-of-the-art performance with a significant speedup of up to 7-22 times over prior diffusion-based portrait animation models, making it suitable for real-time live streaming applications.

💡 The paper introduces PRISM, a three-stage pipeline designed to mitigate distributional drift in multimodal models. The standard approach of supervised fine-tuning followed by reinforcement learning with verifiable rewards often results in distributional drift, which can lead to poor performance in multimodal reasoning tasks. This drift occurs because supervised fine-tuning does not preserve the model's original capabilities and does not faithfully match the supervision distribution, and this problem is exacerbated in multimodal reasoning where perception errors and reasoning failures follow distinct drift patterns.

To address this issue, PRISM inserts an explicit distribution-alignment stage between supervised fine-tuning and reinforcement learning. This stage uses a black-box adversarial game between the policy and a Mixture-of-Experts discriminator with dedicated perception and reasoning experts. The game provides disentangled corrective signals that steer the policy toward the supervision distribution without requiring access to teacher logits.

The authors use a large number of public demonstrations for supervised fine-tuning, but they curate additional high-quality demonstrations for the distribution alignment stage. They evaluate PRISM on several multimodal benchmarks and reinforcement learning algorithms, and the results show that PRISM consistently improves downstream performance. Specifically, PRISM improves average accuracy by 4.4 and 6.0 points over the baseline approach on two different model sizes. The code, data, and model checkpoints are publicly available, making it easy for others to reproduce and build upon the results. Overall, PRISM offers a new approach to mitigating distributional drift in multimodal models, and its results demonstrate the effectiveness of this approach in improving performance on multimodal reasoning tasks.

💡 The paper introduces UniVidX, a unified multimodal framework for versatile video generation using video diffusion model priors. The problem with existing methods is that they train separate models for each task, limiting the modeling of correlations across different modalities. UniVidX addresses this issue by formulating pixel-aligned tasks as conditional generation in a shared multimodal space, allowing it to adapt to modality-specific distributions while preserving the native priors of the video diffusion model.

The framework consists of three key designs: Stochastic Condition Masking, Decoupled Gated LoRA, and Cross-Modal Self-Attention. Stochastic Condition Masking enables omni-directional conditional generation by randomly partitioning modalities into clean conditions and noisy targets during training. Decoupled Gated LoRA preserves the strong priors of the video diffusion model by introducing per-modality LoRAs that are activated when a modality serves as the generation target. Cross-Modal Self-Attention facilitates information exchange and inter-modal alignment by sharing keys and values across modalities while keeping modality-specific queries.

The authors instantiate UniVidX in two domains: UniVid-Intrinsic for RGB videos and intrinsic maps, and UniVid-Alpha for blended RGB videos and their constituent RGBA layers. The results show that both models achieve performance competitive with state-of-the-art methods across distinct tasks and generalize robustly to in-the-wild scenarios, even when trained on fewer than 1000 videos. Overall, UniVidX provides a unified framework for versatile video generation, allowing for more efficient and effective modeling of correlations across different modalities.

💡 The paper presents MolmoAct2, an open action reasoning model for robotics that improves upon previous systems in several ways. Current vision-language-action models aim to provide a single generalist controller for robots, but they have limitations, such as being closed, requiring expensive hardware, or having high latency. MolmoAct2 addresses these issues by introducing several new components, including a specialized vision-language-model backbone called MolmoER, which is trained on a large corpus of data and is designed for spatial and embodied reasoning. The model also includes three new datasets, including the largest open bimanual dataset to date, and an open-weight action tokenizer called OpenFAST. The architecture of the model has been redesigned to include a continuous-action expert and an adaptive-depth reasoning variant called MolmoThink, which reduces latency by only re-predicting depth tokens for scene regions that change between timesteps. The results of the paper show that MolmoAct2 outperforms strong baselines in several simulation and real-world benchmarks, and the model weights, training code, and training data are released for use by others. Overall, MolmoAct2 is a fully open action reasoning model that is designed for practical deployment and advances the state of the art in robotics.

💡 The paper introduces HeavySkill, a framework that internalizes complex reasoning as a skill within a model's parameters, rather than relying on external orchestration. The problem with current approaches is that they use intricate system designs that obscure the underlying mechanism driving performance. HeavySkill proposes a two-stage pipeline consisting of parallel reasoning and summarization, which can operate beneath any agentic harness. The method involves identifying heavy thinking as an inner skill that can be learned and scaled via reinforcement learning. The authors conducted a systematic empirical study of HeavySkill across diverse domains and found that it consistently outperforms traditional Best-of-N strategies. The results show that stronger language models can even approach Pass@N performance, and that the depth and width of heavy thinking can be further scaled via reinforcement learning. This offers a promising path toward self-evolving language models that internalize complex reasoning without relying on brittle orchestration layers. Overall, the paper contributes a new perspective on complex reasoning, demonstrating that internalizing heavy thinking as a skill can lead to superior performance and more robust models.

💡 The paper discusses the design and implementation of the PyTorch distributed data parallel module, which aims to optimize large-scale model training by scaling out to multiple computational resources. The need for this arises from the increasing demand for large datasets and models in deep learning research and applications. Data parallelism is a popular solution for distributed training, where the model is replicated on each resource to generate gradients independently, and then these gradients are communicated at each iteration to keep the model replicas consistent.

However, optimizing the distributed training efficiency is non-trivial due to the subtle dependencies between computation and communication. To address this, the PyTorch distributed data parallel module provides several techniques to accelerate distributed training, including gradient bucketing, computation-communication overlap, and selective synchronization.

The paper evaluates the effectiveness of these techniques and shows that when configured appropriately, the PyTorch distributed data parallel module can achieve near-linear scalability. This means that as the number of computational resources increases, the training time decreases proportionally, allowing for much faster training of large models. The evaluation results demonstrate that the module can achieve near-linear scalability using up to 256 GPUs, making it a highly effective solution for large-scale deep learning model training.

Overall, the paper contributes to the development of efficient distributed training methods, which is essential for the advancement of deep learning research and applications. The PyTorch distributed data parallel module provides a scalable and efficient solution for training large models, and its evaluation demonstrates the potential for significant speedups in training times.

💡 The paper introduces Continuous Audio Language Models, a new approach to audio generation that addresses the limitations of traditional discrete audio language models. Discrete models represent audio as sequences of discrete tokens, which are extracted from lossy codecs with limited bitrate, resulting in a trade-off between audio quality and computational cost. To overcome this issue, the authors propose Continuous Audio Language Models, which instantiate a large Transformer backbone that produces a contextual embedding at every time step. This sequential information then conditions a multilayer perceptron to generate the next continuous frame of an audio Variational Autoencoder through consistency modeling. By avoiding lossy compression, Continuous Audio Language Models achieve higher quality at lower computational cost than their discrete counterparts. Experiments on speech and music demonstrate improved efficiency and fidelity over state-of-the-art discrete audio language models, facilitating lightweight, high-quality audio generation. The approach enables the generation of high-quality audio samples, which are made available for demonstration purposes. Overall, the paper contributes a novel method for continuous audio language modeling, which has the potential to improve the efficiency and quality of audio generation tasks.