🔥 SCAIL-2: Unifying Controlled Character Animation with End-to-end In-Context Conditioning

💡 The paper presents SCAIL-2, a framework for controlled character animation that enables end-to-end motion transfer from driving videos to reference characters without using intermediate representations. Prior methods relied on intermediate representations such as pose skeletons or masked backgrounds, which led to information loss. SCAIL-2 addresses this issue by directly concatenating driving videos to the sequence, allowing the model to obtain all required visual information from the input video.

To overcome the lack of end-to-end data, the authors unify sub-tasks of character animation with decoupled conditions and create a pipeline to synthesize a large dataset called MotionPair-60K, which contains heterogeneous tasks of character animation. The framework utilizes in-context mask conditioning and mode-specific RoPE as soft guidance beyond textual instructions and raw visual information.

The authors also propose Bias-Aware DPO to mitigate errors caused by synthetic discrepancies in detailed regions. This approach constructs preference items to address the issue. Extensive experiments demonstrate that SCAIL-2 substantially outperforms existing state-of-the-art approaches in various character animation tasks.

The key contributions of the paper are the development of an end-to-end character animation framework that bypasses intermediate representations, the creation of a large synthetic dataset for motion transfer, and the proposal of a novel method to address synthetic discrepancies. The results show that SCAIL-2 achieves superior performance compared to existing methods, and the authors plan to release a large subset of synthetic data and model weights to facilitate further research.

📅 Published on Jun 9

🔗 Links:
• GitHub: https://github.com/huggingface
• arXiv: https://arxiv.org/abs/2606.10804
• PDF: https://arxiv.org/pdf/2606.10804
• Project Page: https://teal024.github.io/SCAIL-2/

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://xn--r1a.website/PaperNexus

#CharacterAnimation #MotionTransfer #EndToEndLearning #InContextConditioning #ComputerVision

🔥 WorldOlympiad: Can Your World Model Survive a Triathlon?

💡 The paper introduces WorldOlympiad, a comprehensive benchmark for evaluating video-based world models. The problem with current generative models is that they often focus on visual quality, but lack physical faithfulness, geometric consistency, and interaction fidelity. To address this gap, WorldOlympiad decomposes world-model evaluation into three dimensions: physical faithfulness, geometric consistency, and interaction fidelity.

The physical track assesses whether generated videos follow interpretable rules in mechanics, thermal phenomena, and material properties using object segmentation and machine learning models as judges. The geometry track evaluates structural consistency, cross-view coherence, and camera-trajectory alignment by reconstructing generated videos with Gaussian splatting. The interaction track assesses whether generated rollouts follow complex action prompts and maintain smooth transitions across consecutive video chunks.

WorldOlympiad covers three major downstream scenarios, including gaming, robotics, and general real-world videos, capturing diverse challenges from interactive control and embodied manipulation to open-domain motion and camera dynamics. The experiments on state-of-the-art models reveal substantial gaps in physical reasoning, 3D consistency, and long-horizon interaction, underscoring the need for more structured evaluation protocols for generative world models.

The main contribution of the paper is the introduction of WorldOlympiad, a scalable and interpretable evaluation suite that exposes failure modes beyond generic video quality. The results show that current generative models have significant limitations in terms of physical faithfulness, geometric consistency, and interaction fidelity, highlighting the need for further research in these areas. Overall, WorldOlympiad provides a comprehensive framework for evaluating video-based world models and can help guide the development of more realistic and physically consistent generative models.

📅 Published on Jun 9

🔗 Links:
• GitHub: https://github.com/huggingface
• arXiv: https://arxiv.org/abs/2606.11129
• PDF: https://arxiv.org/pdf/2606.11129
• Project Page: https://alibaba-damo-academy.github.io/WorldOlympiad/

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://xn--r1a.website/PaperNexus

#WorldModelEvaluation #VideoBasedWorldModels #PhysicalFaithfulness #GeometricConsistency #InteractionFidelity

🔥 Role-Agent: Bootstrapping LLM Agents via Dual-Role Evolution

💡 The paper introduces the Role-Agent framework, which aims to improve the performance of Large Language Model agents by addressing the limitations of inefficient interaction feedback and static training environments. The problem with current Large Language Model agents is that their learning is hindered by the lack of effective feedback and the inability to adapt to changing environments, resulting in limited generalization.

To address this issue, the Role-Agent framework enables a single Large Language Model to function as both the agent and the environment, allowing for a bootstrapped co-evolution process. This framework consists of two components: World-In-Agent and Agent-In-World. The World-In-Agent component uses the Large Language Model as the agent to predict future states after each action, and the alignment between predicted and actual states is used as a reward to encourage environment-aware reasoning.

The Agent-In-World component analyzes failure modes from failed trajectories and retrieves tasks with similar failure patterns, thereby reshaping the training data distribution for targeted practice. This allows the Large Language Model to focus on improving its performance in areas where it is struggling.

The results of the experiments show that the Role-Agent framework consistently improves performance, with an average gain of over 4 percent over strong baselines. This demonstrates the effectiveness of the Role-Agent framework in improving the performance of Large Language Model agents by enabling them to adapt to changing environments and focus on targeted practice. Overall, the Role-Agent framework provides a novel approach to improving the performance of Large Language Model agents, and its results have significant implications for the development of more effective and adaptive language models.

📅 Published on Jun 9

🔗 Links:
• GitHub: https://github.com/huggingface
• arXiv: https://arxiv.org/abs/2606.10917
• PDF: https://arxiv.org/pdf/2606.10917

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://xn--r1a.website/PaperNexus

#LargeLanguageModels #AgentEnvironmentInteraction #DualRoleEvolution #BootstrappedLearning #CoEvolutionaryAlgorithms

🔥 ABot-Earth 0.5: Generative 3D Earth Model

💡 The paper presents ABot-Earth 0.5, a generative framework that creates realistic 3D environments from satellite imagery. The problem addressed is the need for large-scale 3D reconstruction, which is currently expensive and technically challenging. The authors propose a novel generative model based on 3D Gaussian Splatting representation, which is trained on a diverse set of real-world urban reconstructions. This model learns to generate realistic geometry and textures, and can synthesize novel 3D scenes conditioned solely on satellite imagery in under 10 minutes per square kilometer. The framework is designed for accessibility and includes hierarchical level-of-detail structures, allowing for real-time interactive visualization on web-based map engines. The results demonstrate exceptional realism and mitigate the sim-to-real domain gap, enabling applications such as closed-loop UAV navigation. The contributions of ABot-Earth 0.5 include providing an ultra-low-cost and high-efficiency solution for large-scale 3D reconstruction, making it accessible for global digital earth visualization and empowering the future of Embodied AI applications.

📅 Published on Jun 8

🔗 Links:
• GitHub: https://github.com/huggingface
• arXiv: https://arxiv.org/abs/2606.09967
• PDF: https://arxiv.org/pdf/2606.09967
• Project Page: https://abot-earth.amap.com/

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://xn--r1a.website/PaperNexus

#Generative3DModeling #3DGaussianSplatting #SatelliteImageryReconstruction #LargeScale3DReconstruction #GeospatialModeling

🔥 Lip Forcing: Few-Step Autoregressive Diffusion for Real-time Lip Synchronization

💡 The paper presents Lip Forcing, a novel autoregressive diffusion method for real-time video-to-video lip synchronization. The problem with existing diffusion-based lip synchronization models is that they require full-sequence bidirectional attention and many denoising steps, making them impractical for real-time inference. To address this issue, the authors propose a method that distills a large audio-conditioned bidirectional video diffusion teacher into smaller causal students.

The method involves training the students to generate each chunk in only two denoising steps without inference-time conditional feedback guidance, enabling real-time lip synchronization. The authors also analyze the tradeoff between reference fidelity and synchronization and derive three components to improve the model: Sync-Window DMD, a two-step inference schedule, and a SyncNet-based reward.

The results show that the proposed method achieves real-time performance while maintaining strong visual quality and audio-visual alignment. The authors validate Lip Forcing at two student scales, a 1.3B student and a 14B student, both distilled from the 14B teacher. The 1.3B student achieves real-time streaming at 31 FPS, 17.6 times faster than its same-scale bidirectional model. The 14B student runs 39.8 times faster than its teacher at comparable reference fidelity. Additionally, the time-to-first-frame is sub-millisecond at both scales, far below every diffusion baseline. Overall, the paper contributes a novel method for real-time lip synchronization that achieves strong performance and efficiency.

📅 Published on Jun 9

🔗 Links:
• GitHub: https://github.com/huggingface
• arXiv: https://arxiv.org/abs/2606.11180
• PDF: https://arxiv.org/pdf/2606.11180
• Project Page: https://cvlab-kaist.github.io/LipForcing/

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://xn--r1a.website/PaperNexus

#LipSynchronization #AutoregressiveDiffusion #RealTimeVideoProcessing #VideoToVideoSynthesis #DiffusionBasedModels

🔥 Test-Time Gradient Guidance of Flow Policies in Reinforcement Learning

💡 The paper proposes a reinforcement learning algorithm called QGF that improves policies at test time by using a value gradient to guide a pre-trained flow policy. The problem addressed is that incorporating flow models into reinforcement learning pipelines for policy improvement can be difficult due to stability and scalability issues. The method involves pre-training a reference flow policy and a value function critic, then using the value gradient to guide the reference policy to generate higher-value actions at test time, without any additional policy learning. This approach avoids the instability of actor-critic training and sidesteps the need for specialized training objectives or backpropagating through denoising processes. The results show that QGF outperforms prior test-time reinforcement learning methods on single-task and goal-conditioned offline benchmarks with high-dimensional action spaces, and is competitive with state-of-the-art training-time algorithms while being much cheaper to run. Additionally, QGF exhibits favorable scaling with model size, offering a practical and effective alternative reinforcement learning algorithm with expressive policies. Overall, the paper contributes a new approach to reinforcement learning that improves policies at test time, avoiding training-time instability while maintaining competitive performance.

📅 Published on Jun 9

🔗 Links:
• GitHub: https://github.com/huggingface
• arXiv: https://arxiv.org/abs/2606.11087
• PDF: https://arxiv.org/pdf/2606.11087
• Project Page: https://q-guided-flow.github.io/

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://xn--r1a.website/PaperNexus

#ReinforcementLearningAlgorithms #FlowPolicyOptimization #TestTimePolicyImprovement #ValueGradientGuidance #QGFAlgorithm

🔥 Toward Generalist Autonomous Research via Hypothesis-Tree Refinement

💡 The paper introduces Arbor, a general framework for autonomous scientific research that enables an AI agent to run a repeated loop of exploration, experimentation, and abstraction over long horizons. The problem addressed is how to design an AI system that can conduct autonomous research across multiple domains, iteratively improving research outcomes without human supervision.

The method used is a combination of strategic coordination, isolated hypothesis testing, and a persistent knowledge tree, called Hypothesis Tree Refinement. The Arbor framework consists of a long-lived coordinator that manages global research strategy, short-lived executors that implement and test individual hypotheses, and a persistent tree that links hypotheses, artifacts, evidence, and distilled insights across time.

As results return, Arbor updates the tree, propagates reusable lessons, refines the search frontier, and admits verified improvements. This design turns autonomous research from a sequence of local attempts into a cumulative process in which strategy, execution, and evidence are carried across time.

The results show that Arbor achieves the best held-out result on all six real research tasks in model training, harness engineering, and data synthesis, attaining more than 2.5 times the average relative held-out gain of other methods under the same task interface and resource budget. Additionally, Arbor reaches 86.36 percent Any Medal with GPT-5.5 on MLE-Bench Lite, the strongest result in the comparison. Overall, the paper contributes a novel framework for autonomous research that demonstrates significant improvements in research outcomes across multiple domains.

📅 Published on Jun 10

🔗 Links:
• GitHub: https://github.com/huggingface
• arXiv: https://arxiv.org/abs/2606.11926
• PDF: https://arxiv.org/pdf/2606.11926
• Project Page: https://ruc-nlpir.github.io/Arbor/

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://xn--r1a.website/PaperNexus

#AutonomousScientificResearch #HypothesisTreeRefinement #ArtificialIntelligenceInScience #GeneralistAI #AutonomousResearchSystems

🔥 i1: A Simple and Fully Open Recipe for Strong Text-to-Image Models

💡 The paper presents a comprehensive study of text-to-image diffusion models, aiming to identify key design choices and training insights that lead to strong model performance. The problem addressed is the lack of fully open models that match the performance of state-of-the-art models, which hinders further research in the field. To tackle this, the authors conducted over 300 controlled experiments, totaling 700K TPU v6e hours, to investigate modeling and data design choices in text-to-image diffusion training and inference.

The method used involved a systematic investigation of various design decisions, such as dataset mixing and text encoder adapters, to identify simple yet effective approaches to training strong models. The authors found several empirical findings, including the use of equal weighting for mixing curated datasets and the benefits of larger text encoder adapters.

The results of the study led to the development of i1, a 3B-parameter text-to-image diffusion model trained using only publicly available datasets. The i1 model is competitive with leading models on five representative benchmarks and outperforms the best existing fully open model by 29.5 absolute percentage points on average. The authors provide the i1 checkpoints, training and inference code, and the data processing pipeline, making it a fully open model that can serve as a foundation for future research in text-to-image diffusion models.

Overall, the paper contributes to the field by providing a practical foundation for open research in text-to-image diffusion models, highlighting the importance of transparency and reproducibility in AI research. The release of the i1 model and its associated code and data processing pipeline enables the research community to build upon and improve the model, driving further progress in the field.

📅 Published on Jun 9

🔗 Links:
• GitHub: https://github.com/huggingface
• arXiv: https://arxiv.org/abs/2606.11289
• PDF: https://arxiv.org/pdf/2606.11289
• Project Page: https://zlab-princeton.github.io/i1/

🤖 Models citing this paper:
• https://huggingface.co/zlab-princeton/i1-3B

📊 Datasets citing this paper:
• https://huggingface.co/datasets/zlab-princeton/i1-captions
• https://huggingface.co/datasets/zlab-princeton/i1-gptedit-tfrecord

🚀 Spaces citing this paper:
• https://huggingface.co/spaces/multimodalart/i1-3B

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://xn--r1a.website/PaperNexus

#TextToImageModels #DiffusionModels #TextEncoderAdapters #ImageSynthesis #DeepLearningModels

🔥 DeNovoSWE: Scaling Long-Horizon Environments for Generating Entire Repositories from Scratch

💡 The paper introduces DeNovoSWE, a large scale dataset designed to train code agents to generate entire software repositories from documentation, which is a challenging long horizon software engineering task. The problem addressed in this paper is the scarcity of large scale datasets for whole repository generation, which hinders the training of code agents for such tasks. To address this issue, the authors propose a method to automatically construct a dataset through a sandboxed agentic workflow, which enables scalable curation without human annotation. The dataset is constructed using a divide and conquer approach and a critic repair philosophy, and a difficulty aware trajectory filtering strategy is used to balance data quality and diversity. The DeNovoSWE dataset comprises 4818 high quality instances, each requiring the generation of a complete repository from documentation. The results show that fine tuning a code agent on DeNovoSWE substantially improves its performance on long horizon software engineering tasks, with a significant increase in score on the BeyondSWE Doc2Repo benchmark from 5.8 percent to 47.2 percent. Overall, the paper contributes to the development of code agents that can generate entire software repositories from documentation, which has the potential to significantly improve software engineering productivity.

📅 Published on Jun 9

🔗 Links:
• GitHub: https://github.com/huggingface
• arXiv: https://arxiv.org/abs/2606.10728
• PDF: https://arxiv.org/pdf/2606.10728

📊 Datasets citing this paper:
• https://huggingface.co/datasets/AweAI-Team/DeNovoSWE
• https://huggingface.co/datasets/AweAI-Team/DeNovoSWE-Trajectory-Filtered
• https://huggingface.co/datasets/AweAI-Team/DeNovoSWE-Trajectory-Raw

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://xn--r1a.website/PaperNexus

#SoftwareRepositoryGeneration #CodeAgents #LongHorizonEnvironments #AutomatedDatasetConstruction #RepositoryGenerationFromDocumentation

🔥 TRACER: Trace-Based Adaptive Cost-Efficient Routing for LLM Classification

💡 The paper introduces TRACER, a system that uses production traces to train machine learning surrogates for large language model classification. The problem addressed is that large language models can be costly to deploy and may not always be necessary for every input. The authors propose a method where a lightweight surrogate model is trained on the production logs of the large language model, which can absorb a significant portion of future traffic at near-zero marginal inference cost.

The TRACER system trains the surrogate model on the production traces and governs its deployment through a parity gate, which activates the surrogate only when its agreement with the large language model exceeds a user-specified threshold. This approach provides interpretability into the handling boundaries of the surrogate model, describing which input regions it handles, where it plateaus, and why it defers.

The results show that TRACER achieves significant surrogate coverage on a 77-class intent benchmark, with coverage ranging from 83 to 100 percent depending on the quality target. On a 150-class benchmark, the surrogate fully replaces the teacher model. Additionally, the parity gate correctly refuses deployment on a natural language inference task when the embedding representation cannot support reliable separation. The TRACER system is available as open-source software, providing a cost-efficient and interpretable solution for large language model classification.

📅 Published on Apr 16

🔗 Links:
• GitHub: https://github.com/huggingface
• arXiv: https://arxiv.org/abs/2604.14531
• PDF: https://arxiv.org/pdf/2604.14531
• Project Page: https://www.tracer.deeprecall.io

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://xn--r1a.website/PaperNexus

#LLMClassification #TraceBasedRouting #AdaptiveCostEfficientRouting #LargeLanguageModelOptimization #MachineLearningSurrogates

🔥 Embodied-R1.5: Evolving Physical Intelligence via Embodied Foundation Models

💡 The paper introduces Embodied-R1.5, a unified embodied foundation model that integrates various embodied reasoning capabilities, such as cognition, task planning, correction, and pointing, into a single architecture. The goal is to achieve general physical intelligence. To train the model, the authors developed three automated data construction pipelines, resulting in a large-scale data system of over 15 billion tokens. They also designed a multi-task balanced reinforcement learning approach to alleviate conflicts between different tasks.

The model consists of a Planner-Grounder-Corrector framework, which enables it to autonomously execute and self-correct over long-horizon tasks. With only 8 billion parameters, Embodied-R1.5 achieves state-of-the-art performance on 16 out of 24 embodied vision-language benchmarks, surpassing leading models. The model can also be fine-tuned into a vision-language agent with a small amount of data, outperforming leading models across popular manipulation benchmark suites.

The authors conducted extensive zero-shot real-robot experiments, demonstrating the model's strong generalization to the physical world. The experiments validated the model's performance in instruction following, affordance grounding, articulated object manipulation, and long-horizon complex tasks. The paper's contributions include the introduction of the Embodied-R1.5 model, the development of a large-scale data system, and the creation of an evaluation framework tailored for embodied tasks. The model weights, datasets, training code, and evaluation framework are open-sourced to facilitate future research in embodied foundation models.

The problem addressed in the paper is the development of a unified embodied foundation model that can achieve general physical intelligence. The method used to address this problem is the integration of various embodied reasoning capabilities into a single architecture, along with the development of a large-scale data system and a multi-task balanced reinforcement learning approach. The results show that Embodied-R1.5 achieves state-of-the-art performance on various benchmarks and demonstrates strong generalization to the physical world. Overall, the paper contributes to the development of embodied foundation models and has the potential to facilitate future research in this area.

📅 Published on Jun 9

🔗 Links:
• GitHub: https://github.com/huggingface
• arXiv: https://arxiv.org/abs/2606.11324
• PDF: https://arxiv.org/pdf/2606.11324
• Project Page: https://embodied-r.github.io/

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://xn--r1a.website/PaperNexus

#EmbodiedIntelligence #PhysicalReasoning #FoundationModels #CognitiveArchitectures #ArtificialGeneralIntelligence