🔥 Lift4D: Harmonizing Single-View 3D Estimation for 4D Reconstruction In-the-Wild

💡 The paper presents Lift4D, a test-time optimization framework for reconstructing dynamic non-rigid objects from monocular video. The problem addressed is the difficulty in reconstructing 4D representations of dynamic objects from single-view video due to the scarcity of 4D training data and the limitations of prior approaches that either directly predict 4D representations or initialize a 3D representation and refine it based on video evidence.

The method involves adapting a single-view 3D reconstruction model to yield temporally consistent per-frame predictions, which provides a coherent initialization for a deformable 3D Gaussian Splatting representation. This representation is then optimized to match the input video through an occlusion-aware optimization that recovers visible surface details and completes unobserved regions using a view-conditioned diffusion prior.

The results show that Lift4D improves over prior 4D reconstruction methods, particularly on challenging in-the-wild sequences with severe occlusions and non-rigid motion. The framework effectively handles complex scenarios by integrating visual cues from direct observations with data-driven priors over geometry and appearance, making it a significant contribution to the field of 4D reconstruction from monocular video.

📅 Published on Jun 22

🔗 Links:
• GitHub: https://github.com/huggingface
• arXiv: https://arxiv.org/abs/2606.23688
• PDF: https://arxiv.org/pdf/2606.23688
• Project Page: https://lift4d.github.io/

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://xn--r1a.website/PaperNexus

#4DReconstruction #SingleView3DEstimation #MonocularVideoAnalysis #DynamicNonRigidObjectReconstruction #TestTimeOptimizationFrameworks

🔥 MemGUI-Agent: An End-to-End Long-Horizon Mobile GUI Agent with Proactive Context Management

💡 The paper introduces MemGUI-Agent, a mobile GUI agent designed to address the limitations of existing agents on long-horizon tasks. Current agents struggle with retaining intermediate facts across many steps and app transitions, leading to unreliable performance. This limitation is attributed to the ReAct-style prompting approach, which passively accumulates per-step records, causing prompt explosion and dilution of critical cross-app facts.

To address this issue, the authors propose MemGUI-Agent, which uses proactive context management through Context-as-Action, or ConAct. ConAct casts context management as first-class actions emitted by the same policy that selects UI actions. This approach maintains three structured context fields: folded action history, folded UI state, and recent step record, preserving critical UI facts while keeping context compact.

The authors also introduce MemGUI-3K, a dataset with 2,956 trajectories and full ConAct annotations for supervised training and offline analysis. Training an 8B model on MemGUI-3K results in MemGUI-8B-SFT, an 8B MemGUI-Agent that achieves the best open-data 8B performance on MemGUI-Bench and generalizes to the out-of-distribution MobileWorld benchmark.

The contributions of the paper are threefold. Firstly, it identifies the limitations of existing mobile GUI agents on long-horizon tasks and attributes them to the ReAct-style prompting approach. Secondly, it proposes MemGUI-Agent with proactive context management through ConAct, which addresses the limitations of existing agents. Finally, it introduces MemGUI-3K, a dataset for supervised training and offline analysis, and demonstrates the effectiveness of MemGUI-8B-SFT, an 8B MemGUI-Agent trained on this dataset. The code, data, and trained models will be released to facilitate further research and development.

📅 Published on Jun 18

🔗 Links:
• GitHub: https://github.com/huggingface
• arXiv: https://arxiv.org/abs/2606.19926
• PDF: https://arxiv.org/pdf/2606.19926
• Project Page: https://memgui-agent.github.io/

🤖 Models citing this paper:
• https://huggingface.co/lgy0404/MemGUI-8B-SFT

📊 Datasets citing this paper:
• https://huggingface.co/datasets/lgy0404/MemGUI-3K

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://xn--r1a.website/PaperNexus

#MobileGUIAutomation #LongHorizonTaskLearning #ProactiveContextManagement #ContextAsAction #EndToEndGUIAgents

🔥 MobileForge: Annotation-Free Adaptation for Mobile GUI Agents with Hierarchical Feedback-Guided Policy Optimization

💡 The paper introduces MobileForge, a system for adapting mobile graphical user interface agents to real target apps without requiring manual annotations. The problem addressed is that current mobile GUI agents require costly and time-consuming human-written tasks, demonstrations, or reward labels to adapt to new apps. Existing annotation-free GUI learning methods lack a unified approach to connect target-app exploration, curriculum mining, rollout execution, and feedback, and policy optimization often relies on isolated rollouts and coarse rewards.

MobileForge consists of two main components: MobileGym, which generates tasks and evaluates rollouts based on real mobile app interaction, and Hierarchical Feedback-Guided Policy Optimization, which uses trajectory outcomes, step-level process feedback, and corrective hints to update the policy. This approach allows for efficient adaptation of mobile GUI agents to new apps without requiring manual annotations.

The results show that MobileForge can adapt a mobile GUI agent to achieve 67.2 percent Pass@3 on AndroidWorld, which is close to the performance of a specialized model trained on closed data. Further adaptation using MobileForge reaches 77.6 percent Pass@3 on AndroidWorld and 41.0 percent success on the out-of-domain MobileWorld GUI-only split, establishing the strongest open-data mobile GUI agent in the evaluation. Overall, MobileForge provides a unified and efficient approach to adapting mobile GUI agents to new apps without requiring manual annotations, making it a significant contribution to the field.

📅 Published on Jun 18

🔗 Links:
• GitHub: https://github.com/huggingface
• arXiv: https://arxiv.org/abs/2606.19930
• PDF: https://arxiv.org/pdf/2606.19930
• Project Page: https://mobile-forge.github.io

📊 Datasets citing this paper:
• https://huggingface.co/datasets/lgy0404/mobileforge-exploration-trajectories
• https://huggingface.co/datasets/lgy0404/mobileforge-training-data
• https://huggingface.co/datasets/lgy0404/mobileforge-benchmark-results

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://xn--r1a.website/PaperNexus

#MobileGUIAgents #HierarchicalFeedbackGuidedPolicyOptimization #AnnotationFreeLearning #MobileGraphicalUserInterface #PolicyOptimizationForMobileApps

🔥 NatureBench: Can Coding Agents Match the Published SOTA of Nature-Family Papers?

💡 The paper introduces NatureBench, a benchmark of 90 scientific tasks derived from Nature publications, to assess the ability of AI coding agents to achieve discovery rather than just reproduction. The benchmark is built on NatureGym, an automated pipeline that constructs a standardized environment for each task, addressing the environment fragmentation problem that has limited the credibility of prior benchmarks. The authors evaluate ten frontier agent configurations under a strict protocol and find that the strongest model surpasses the state of the art on only 17.8 percent of tasks. The analysis reveals that agents succeed primarily through methodological translation, converting scientific tasks into familiar supervised prediction problems, rather than through genuine scientific innovation. The main reasons for failure are wrong method choice and insufficient compute budget, rather than task misunderstanding. The benchmark, pipeline, and a public leaderboard are released to facilitate further research. The paper contributes to the understanding of the limitations of current AI coding agents in achieving discovery and highlights the need for genuine scientific innovation in AI research.

📅 Published on Jun 23

🔗 Links:
• GitHub: https://github.com/huggingface
• arXiv: https://arxiv.org/abs/2606.24530
• PDF: https://arxiv.org/pdf/2606.24530
• Project Page: https://frontisai.github.io/NatureBench/

📊 Datasets citing this paper:
• https://huggingface.co/datasets/FrontisAI/NatureBench

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://xn--r1a.website/PaperNexus

#ArtificialIntelligenceInScience #NatureBench #AICodingAgents #ScientificDiscoveryWithAI #BenchmarkingAIAgents

🔥 DiffusionBench: On Holistic Evaluation of Diffusion Transformers

💡 The paper introduces a unified framework called NanoGen for training and evaluating diffusion transformers, which are used in image generation tasks. The current evaluation setup for diffusion transformers is limited to class-conditional generation on ImageNet, which may not reflect real progress in generative modeling. The authors argue that text-to-image generation is a more comprehensive task, but it is often skipped due to perceived high costs and inconvenience. However, the authors show that with NanoGen, training and evaluating text-to-image models requires comparable compute to ImageNet.

The NanoGen framework supports various diffusion methods and can be easily configured to train models on both ImageNet and text-to-image tasks. The authors trained 21 latent diffusion models using NanoGen and found that the ranking of methods on ImageNet and text-to-image tasks shows no strong correlation. This suggests that a method that improves performance on ImageNet may not necessarily improve performance on text-to-image generation.

To address this issue, the authors propose a holistic benchmark called DiffusionBench, which summarizes results on both ImageNet and text-to-image tasks. The authors recommend reporting DiffusionBench in place of ImageNet alone, as methods that improve DiffusionBench are more likely to reflect broader progress in generative modeling. The main contribution of the paper is the introduction of NanoGen and DiffusionBench, which provide a more comprehensive evaluation setup for diffusion transformers and can help to advance research in generative modeling.

📅 Published on Jun 23

🔗 Links:
• GitHub: https://github.com/huggingface
• arXiv: https://arxiv.org/abs/2606.24888
• PDF: https://arxiv.org/pdf/2606.24888
• Project Page: https://end2end-diffusion.github.io/diffusion-bench/

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://xn--r1a.website/PaperNexus

#DiffusionTransformers #ImageGenerationTasks #TextToImageGeneration #GenerativeModeling #DiffusionBasedArchitectures

🔥 AOHP: An Open-Source OS-Level Agent Harness for Personalized, Efficient and Secure Interaction

💡 The paper introduces AOHP, an open source operating system framework built on the Android Open Source Project, designed to support AI agents as first class entities. The motivation behind AOHP is to address the limitations of existing operating systems, which are application centric and do not provide native support for AI agents, resulting in execution overhead and safety risks. AOHP treats agents as first class OS actors, enabling adaptive user interfaces and agent friendly runtime environments. The framework introduces three agent oriented system mechanisms: personalized service composition, efficient agent interfaces, and secure information flow. The authors evaluated AOHP through preliminary experiments on challenging tasks and found that it shows significant advantages in task completion rate, execution cost, and security policy compliance. Specifically, AOHP achieved a 21.12 percent increase in task completion rate and a 51.55 percent reduction in token cost. The paper contributes to the research community by providing an open testbed to explore the architectural primitives desired for agent mediated interaction, and demonstrates the potential of AOHP to enhance the efficiency, security, and adoption of AI agents in operating systems.

📅 Published on Jun 22

🔗 Links:
• GitHub: https://github.com/huggingface
• Project Page: https://huggingface.co/papers?q=Android%20Open%20Source%20Project
• arXiv: https://arxiv.org/abs/2606.23449
• PDF: https://arxiv.org/pdf/2606.23449

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://xn--r1a.website/PaperNexus

#OperatingSystemSecurity #AIIntegrationInOS #AgentOrientedProgramming #OpenSourceAndroid #ArtificialIntelligenceInOS

🔥 Qwen-AgentWorld: Language World Models for General Agents

💡 The paper Qwen-AgentWorld introduces language-based world models for general agents, enabling the simulation of agentic environments across multiple domains and improving downstream task performance. The authors focus on building foundation models for agentic environment simulation and introduce two language world models, Qwen-AgentWorld-35B-A3B and Qwen-AgentWorld-397B-A17B, which can simulate agentic environments in 7 domains using long chain-of-thought reasoning.

The models were developed through a three-stage training pipeline, consisting of CPT, SFT, and RL, which inject general-purpose world modeling capabilities, activate next-state-prediction reasoning, and sharpen simulation fidelity, respectively. The training data consists of over 10 million environment interaction trajectories from 7 domains in real-world environments.

To evaluate the language world models, the authors present AgentWorldBench, a comprehensive benchmark constructed from real-world interactions of 5 frontier models on 9 established benchmarks. The results demonstrate that Qwen-AgentWorld significantly outperforms existing frontier models.

The paper also explores two complementary paradigms through which world modeling enhances general agents. First, Qwen-AgentWorld can be used as a decoupled environment simulator, supporting scalable and controllable simulation of thousands of real-world environments for agentic reinforcement learning, resulting in gains that surpass real-environment training alone. Second, world-model training can act as a highly effective warm-up, improving downstream performance across 7 agentic benchmarks.

Overall, the paper contributes to the development of language-based world models for general agents, demonstrating their potential for simulating agentic environments and improving downstream task performance. The code for Qwen-AgentWorld is made available, allowing for further research and development in this area.

📅 Published on Jun 23

🔗 Links:
• GitHub: https://github.com/huggingface
• arXiv: https://arxiv.org/abs/2606.24597
• PDF: https://arxiv.org/pdf/2606.24597

🤖 Models citing this paper:
• https://huggingface.co/Qwen/Qwen-AgentWorld-35B-A3B
• https://huggingface.co/unsloth/Qwen-AgentWorld-35B-A3B-GGUF
• https://huggingface.co/lovedheart/Qwen-AgentWorld-35B-A3B-NVFP4

📊 Datasets citing this paper:
• https://huggingface.co/datasets/Qwen/AgentWorldBench
• https://huggingface.co/datasets/XuehangCang/agentworld-safety-qa

🚀 Spaces citing this paper:
• https://huggingface.co/spaces/geraldamasi/agentworld-demo
• https://huggingface.co/spaces/tuandunghcmut/qwen-agentworld-agent
• https://huggingface.co/spaces/chabab/qwen-agentworld-35b-zerogpu

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://xn--r1a.website/PaperNexus

#LanguageWorldModels #AgenticEnvironmentSimulation #FoundationModelsForAgents #LongChainOfThoughtReasoning #AgentBasedWorldModels

🔥 DomainShuttle: Freeform Open Domain Subject-driven Text-to-video Generation

💡 The paper introduces DomainShuttle, a method for open domain subject-driven text-to-video generation that achieves high fidelity and flexibility across different scenarios. The problem with existing methods is that they focus on maximizing subject fidelity in in-domain scenarios, which limits their ability to adapt to cross-domain scenarios where subject-irrelevant properties need to vary according to the text prompt. DomainShuttle addresses this issue by introducing domain-aware modeling and a dual RoPE scheme. The method uses Domain-MoT to decouple videos and reference features, and domain-aware AdaLN for domain-specific modeling of reference images. It also uses the Video-Reference DualRoPE scheme to enable precise subject-level spatial modeling, and Cross-Pair Consistent Loss to extract intrinsic subject features unaffected by irrelevant features. The results of extensive experiments show that DomainShuttle achieves significant performance improvements over existing methods, demonstrating high subject fidelity and generative flexibility across diverse open domain application scenarios. This means that DomainShuttle can generate high-quality videos that retain the key features of the subject while also allowing for flexible editing and adaptation to different styles, semantic combinations, or domain attributes. Overall, the paper proposes a novel approach to text-to-video generation that can flexibly shuttle between different domains, making it a valuable contribution to the field of open domain subject-driven text-to-video generation.

📅 Published on Jun 24

🔗 Links:
• GitHub: https://github.com/huggingface
• arXiv: https://arxiv.org/abs/2606.26058
• PDF: https://arxiv.org/pdf/2606.26058
• Project Page: https://cn-makers.github.io/DomainShuttle/

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://xn--r1a.website/PaperNexus

#TextToVideoGeneration #OpenDomainGeneration #SubjectDrivenGeneration #DomainAwareModeling #TextToVideoSynthesis

🔥 ShutterMuse: Capture-Time Photography Guidance with MLLMs

💡 The paper introduces a new benchmark and dataset for photography assistance, focusing on capture-time guidance for both camera framing and subject pose. Existing models primarily evaluate post-hoc crop prediction and overlook subject-side recommendations, leaving a gap in the capabilities of multimodal large language models. To address this, the researchers developed CaptureGuide-Bench, a benchmark with two tasks: photographer-side composition decision and refinement, and subject-side scene-conditioned pose recommendation. They also constructed CaptureGuide-Dataset, comprising 130K samples with textual rationales and visual annotations.

The researchers then developed ShutterMuse, a unified multimodal large language model trained with supervised and reinforcement fine-tuning. ShutterMuse provides both composition guidance and pose recommendations during image capture. The evaluation reveals that general-purpose models can make composition decisions but lack precise refinement localization, while specialized aesthetic cropping models localize crops effectively but are limited to refinement and do not provide pose guidance.

The experiments on CaptureGuide-Bench show that ShutterMuse achieves the best overall photographer-side performance among evaluated baselines and competitive subject-side pose recommendation with lower inference cost. This demonstrates the potential of multimodal large language models as interactive assistants for photography during image capture, addressing the need for capture-time guidance in real-world photography. The paper contributes to the development of models that can provide effective guidance for both camera framing and subject pose, making it a significant step forward in the field of photography assistance.

📅 Published on Jun 24

🔗 Links:
• GitHub: https://github.com/huggingface
• arXiv: https://arxiv.org/abs/2606.25763
• PDF: https://arxiv.org/pdf/2606.25763
• Project Page: https://lijayutnt.github.io/ShutterMuse/

🤖 Models citing this paper:
• https://huggingface.co/ShutterMuse/ShutterMuse

📊 Datasets citing this paper:
• https://huggingface.co/datasets/ShutterMuse/CaptureGuide-Bench

🚀 Spaces citing this paper:
• https://huggingface.co/spaces/ShutterMuse/ShutterMuse-Video-Demo

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://xn--r1a.website/PaperNexus

#MultimodalLargeLanguageModels #CaptureTimeGuidance #PhotographyAssistance #CameraFramingTechniques #MultimodalLearningModels

🔥 MemSlides: A Hierarchical Memory Driven Agent Framework for Personalized Slide Generation with Multi-turn Local Revision

💡 The paper presents MemSlides, a hierarchical memory framework for personalized presentation agents that enables stable personalization and reliable local edits across multi-turn revisions. The problem addressed is that current presentation generation methods are unable to preserve stable user preferences across tasks and retain newly introduced preferences and constraints during multi-turn revision. To solve this, MemSlides separates long-term user profiles, working memory for session constraints, and tool memory for reusable execution experiences.

The method involves a hierarchical memory design that stores intent-conditioned profiles for round-0 personalization in user profile memory, carries active preferences and session constraints across revision rounds in working memory, and stores reusable execution experience for reliable localized editing in tool memory. This memory design is paired with scoped slide-local revision, which allows targeted updates to act on the smallest affected region instead of regenerating the full deck.

The results show that user profile memory improves persona-alignment judgments, tool-memory injection improves closed-loop modify behavior, and working memory is able to carry over preferences. The experiments were conducted in controlled settings, including a multi-persona, multi-intent profile bank and diagnostic matched-pair settings. The results suggest that effective personalization in presentation authoring depends on separating persistent user profiles, session-level working memory, and reusable execution experience across generation and localized revision. Overall, MemSlides provides a framework for personalized presentation agents that can generate and revise presentations in a way that is tailored to individual users and their preferences.

📅 Published on Jun 15

🔗 Links:
• GitHub: https://github.com/huggingface
• arXiv: https://arxiv.org/abs/2606.17162
• PDF: https://arxiv.org/pdf/2606.17162
• Project Page: https://memslides.github.io/

🚀 Spaces citing this paper:
• https://huggingface.co/spaces/huohua325/MemSlides

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://xn--r1a.website/PaperNexus

#PersonalizedPresentationAgents #HierarchicalMemoryFramework #MultiTurnRevision #LocalEditPreservation #IntelligentSlideGeneration

🔥 UnityShots: Memory-Driven Multi-Shot Audio-Video Generation with Boundary-Aware Gating

💡 The paper presents UnityShots, a memory-driven audio-video generation system that can generate coherent multi-shot videos. The problem addressed is that existing approaches to generating multi-shot videos either cannot scale or do not maintain consistent subject appearance and audio across video cuts. To solve this, UnityShots uses a combination of fixed-size long-term and short-term memory slots, boundary-conditioned gates, and discrete cut-type priors to maintain consistency across shots. The system consists of two streams, video and audio, where the video stream uses two fixed-size slots to store information about the opening shot and the immediately preceding shot, and the audio stream uses a reference speaker token to preserve vocal timbre. The boundary-conditioned gate updates the memory slots at every cut, and the discrete cut-type prior allows for control over transition strength between shots. The system was trained on annotated cinematic and music-video shots and evaluated on a benchmark of 200 multi-cultural multi-shot sequences. The results show that UnityShots outperforms open-source baselines on every cross-shot coherence metric and matches the performance of the strongest closed-source system. The paper also releases a benchmark of multi-shot sequences with per-shot reference identities, reference audio, and per-boundary transition labels, which can be used for future research. Overall, UnityShots provides a new approach to generating coherent multi-shot videos that can maintain consistent subject appearance and audio across video cuts.

📅 Published on Jun 19

🔗 Links:
• GitHub: https://github.com/huggingface
• arXiv: https://arxiv.org/abs/2606.21661
• PDF: https://arxiv.org/pdf/2606.21661
• Project Page: https://jackailab.github.io/Projects/UnityShots/

📊 Datasets citing this paper:
• https://huggingface.co/datasets/KlingTeam/UnityShotsBench

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://xn--r1a.website/PaperNexus

#AudioVideoGeneration #MultiShotVideo #BoundaryAwareGating #MemoryDrivenGeneration #CoherentVideoGeneration