🔥 OmniVideo-100K: A Dataset for Audio-Visual Reasoning through Structured Scripts and Evidence Chains

💡 The paper introduces a new dataset and method for improving audio-visual question answering systems. Current systems typically process videos in short clips and generate separate descriptions for audio and visual modalities, which can lead to inconsistent descriptions and a lack of cross-modal reasoning. To address this, the authors propose a two-part approach: entity-anchored video scripting, which transforms videos into structured scripts with summaries, main entity lists, and segment-wise audio-visual descriptions, and clue-guided QA generation, which prompts models to mine cross-segment clues from the script and generate QA pairs based on these clues.

The entity-anchored video scripting mechanism ensures cross-segment referential consistency and reconstructs audio-visual associations, while the clue-guided QA generation mechanism encourages models to generate questions that require long-term temporal connections and deep cross-modal reasoning. The authors use this pipeline to construct a new dataset called OmniVideo-100K, which consists of structured scripts and QA pairs, as well as a human-verified test set called OmniVideo-Test.

The results show that fine-tuning models on OmniVideo-100K yields significant performance gains, with improvements of up to 20.59% on the OmniVideo-Test set. The models also demonstrate strong generalization, with improvements of up to 12.64% on established benchmarks such as Daily-Omni and JointAVBench. Overall, the paper contributes a new dataset and method for improving audio-visual question answering systems, with a focus on cross-modal reasoning and temporal consistency.

📅 Published on Jun 12

🔗 Links:
• GitHub: https://github.com/huggingface
• arXiv: https://arxiv.org/abs/2606.14702
• PDF: https://arxiv.org/pdf/2606.14702
• Project Page: https://yzlmhzz.github.io/OmniVideo-100K/

📊 Datasets citing this paper:
• https://huggingface.co/datasets/MiG-NJU/OmniVideo-100K
• https://huggingface.co/datasets/MiG-NJU/OmniVideo-Test

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://xn--r1a.website/PaperNexus

#AudioVisualReasoning #MultimodalLearning #VideoUnderstanding #CrossModalReasoning #AudioVisualQuestionAnswering

🔥 Orchestra-o1: Omnimodal Agent Orchestration

💡 The paper presents Orchestra-o1, an omnimodal agent orchestration framework that enables efficient collaboration across multiple modalities such as text, image, audio, and video. The existing agent orchestration frameworks are limited to a narrow set of modalities and struggle to generalize to complex settings where heterogeneous modalities coexist and interact. To address this limitation, Orchestra-o1 introduces a unified orchestration mechanism that enables modality-aware task decomposition, online sub-agent specialization, and parallel sub-task execution. This allows agent systems to effectively tackle complex real-world tasks involving heterogeneous information sources. The framework is trained using decision-aligned group relative policy optimization, an efficient agentic reinforcement learning approach. The results show that Orchestra-o1 achieves superior performance on complex multimodal benchmarks, surpassing the second-best approach by 10.3 percent accuracy on the OmniGAIA benchmark. Additionally, the trained Orchestra-o1-8B model achieves state-of-the-art performance against all existing open-source omnimodal agents, demonstrating the effectiveness of the proposed framework. Overall, the paper contributes to the development of omnimodal agent orchestration frameworks that can efficiently collaborate across multiple modalities, enabling the creation of more complex and powerful agent systems.

📅 Published on Jun 10

🔗 Links:
• GitHub: https://github.com/huggingface
• arXiv: https://arxiv.org/abs/2606.13707
• PDF: https://arxiv.org/pdf/2606.13707

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://xn--r1a.website/PaperNexus

#OmnimodalAgentOrchestration #MultimodalLearning #AgentCollaborationFrameworks #ModalityAwareTaskDecomposition #HeterogeneousModalitiesIntegration

🔥 Memory is Reconstructed, Not Retrieved: Graph Memory for LLM Agents

💡 The paper proposes a new framework called MRAgent that improves the ability of large language model agents to reason over long interaction histories. Current memory-augmented agents struggle with this task because they rely on a static retrieve-then-reason approach, which prevents them from dynamically adapting memory access to new evidence discovered during inference. To address this issue, MRAgent combines an associative memory graph with an active reconstruction mechanism. The memory graph represents information as a network of cues, tags, and contents, where tags serve as semantic bridges between cues and contents. The active reconstruction mechanism integrates language model reasoning directly into memory access, allowing the agent to iteratively explore and refine retrieval paths based on accumulated evidence. This approach enables the agent to dynamically adapt memory retrieval to the reasoning context, avoiding the need to consider all possible retrieval paths and reducing computational costs. The authors evaluate MRAgent on two benchmarks, LoCoMo and LongMemEval, and demonstrate significant improvements over strong baselines, with up to 23% better performance, while also reducing token and runtime costs. Overall, the paper contributes a new framework for long-horizon memory reasoning that is more efficient and effective than existing approaches.

📅 Published on Jun 4

🔗 Links:
• GitHub: https://github.com/huggingface
• arXiv: https://arxiv.org/abs/2606.06036
• PDF: https://arxiv.org/pdf/2606.06036

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://xn--r1a.website/PaperNexus

#GraphMemoryModels #LLMAgents #MemoryReconstruction #AssociativeMemoryGraphs #LongTermReasoningMechanisms

🔥 FastContext: Training Efficient Repository Explorer for Coding Agents

💡 The paper introduces FastContext, a dedicated exploration subagent designed to improve the efficiency of repository exploration in large language model coding agents. The problem addressed is that repository exploration is a major bottleneck in coding agents, consuming a substantial token budget and polluting the agent's context with irrelevant code snippets.

The method involves separating repository exploration from code solving using specialized exploration models. FastContext is invoked on demand and issues parallel tool calls to return concise file paths and line ranges as focused context. The exploration models used in FastContext are powered by 4B-30B parameters and are bootstrapped from strong reference-model trajectories. They are then refined with task-grounded rewards for broad first-turn search, multi-turn evidence gathering, and precise citation generation.

The results show that integrating FastContext into a coding agent improves end-to-end resolution rates by up to 5.5 percent while reducing coding-agent token consumption by up to 60 percent, with minimal overhead. The paper demonstrates that repository exploration can be effectively handled by specialized models, separate from the code solving process. The code and data for FastContext are made available, allowing for further research and development in this area. Overall, the paper presents a significant contribution to the field of coding agents and software engineering, providing a more efficient and effective approach to repository exploration.

📅 Published on Jun 12

🔗 Links:
• GitHub: https://github.com/huggingface
• arXiv: https://arxiv.org/abs/2606.14066
• PDF: https://arxiv.org/pdf/2606.14066
• Project Page: https://huggingface.co/microsoft/FastContext-1.0-4B-SFT

🤖 Models citing this paper:
• https://huggingface.co/microsoft/FastContext-1.0-4B-SFT
• https://huggingface.co/microsoft/FastContext-1.0-4B-RL

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://xn--r1a.website/PaperNexus

#EfficientRepositoryExploration #CodingAgents #LargeLanguageModels #RepositoryExplorationSubagents #SpecializedExplorationModels

🔥 JoyAI-VL-Interaction: Real-Time Vision-Language Interaction Intelligence

💡 The paper introduces a new paradigm for vision-language models, shifting from turn-based systems that require user prompting to a model that operates in real-time, making autonomous decisions about when to respond or delegate. The problem with current large models is that they only answer when addressed and do not interact in real-time, even in video-call apps. To address this, the authors propose a model that continuously watches what is happening and decides on its own whether to speak or stay silent.

The authors make two main contributions. First, they release JoyAI-VL-Interaction, an 8B-scale vision-first vision-language interaction model that makes the response decision internally, choosing each second to stay silent, respond, or delegate to a background model. The model excels at vision-triggered responsiveness and time awareness. They also provide a transferable training recipe that allows for capabilities to emerge that were not explicitly trained for, such as guiding a shopper through changing app screens or improvising a lecture from a slide deck.

Second, they release a complete deployable system built around the model, which streams any ongoing video into the model, making it genuinely present in the world. The system has pluggable components, including ASR/TTS modules, memory, visualization UI, and a background brain that can connect to any API or agent.

The results show that human raters prefer JoyAI-VL-Interaction over in-app video-call assistants by a wide margin across six real-world scenarios. This is the first open, vision-driven interaction model released together with its training recipe, data, and complete deployable system, making it a significant contribution to the field of interaction models.

📅 Published on Jun 10

🔗 Links:
• GitHub: https://github.com/huggingface
• arXiv: https://arxiv.org/abs/2606.14777
• PDF: https://arxiv.org/pdf/2606.14777
• Project Page: https://joyai-vl-video-future-academy-jd.github.io/JoyAI-VL-Interaction/

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://xn--r1a.website/PaperNexus

#VisionLanguageModels #RealTimeInteraction #AutonomousDecisionMaking #VisionFirstApproach #MultimodalIntelligence

🔥 DreamX-World 1.0: A General-Purpose Interactive World Model

💡 DreamX-World 1.0 is a general-purpose interactive text-to-video model that generates long-horizon content with camera control and scene persistence. The problem addressed by this model is the need for a controllable and interactive world model that can generate high-quality video content. To solve this problem, the authors introduced several new methods, including a lightweight variant of projective positional encoding called E-PRoPE, which retains projective camera geometry while applying camera-aware attention to spatially reduced tokens.

The authors also converted a bidirectional video generator into a few-step autoregressive world model using causal forcing, DMD-style distillation, and long-rollout training. This training process exposes the model to its own generated history, reducing style and color drift that accumulates across autoregressive chunks. Additionally, the authors introduced Memory-Conditioned Scene Persistence, which retrieves earlier views through camera-geometry-based retrieval, and residual recycling, which makes the conditioning path less sensitive to imperfect memory latents.

The model also includes Event Instruction Tuning, which adds composable event control, and reinforcement learning alignment, which recovers camera control and visual quality after distillation. To improve efficiency, the authors used mixed-precision DiT execution, residual reuse, 75%-pruned VAE decoding, and asynchronous pipeline parallelism, allowing the model to reach up to 16 FPS on eight RTX 5090 GPUs.

The results show that DreamX-World 1.0 achieves a camera-control score of 73.75 and an overall score of 84.76, outperforming HY-WorldPlay 1.5 and LingBot-World in overall score. The model's ability to generate high-quality video content with camera control and scene persistence makes it a significant contribution to the field of interactive world models. Overall, DreamX-World 1.0 is a powerful tool for generating interactive and controllable video content, with potential applications in a variety of fields, including gaming, simulation, and education.

📅 Published on Jun 15

🔗 Links:
• GitHub: https://github.com/huggingface
• Project Page: https://huggingface.co/papers?q=projective%20positional%20encoding
• arXiv: https://arxiv.org/abs/2606.16993
• PDF: https://arxiv.org/pdf/2606.16993

🤖 Models citing this paper:
• https://huggingface.co/GD-ML/DreamX-World-5B

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://xn--r1a.website/PaperNexus

#TextToVideoSynthesis #InteractiveWorldModels #VideoContentGeneration #ScenePersistence #CameraControlMechanisms

🔥 Geometric Action Model for Robot Policy Learning

💡 The paper proposes a Geometric Action Model for robot policy learning that leverages pretrained geometric foundation models to enable language-conditioned manipulation policies in 3D physical environments. The problem addressed is that current vision-language-action models and video world-action models operate primarily on 2D image frames or 2D-derived latent spaces, leaving implicit the 3D geometry required for contact-rich manipulation.

The proposed method, Geometric Action Model, repurposes a pretrained geometric foundation model as a shared substrate for perception, temporal prediction, and action decoding. It splits the model at an intermediate layer, using the shallow layers as an observation encoder and inserting a causal future predictor to forecast future latent tokens conditioned on language, proprioception, and action history. The predicted future tokens are then routed through the remaining model blocks for feature propagation and decoding, allowing a single backbone to produce both future geometry and actions.

The results show that the Geometric Action Model is more accurate, more robust, faster, and lighter than current foundation-model-scale baselines across a broad suite of simulation and real-robot manipulation benchmarks. This design equips the geometric foundation model with language-conditioned temporal world modeling through minimal architectural modification while preserving its rich geometric priors, making it a significant contribution to robot policy learning in 3D physical environments.

📅 Published on Jun 15

🔗 Links:
• GitHub: https://github.com/huggingface
• arXiv: https://arxiv.org/abs/2606.17046
• PDF: https://arxiv.org/pdf/2606.17046
• Project Page: https://cvlab-kaist.github.io/Geometric-Action-Model/

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://xn--r1a.website/PaperNexus

#GeometricDeepLearning #RobotPolicyLearning #LanguageConditionedManipulation #3DPhysicalEnvironmentModeling #GeometricFoundationModels

🔥 VibeThinker-3B: Exploring the Frontier of Verifiable Reasoning in Small Language Models

💡 The paper introduces VibeThinker-3B, a compact language model with 3 billion parameters, that achieves state-of-the-art performance on verifiable reasoning tasks, challenging the conventional assumption that large models are necessary for such tasks. The model was developed using a specialized training pipeline that includes curriculum-based supervised fine-tuning, multi-domain reinforcement learning, and offline self-distillation. The model was evaluated on several highly demanding verifiable tasks and achieved impressive results, including a score of 94.3 on AIME26, 80.2 Pass@1 on LiveCodeBench v6, and a 96.1 percent acceptance rate on recent unseen LeetCode contests. These results place VibeThinker-3B in the performance band of first-tier reasoning systems, matching or exceeding the performance of much larger models. The paper also shows that the model's performance does not compromise its instruction controllability, with a score of 93.4 on IFEval. The results of this study support the Parametric Compression-Coverage Hypothesis, which suggests that verifiable reasoning can be compressed into compact reasoning cores, while open-domain knowledge and general-purpose competence require larger models with broader parameter coverage. Overall, the paper demonstrates that compact models can be a complementary path to achieving frontier-level performance on verifiable reasoning tasks, and that they are not just efficient substitutes for larger models.

📅 Published on Jun 15

🔗 Links:
• GitHub: https://github.com/huggingface
• arXiv: https://arxiv.org/abs/2606.16140
• PDF: https://arxiv.org/pdf/2606.16140
• Project Page: https://github.com/WeiboAI/VibeThinker

🤖 Models citing this paper:
• https://huggingface.co/WeiboAI/VibeThinker-3B
• https://huggingface.co/KakTakOne/VibeThinker-3B-GGUF
• https://huggingface.co/ffkbblu/pepekberbulu

🚀 Spaces citing this paper:
• https://huggingface.co/spaces/Mike0021/vibethinker-3b-zerogpu
• https://huggingface.co/spaces/ffkbblu/trst

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://xn--r1a.website/PaperNexus

#VerifiableReasoning #SmallLanguageModels #CompactModelArchitecture #ReinforcementLearningForNLP #EfficientLanguageModeling

🔥 GLM-5: from Vibe Coding to Agentic Engineering

💡 The paper introduces GLM-5, a next-generation foundation model that advances the field of coding and software engineering. The current paradigm of vibe coding, which relies on intuitive and often imprecise coding practices, is limited in its ability to handle complex real-world software engineering tasks. To address this, the authors propose GLM-5, which builds upon the agentic, reasoning, and coding capabilities of its predecessor and incorporates several key innovations.

The method used to develop GLM-5 involves the adoption of DSA, which significantly reduces training and inference costs while maintaining long-context fidelity. Additionally, the authors implement a new asynchronous reinforcement learning infrastructure that improves post-training efficiency by decoupling generation from training. Novel asynchronous agent RL algorithms are also proposed to further improve RL quality, enabling the model to learn from complex, long-horizon interactions more effectively.

The results of the paper demonstrate the effectiveness of GLM-5, which achieves state-of-the-art performance on major open benchmarks. Most notably, GLM-5 demonstrates unprecedented capability in real-world coding tasks, surpassing previous baselines in handling end-to-end software engineering challenges. The model's ability to handle complex coding tasks and its potential to transition the paradigm of vibe coding to agentic engineering make it a significant contribution to the field of data science and software engineering. Overall, the paper presents a major advancement in foundation models and has the potential to impact the way software engineering is practiced.

📅 Published on Feb 17

🔗 Links:
• GitHub: https://github.com/huggingface
• arXiv: https://arxiv.org/abs/2602.15763
• PDF: https://arxiv.org/pdf/2602.15763
• Project Page: https://huggingface.co/spaces/GenAISecurityProject/OWASP-AIBOM-Generator

🤖 Models citing this paper:
• https://huggingface.co/zai-org/GLM-5
• https://huggingface.co/zai-org/GLM-5.1
• https://huggingface.co/zai-org/GLM-5.2

📊 Datasets citing this paper:
• https://huggingface.co/datasets/zai-org/terminal-bench-2-verified
• https://huggingface.co/datasets/harithoppil/terminal-bench-2-verified

🚀 Spaces citing this paper:
• https://huggingface.co/spaces/pliny-the-prompter/obliteratus
• https://huggingface.co/spaces/akhaliq/anycoder
• https://huggingface.co/spaces/GenAISecurityProject/OWASP-AIBOM-Generator

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://xn--r1a.website/PaperNexus

#AgenticEngineering #FoundationModels #VibeCoding #SoftwareEngineeringInnovations #ReinforcementLearningTechniques

🎁❗️TODAY FREE❗️🎁

Entry to our VIP channel is completely free today. Tomorrow it will cost $500! 🔥

JOIN 👇

https://xn--r1a.website/+n8Rs5SujPaVkZmEy
https://xn--r1a.website/+n8Rs5SujPaVkZmEy
https://xn--r1a.website/+n8Rs5SujPaVkZmEy

🔥 Next-Latent Prediction Transformers Learn Compact World Models

💡 The paper introduces Next-Latent Prediction, a method that enhances transformer architectures by adding self-supervised latent state prediction to the standard next-token training. The problem with standard transformers is that they lack an incentive to compress history into compact latent states, leading to poor generalization. To address this, the authors propose Next-Latent Prediction, which trains a transformer to learn latent representations that can predict the next latent state given the next output token. This approach injects a recurrent inductive bias into transformers, encouraging them to form compact internal world models with their own belief states and transition dynamics. The method is simple and efficient, and it does not change the architecture, parallel training, or inference of the transformer. The authors show that this approach leads to significant gains in downstream accuracy, representation compression, and lookahead planning across various benchmarks, including world modeling, reasoning, planning, and language modeling. The results demonstrate that Next-Latent Prediction is a effective paradigm for shaping transformer representations toward stronger generalization.

📅 Published on Nov 8, 2025

🔗 Links:
• GitHub: https://github.com/huggingface
• arXiv: https://arxiv.org/abs/2511.05963
• PDF: https://arxiv.org/pdf/2511.05963
• Project Page: https://jaydenteoh.github.io/blog/2026/nextlat

📊 Datasets citing this paper:
• https://huggingface.co/datasets/JaydenTeoh/manhattan

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://xn--r1a.website/PaperNexus

#NextLatentPrediction #TransformerArchitectures #SelfSupervisedLearning #LatentStatePrediction #CompactWorldModels