🔥 VideoChat3: Fully Open Video MLLM for Efficient and Generalist Video Understanding

💡 The paper introduces VideoChat3, a fully open, efficient, and generalist video-centric multimodal large language model for video understanding. Current open-source models are limited in several ways, struggling to generalize across diverse video types and being computationally demanding, which restricts their efficiency and scalability. Most models are also only partially open, with key components such as training code, strategy, or datasets unavailable, hindering reproducibility and slowing community-driven development.

To address these issues, VideoChat3 advances video understanding through two complementary designs. For efficiency, it introduces the Inflated 3D Vision Transformer and Adaptive Frame Resolution for Streaming Video Perception, enabling efficient spatiotemporal representation and reducing the cost of processing video inputs during training and inference. For effectiveness, it develops a scalable video data synthesis pipeline that curates three diverse, high-quality training datasets, covering general, long-form, and streaming video scenarios, which improves the model's generalization across domains.

By integrating these designs, VideoChat3 achieves a rare balance of broad generalization and computational efficiency. Experiments across general, long-form, and streaming benchmarks demonstrate that VideoChat3 surpasses prior open-source models with equal or larger parameter counts, using only 4B parameters and achieving higher efficiency. The paper's contributions include a fully open and efficient video understanding model, a scalable video data synthesis pipeline, and state-of-the-art results on various video understanding benchmarks.

📅 Published on Jul 16

🔗 Links:
• GitHub: https://github.com/huggingface
• arXiv: https://arxiv.org/abs/2607.14935
• PDF: https://arxiv.org/pdf/2607.14935
• Project Page: https://mcg-nju.github.io/VideoChat3

🤖 Models citing this paper:
• https://huggingface.co/MCG-NJU/VideoChat3-4B
• https://huggingface.co/MCG-NJU/I3D-ViT

📊 Datasets citing this paper:
• https://huggingface.co/datasets/MCG-NJU/VideoChat3-LV116k
• https://huggingface.co/datasets/MCG-NJU/VideoChat3-Academic2M
• https://huggingface.co/datasets/MCG-NJU/VideoChat3-OL617k

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://xn--r1a.website/PaperNexus

#VideoUnderstanding #MultimodalLearning #LargeLanguageModels #VideoCentricAI #EfficientComputerVision

🔥 BadWAM: When World-Action Models Dream Right but Act Wrong

💡 This paper introduces BadWAM, a unified framework for modeling and evaluating World-Action Drift Attacks on World-Action Models. World-Action Models are a promising foundation for embodied control, learning representations that couple action generation with future world prediction. However, this paper shows that the assumption that World-Action Models are robust and safe is fragile. The authors introduce a new class of attacks that use small visual perturbations to break the alignment between what a World-Action Model imagines and what it executes. The BadWAM framework characterizes this attack surface along two natural criteria: attack strength and stealthiness. The authors evaluate BadWAM across different variants of World-Action Models and show that their attacks substantially reduce task success rates under closed-loop execution. For example, their action-only attack reduces model performance from 96.5 percent to 43.1 percent success. The results of their imagination-preserving attack further expose a World-Action Model-specific vulnerability, where moderate future-preserving regularization can maintain strong attack performance while reducing future imagination drift. The paper's contributions highlight the potential risks and vulnerabilities of World-Action Models and provide a framework for evaluating and improving their robustness and safety.

📅 Published on Jul 16

🔗 Links:
• GitHub: https://github.com/huggingface
• arXiv: https://arxiv.org/abs/2607.15207
• PDF: https://arxiv.org/pdf/2607.15207
• Project Page: https://liqiiiii.github.io/BadWAM/

📊 Datasets citing this paper:
• https://huggingface.co/datasets/cy0307/awesome-egocentric-atlas

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://xn--r1a.website/PaperNexus

#WorldActionModels #EmbodiedControl #AdversarialAttacks #WorldActionDrift #RobustnessInAI

🔥 SearchOS-V1: Towards Robust Open-Domain Information-Seeking Agent Collaboration

💡 The paper introduces SearchOS, a system level multi agent framework that aims to improve the collaboration of information seeking agents in open domain search tasks. The problem addressed is that current single and multi agent systems struggle to track task progress as interaction histories grow, leading to repetitive loops, wasted search budgets, and compromised output quality.

To address this, the authors formulate open domain information seeking as relational schema completion with grounded citations, where agents discover entities, populate attributes across linked tables, and anchor each value to source evidence. They design Search Oriented Context Management, which externalizes the evolving state into a Frontier Task, an Evidence Graph, a Coverage Map, and a Failure Memory.

Built on this, SearchOS applies a pipeline parallel scheduling mechanism that overlaps the execution of sub agents and continuously refills free slots with tasks targeting unresolved coverage gaps to improve utilization and throughput. The system also introduces a Search Tool Middleware Harness that intercepts model and tool interactions to record grounded evidence and react to stalls or budget exhaustion, and provides a reusable hierarchical skill system comprising strategy and access skills to augment the agents' search process and avoid repeating failed search patterns across runs.

The results show that SearchOS leads all metrics among the evaluated single and multi agent baselines on Wide Search and GISa, paving the way towards robust information seeking collaboration. The paper's contributions include the formulation of open domain information seeking as relational schema completion, the design of Search Oriented Context Management, and the introduction of a pipeline parallel scheduling mechanism and a Search Tool Middleware Harness, all of which work together to improve the collaboration of information seeking agents.

📅 Published on Jul 16

🔗 Links:
• GitHub: https://github.com/huggingface
• arXiv: https://arxiv.org/abs/2607.15257
• PDF: https://arxiv.org/pdf/2607.15257

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://xn--r1a.website/PaperNexus

#InformationSeekingAgents #OpenDomainSearch #MultiAgentSystems #RelationalSchemaCompletion #CollaborativeSearchFrameworks

Boost me and we both win! Sign up on Kimi and we each get a guaranteed benefit — up to 1-Year Membership Credits: https://kimi-bot.com/activities/viral-referral/share?scenario=invite&from=share_poster&invitation_code=PJMK9U

Follow the Ai Tools Daily channel on WhatsApp:
https://whatsapp.com/channel/0029VbChm8XAojYoblmIW60h

🔥 Moonshine: Speech Recognition for Live Transcription and Voice Commands

💡 The paper introduces Moonshine, a speech recognition model designed for live transcription and voice command processing. The problem addressed is the high computational requirements of traditional speech recognition models, which can be a barrier for real-time and resource-constrained applications. To solve this, the authors propose an encoder-decoder transformer architecture that uses Rotary Position Embedding instead of traditional absolute position embeddings. This approach allows the model to be trained on speech segments of various lengths without using zero-padding, making it more efficient during inference time. The results show that the Moonshine model, specifically the Moonshine Tiny version, achieves a 5x reduction in compute requirements for transcribing a 10-second speech segment compared to OpenAI's Whisper tiny.en model, without increasing word error rates on standard evaluation datasets. This makes Moonshine a promising solution for real-time and resource-constrained speech recognition applications.

📅 Published on Oct 21, 2024

🔗 Links:
• GitHub: https://github.com/huggingface
• arXiv: https://arxiv.org/abs/2410.15608
• PDF: https://arxiv.org/pdf/2410.15608

🤖 Models citing this paper:
• https://huggingface.co/UsefulSensors/moonshine
• https://huggingface.co/UsefulSensors/moonshine-base
• https://huggingface.co/UsefulSensors/moonshine-tiny

🚀 Spaces citing this paper:
• https://huggingface.co/spaces/Taf2023/higgs-audio-v3-tts
• https://huggingface.co/spaces/multimodalart/MisoTTS
• https://huggingface.co/spaces/microsoft/paza-bench

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://xn--r1a.website/PaperNexus

#SpeechRecognitionTechnology #LiveTranscriptionSystems #VoiceCommandProcessing #TransformerArchitecture #RotaryPositionEmbedding

🔥 Flavors of Moonshine: Tiny Specialized ASR Models for Edge Devices

💡 The paper challenges the common assumption that multilingual automatic speech recognition models are superior to monolingual models. Instead, the authors demonstrate that training monolingual models on a balanced mix of high-quality human-labeled, pseudo-labeled, and synthetic data can achieve better performance for small model sizes. The proposed approach, called Flavors of Moonshine, involves training tiny specialized models for underrepresented languages. The results show that these models, with only 27 million parameters, outperform larger multilingual models, including the Whisper Tiny, Whisper Small, and Whisper Medium models. On average, the Moonshine models achieve error rates 48 percent lower than the Whisper Tiny model. The authors release models for six languages, including Arabic, Chinese, Japanese, Korean, Ukrainian, and Vietnamese, under an open-source license. The contributions of this paper advance the state of the art for small automatic speech recognition models, enabling accurate on-device speech recognition for languages that previously had limited support.

📅 Published on Sep 2, 2025

🔗 Links:
• GitHub: https://github.com/huggingface
• arXiv: https://arxiv.org/abs/2509.02523
• PDF: https://arxiv.org/pdf/2509.02523

🤖 Models citing this paper:
• https://huggingface.co/UsefulSensors/moonshine-tiny-ja
• https://huggingface.co/UsefulSensors/moonshine-tiny-ar
• https://huggingface.co/UsefulSensors/moonshine-tiny-zh

🚀 Spaces citing this paper:
• https://huggingface.co/spaces/wmoto-ai/moonshine-tiny-ja-demo
• https://huggingface.co/spaces/Haitam03/whisper-tiny-ar-quran
• https://huggingface.co/spaces/Maycsz/video-transcribe-mcp

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://xn--r1a.website/PaperNexus

#AutomaticSpeechRecognition #MultilingualModeling #EdgeDeviceComputing #SpecializedASR #TinyMLModels

🔥 Self-Improvements in Modern Agentic Systems: A Survey

💡 The paper presents a survey of self-improving autonomous agents that are transitioning from research prototypes to deployed systems. The primary goal of these agents is to achieve controllable evolution or adaptation from experience with minimal or no human input. The authors frame modern self-improving agents as adaptive systems that convert experience into accumulated capability gains. They propose a system-level framework that represents a modern agent as a configuration coupling a foundation model with an operational scaffold of prompts, memory, tools, and control logic. Within this framework, self-improvement is formalized as a self-induced update operator that obtains and commits updates to model parameters or scaffold components.

The authors organize prior work by update target and by the signals that drive change, and then review applications and discuss evaluation. They also identify open problems and future directions. The paper provides a comprehensive overview of the current state of self-improving agents and offers a framework for understanding and developing these systems. The authors also provide a resource for tracking technical updates on self-improving agents.

The paper's contributions include a systematic framework for understanding self-improving agents, a review of prior work in the field, and an identification of open problems and future directions. The authors also provide a resource for tracking technical updates in the field. Overall, the paper provides a comprehensive overview of self-improving agents and offers a framework for understanding and developing these systems. The paper's results highlight the potential of self-improving agents to achieve controllable evolution or adaptation from experience with minimal or no human input, and identify areas for future research and development.

📅 Published on Jul 14

🔗 Links:
• GitHub: https://github.com/huggingface
• arXiv: https://arxiv.org/abs/2607.13104
• PDF: https://arxiv.org/pdf/2607.13104
• Project Page: https://selfimproving-agent.github.io/

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://xn--r1a.website/PaperNexus

#AgenticSystems #AutonomousAgents #SelfImprovingSystems #ArtificialIntelligenceEvolution #AdaptiveSystemDesign

🔥 SmolVLA: A Vision-Language-Action Model for Affordable and Efficient Robotics

💡 The paper introduces SmolVLA, a compact and efficient vision-language-action model designed for affordable and efficient robotics. The problem with existing vision-language-action models is that they are typically very large, with billions of parameters, which leads to high training costs and limited deployability on consumer-grade hardware. These models also rely on academic and industrial datasets, overlooking community-collected data from affordable robotic platforms.

To address this issue, the authors developed SmolVLA, a small and efficient model that can be trained on a single GPU and deployed on consumer-grade GPUs or even CPUs. SmolVLA is designed to be community-driven, using community-collected data from affordable robotic platforms, which reduces both training and inference costs while retaining competitive performance.

The method used to achieve this involves adapting vision-language models into vision-language-action models, but with a much smaller size. The authors also introduced an asynchronous inference stack that decouples perception and action prediction from action execution, allowing for higher control rates with chunked action generation.

The results show that SmolVLA achieves performance comparable to vision-language-action models that are 10 times larger, despite its compact size. The model is evaluated on a range of simulated and real-world robotic benchmarks, and the authors release all code, pretrained models, and training data. Overall, SmolVLA provides a more efficient and affordable solution for robotics, making it possible to deploy vision-language-action models on consumer-grade hardware.

📅 Published on Jun 2, 2025

🔗 Links:
• GitHub: https://github.com/huggingface
• arXiv: https://arxiv.org/abs/2506.01844
• PDF: https://arxiv.org/pdf/2506.01844
• Project Page: https://huggingface.co/blog/smolvla

🤖 Models citing this paper:
• https://huggingface.co/lerobot/smolvla_base
• https://huggingface.co/HuggingFaceVLA/smolvla_libero
• https://huggingface.co/jadechoghari/smolvla_metaworld

📊 Datasets citing this paper:
• https://huggingface.co/datasets/lerobot/community_dataset_v3
• https://huggingface.co/datasets/HuggingFaceVLA/community_dataset_v1
• https://huggingface.co/datasets/HuggingFaceVLA/community_dataset_v2

🚀 Spaces citing this paper:
• https://huggingface.co/spaces/HuggingFaceVLA/libero-vla-leaderboard
• https://huggingface.co/spaces/lerobot/robot-learning-tutorial
• https://huggingface.co/spaces/arpitg1304/lerobot_scripts_simplified

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://xn--r1a.website/PaperNexus

#VisionLanguageAction #EfficientRobotics #CompactNeuralNetworks #RoboticsForAll #AffordableAI

🔥 Xiaomi-Robotics-1: Scaling Vision-Language-Action Models with over 100K Hours of Real-World Trajectories

💡 The paper presents Xiaomi-Robotics-1, a foundational vision-language-action model that can follow diverse language instructions to perform a wide range of mobile manipulation tasks in unseen environments. The model is trained using a two-stage training recipe consisting of pre-training and post-training. During pre-training, the model is trained on over 100,000 hours of real-world manipulation trajectories collected via UM devices, and a scalable auto-labeling pipeline is developed to annotate trajectory clips with natural language descriptions of scene state transitions. This provides rich and precise conditioning for action learning.

During post-training, the model is fine-tuned to align with robot embodiment and imperative instructions that humans naturally use to prompt robots. The experiments demonstrate strong scaling behavior, with the model consistently improving with increased data scales and model sizes during pre-training. This scaling behavior directly transfers to post-training, where stronger pre-training models yield better out-of-the-box real-robot performance in unseen environments.

The results show that Xiaomi-Robotics-1 serves as a strong robot foundation policy that can be efficiently fine-tuned on complex, dexterous tasks with high data efficiency. Across multiple simulation benchmarks, Xiaomi-Robotics-1 outperforms state-of-the-art methods, establishing a new state-of-the-art with a 57.6% success rate on RoboCasa 365, surpassing the previous best of 46.6%. Additionally, it achieves an average score of 20.07 on RoboDojo, significantly outperforming the prior state-of-the-art. The code and model checkpoints will be released.

Overall, the paper contributes to the development of a robust and scalable vision-language-action model that can be applied to a wide range of robotic tasks, and demonstrates the effectiveness of the proposed two-stage training recipe and auto-labeling pipeline in achieving state-of-the-art performance.

📅 Published on Jul 16

🔗 Links:
• GitHub: https://github.com/huggingface
• arXiv: https://arxiv.org/abs/2607.15330
• PDF: https://arxiv.org/pdf/2607.15330
• Project Page: https://robotics.xiaomi.com/xiaomi-robotics-1.html

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://xn--r1a.website/PaperNexus

#VisionLanguageModels #MobileManipulation #RoboticsLearning #RealWorldTrajectories #LanguageActionInterfaces

🔥 RAGU: A Multi-Step GraphRAG Engine with a Compact Domain-Adapted LLM

💡 The paper introduces RAGU, a multi-step graph retrieval-augmented generation engine that enhances large language models with structured knowledge. Existing systems construct knowledge graphs in a single extraction pass, producing noisy entities and brittle retrieval. RAGU addresses this by separating extraction from consolidation, using a two-stage typed extraction process, DBSCAN-backed deduplication, LLM summarization, and Leiden community detection.

A key insight is that the skills an in-pipeline LLM needs, such as comprehension, extraction, and reasoning over context, are language skills that grow only weakly with model size, unlike factual world knowledge. Therefore, the authors train MENO-LITE-0.1, a 7B model optimized for language skills, which outperforms Qwen 2.5-32B on knowledge-graph construction and matches it on English GraphRAG tasks.

The results show that RAGU retrieves the most complete context at every factoid level, with evidence recall up to 0.84, and overtakes HippoRAG 2 on synthesis tasks. On multi-hop factoid QA, the apparent HippoRAG 2 advantage is shown to be largely an answer-format artifact. RAGU is installable via pip install graphragu, runs on a single GPU, and is released under MIT. The source code and MENO-LITE-0.1 model are publicly available. Overall, RAGU provides a more effective and efficient approach to graph retrieval-augmented generation, with significant improvements in knowledge-graph construction and question answering tasks.

📅 Published on Jul 13

🔗 Links:
• GitHub: https://github.com/huggingface
• arXiv: https://arxiv.org/abs/2607.11683
• PDF: https://arxiv.org/pdf/2607.11683

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://xn--r1a.website/PaperNexus

#GraphRetrievalAugmentedGeneration #DomainAdaptedLLM #MultiStepGraphEngine #KnowledgeGraphConstruction #RetrievalAugmentedGenerationModels

🔥 RESOURCE2SKILL: Distilling Executable Agent Skills from Human-Created Multimodal Resources

💡 The paper presents RESOURCE2SKILL, a framework that extracts executable skills for software agents from human-created multimodal resources such as tutorial videos, repositories, articles, and reference artifacts. Existing skill libraries are mostly hand-written, text-centric, or derived from agent traces, leaving multimodal human resources largely underused. RESOURCE2SKILL addresses this issue by distilling these resources into executable skills, which are organized as a hierarchical multimodal Skill Wiki. Each entry in the wiki combines structured text, code, visual examples, metadata, and provenance, preserving complementary signals from different sources.

The framework allows agents to retrieve and compose relevant skills from the wiki at inference time. When coverage is insufficient, the same construction operator can acquire new skills online. The authors evaluate RESOURCE2SKILL across seven practical authoring domains and demonstrate that it improves the average overall score by 11.9 percentage points over no-skill agents and outperforms strong harness baselines in 26 of 28 main-aggregate model-domain cells.

The results confirm the value of the multimodal skill format, hierarchical organization, source diversity, selection strategy, and online acquisition. The paper's contributions include a novel framework for extracting executable skills from multimodal resources, a hierarchical multimodal Skill Wiki, and a method for online skill acquisition. Overall, RESOURCE2SKILL provides a new approach to creating skill libraries for software agents, leveraging the wealth of information available in human-created multimodal resources.

📅 Published on Jul 16

🔗 Links:
• GitHub: https://github.com/huggingface
• arXiv: https://arxiv.org/abs/2606.29538
• PDF: https://arxiv.org/pdf/2606.29538
• Project Page: https://aka.ms/Resources2Skill

📊 Datasets citing this paper:
• https://huggingface.co/datasets/microsoft/RESOURCE2SKILL

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://xn--r1a.website/PaperNexus

#MultimodalLearning #ExecutableAgentSkills #HumanCentricAI #SkillExtraction #MultimodalKnowledgeRepresentation