🔥 MonkeyOCRv2: A Visual-Text Foundation Model for Document AI

💡 The paper presents MonkeyOCRv2, a visual-text foundation model for document AI. The problem addressed is that mainstream visual encoders are not effective for document images without document-oriented adaptation, as they are pre-trained on natural images and cannot handle dense text and fine-grained character strokes. To solve this, the authors construct MonkeyDocv2, the largest document-image pre-training corpus, comprising 113 million images spanning 17 languages. They propose a pre-training strategy that jointly learns image-to-text generation and pixel-level document reconstruction, aligning visual representations with textual content and preserving character strokes and layout details.

The authors conduct extensive experiments on five representative document analysis tasks, including text recognition, formular recognition, text detection, document tampering detection, and overlapping text segmentation. Replacing the original encoders with MonkeyOCRv2 consistently improves performance across all five tasks. The model is also validated as the vision encoder of multimodal large language models on more challenging tasks of document parsing and document understanding.

The results show that MonkeyOCRv2 yields a 0.7B document parsing model that sets a new open-source state-of-the-art on the MDPBench benchmark, surpassing the previous best by 2.8% absolute with a vision encoder roughly 11 times smaller. The frozen encoder also powers a document understanding model that outperforms counterparts built on CLIP, DINO, and SAM across eight benchmarks under identical training settings. These results suggest that document-oriented visual pre-training can serve as a foundation for document intelligence in its own right.

📅 Published on Jul 13

🔗 Links:
• GitHub: https://github.com/huggingface
• arXiv: https://arxiv.org/abs/2607.11562
• PDF: https://arxiv.org/pdf/2607.11562

🤖 Models citing this paper:
• https://huggingface.co/zenosai/MonkeyOCRv2-B-Parsing
• https://huggingface.co/zenosai/MonkeyOCRv2-S
• https://huggingface.co/zenosai/MonkeyOCRv2-B

🚀 Spaces citing this paper:
• https://huggingface.co/spaces/zenosai/monkey-ocr-v2

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://xn--r1a.website/PaperNexus

#DocumentAI #VisualTextModels #FoundationModels #DocumentImageAnalysis #OCRTechnology

🔥 MuScriptor: An Open Model for Multi-Instrument Music Transcription

💡 The paper addresses the problem of automatic music transcription, which is often limited to single-instrument recordings or fails on complex real music mixes. Existing methods that use synthetic training data result in models that generalize poorly, leading to largely unusable transcription output in realistic multi-instrument settings. To overcome this, the authors propose a method that combines pre-training on synthetic data with fine-tuning on real music audio and post-training using reinforcement learning. They also introduce conditioning on instrument presence to customize transcriptions. The authors release MuScriptor, an open-weight multi-instrument music transcription model that works on real-world music recordings from a diverse range of musical genres. The model is shown to be effective in transcribing music with multiple instruments, addressing a significant limitation of existing methods. By using a combination of synthetic and real data, and incorporating reinforcement learning and instrument conditioning, the authors achieve a significant improvement in music transcription accuracy, making MuScriptor a valuable tool for music information retrieval and other applications.

📅 Published on Jul 9

🔗 Links:
• GitHub: https://github.com/huggingface
• arXiv: https://arxiv.org/abs/2607.08168
• PDF: https://arxiv.org/pdf/2607.08168
• Project Page: https://muscriptor.kyutai.org/

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://xn--r1a.website/PaperNexus

#MusicTranscription #MultiInstrumentTranscription #AutomaticMusicTranscription #MusicInformationRetrieval #ReinforcementLearningForMusic

🔥 KnowAct-GUIClaw: Know Deeply, Act Perfectly, Personal GUI Assistant with Self-Evolving Memory and Skill

💡 The paper addresses the limitations of the OpenClaw framework, a leading agent framework for complex task automation, which faces insufficient cross-platform GUI interaction support and a well-built self-evolving mechanism. To resolve these issues, the authors propose the Know Deeply, Act Perfectly paradigm for personal assistants, which holds that accumulated user interaction and task-running experience directly improve execution accuracy and efficiency.

Based on this paradigm, the authors introduce KnowAct-GUIClaw, a novel Know-Route-Act-Reflect framework designed to address OpenClaw's GUI manipulation deficits and break through its cross-platform and recursive self-improvement constraints. The framework consists of two main components: the host agent, which leverages accumulated interaction experience and task-relevant knowledge for long-horizon task decomposition and allocation, and a pluggable GUI subagent with an experience-attribute memory system and self-evolving skill library, enabling seamless cross-platform migration and fast-path integration.

The framework continuously stores user profiles and feedback to improve the accuracy of task decomposition and tool calls. Extensive experiments across Android, iOS, Harmony OS, and Windows demonstrate that KnowAct-GUIClaw achieves superior efficiency, accuracy, and cross-platform adaptability. Specifically, the GUIClaw with the open-source Kimi-2.6 model achieves the best performance on the long-horizon Mobile World benchmark, beating all agentical frameworks and closed-source agentical models. Additionally, the knowledge-able memory and execution skills supported by the framework are transferable across diverse base models, improving performance by 8.5 percent with Kimi-2.6.

Overall, the paper contributes a novel framework that addresses the limitations of existing agent frameworks and achieves superior performance, accuracy, and cross-platform adaptability, making it a promising solution for personal GUI assistants.

📅 Published on Jul 15

🔗 Links:
• GitHub: https://github.com/huggingface
• arXiv: https://arxiv.org/abs/2607.12625
• PDF: https://arxiv.org/pdf/2607.12625
• Project Page: https://shibosusu.github.io/KnowAct-GUIClaw/

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://xn--r1a.website/PaperNexus

#PersonalGUIAssistants #SelfEvolvingMemory #GUIManipulation #TaskAutomationFrameworks #CrossPlatformInteraction

🔥 Boogu-Image-0.1: Boosting Open-Source Unified Multimodal Understanding and Generation

💡 The paper introduces Boogu-Image-0.1, an open-source unified multimodal understanding and generation model family that delivers competitive performance in high-quality text-to-image generation, fast inference, instruction-based editing, and bilingual text rendering. The model family consists of Base, Turbo, Edit, and Edit-Turbo variants.

The problem addressed in the paper is that closed-source multimodal systems achieve strong performance through system-level integration rather than a single model, yet their internal practices remain largely undisclosed. The authors demonstrate that targeted improvements in model understanding, data quality, and training pipelines, coupled with genetic inference-time scaling, can substantially enhance generation and editing performance even under highly constrained compute budgets.

The method used to achieve this involves making improvements in model understanding, data quality, and training pipelines. The authors also use genetic inference-time scaling to enhance performance. The model is trained on a dataset of 208.62 million unique images, and the theoretical training cost is approximately 400,000 dollars.

The results show that Boogu-Image-0.1 consistently matches or surpasses other open-source models across standard benchmarks and achieves results approaching leading closed-source systems. The authors share practical discussions and release weights, code, and recipes under Apache 2.0 to advance the open ecosystem for unified multimodal understanding and generation. The code is available on GitHub.

Overall, the paper contributes to the development of open-source multimodal models that can achieve competitive performance with closed-source systems, and provides a valuable resource for the research community.

📅 Published on Jul 14

🔗 Links:
• GitHub: https://github.com/huggingface
• arXiv: https://arxiv.org/abs/2607.13125
• PDF: https://arxiv.org/pdf/2607.13125
• Project Page: https://boogu.org/

🤖 Models citing this paper:
• https://huggingface.co/Boogu/Boogu-Image-0.1-Edit
• https://huggingface.co/Boogu/Boogu-Image-0.1-Base
• https://huggingface.co/Boogu/Boogu-Image-0.1-Turbo

🚀 Spaces citing this paper:
• https://huggingface.co/spaces/RioShiina/ImageGen
• https://huggingface.co/spaces/multimodalart/Boogu-Image
• https://huggingface.co/spaces/Tinchote/ImageGen

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://xn--r1a.website/PaperNexus

#MultimodalUnderstanding #TextToImageGeneration #OpenSourceAI #MultimodalGeneration #InstructionBasedEditing

🔥 SEED: Self-Evolving On-Policy Distillation for Agentic Reinforcement Learning

💡 The paper proposes a self-evolving framework called SEED, which stands for Self-Evolving On-Policy Distillation, to improve the performance of large language models in interactive tasks involving multi-turn interaction, tool use, and environment feedback. The problem addressed is that outcome-based reinforcement learning provides limited guidance on intermediate decisions, leaving a supervision gap between episode-level outcomes and token-level policy learning.

SEED converts completed on-policy trajectories into training-time hindsight skills and distills their behavioral effect back into the policy model. The framework first fine-tunes the policy to analyze completed trajectories and generate natural language skills that capture reusable workflows, decisive observations, or failure-avoidance rules. During reinforcement learning, the current policy both collects trajectories and serves as the analyzer that extracts hindsight skills from them.

The policy updates therefore improve subsequent decision-making and skill analysis together, allowing hindsight supervision to evolve with the policy. SEED then re-scores the sampled actions under ordinary and skill-augmented contexts, converting the skill-induced probability shift into a dense token-level on-policy distillation signal. This signal is jointly optimized with outcome-based reinforcement learning, keeping the auxiliary supervision aligned with the current trajectory distribution.

The results show that SEED consistently improves performance and sample efficiency, exhibiting robust generalization to unseen scenarios, in extensive experiments on text-based and vision-based agentic tasks. The code for SEED is available, making it possible for others to implement and build upon the framework. Overall, SEED provides a novel approach to addressing the supervision gap in outcome-based reinforcement learning, with promising results in a range of tasks.

📅 Published on Jul 16

🔗 Links:
• GitHub: https://github.com/huggingface
• arXiv: https://arxiv.org/abs/2607.14777
• PDF: https://arxiv.org/pdf/2607.14777
• Project Page: https://jinyangwu.github.io/seed/

🤖 Models citing this paper:
• https://huggingface.co/Jinyang23/Seed-AlfWorld-3B

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://xn--r1a.website/PaperNexus

#AgenticReinforcementLearning #SelfEvolvingDistillation #OnPolicyDistillation #MultiTurnInteraction #ReinforcementLearningForLanguageModels

🔥 VideoChat3: Fully Open Video MLLM for Efficient and Generalist Video Understanding

💡 The paper introduces VideoChat3, a fully open, efficient, and generalist video-centric multimodal large language model for video understanding. Current open-source models are limited in several ways, struggling to generalize across diverse video types and being computationally demanding, which restricts their efficiency and scalability. Most models are also only partially open, with key components such as training code, strategy, or datasets unavailable, hindering reproducibility and slowing community-driven development.

To address these issues, VideoChat3 advances video understanding through two complementary designs. For efficiency, it introduces the Inflated 3D Vision Transformer and Adaptive Frame Resolution for Streaming Video Perception, enabling efficient spatiotemporal representation and reducing the cost of processing video inputs during training and inference. For effectiveness, it develops a scalable video data synthesis pipeline that curates three diverse, high-quality training datasets, covering general, long-form, and streaming video scenarios, which improves the model's generalization across domains.

By integrating these designs, VideoChat3 achieves a rare balance of broad generalization and computational efficiency. Experiments across general, long-form, and streaming benchmarks demonstrate that VideoChat3 surpasses prior open-source models with equal or larger parameter counts, using only 4B parameters and achieving higher efficiency. The paper's contributions include a fully open and efficient video understanding model, a scalable video data synthesis pipeline, and state-of-the-art results on various video understanding benchmarks.

📅 Published on Jul 16

🔗 Links:
• GitHub: https://github.com/huggingface
• arXiv: https://arxiv.org/abs/2607.14935
• PDF: https://arxiv.org/pdf/2607.14935
• Project Page: https://mcg-nju.github.io/VideoChat3

🤖 Models citing this paper:
• https://huggingface.co/MCG-NJU/VideoChat3-4B
• https://huggingface.co/MCG-NJU/I3D-ViT

📊 Datasets citing this paper:
• https://huggingface.co/datasets/MCG-NJU/VideoChat3-LV116k
• https://huggingface.co/datasets/MCG-NJU/VideoChat3-Academic2M
• https://huggingface.co/datasets/MCG-NJU/VideoChat3-OL617k

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://xn--r1a.website/PaperNexus

#VideoUnderstanding #MultimodalLearning #LargeLanguageModels #VideoCentricAI #EfficientComputerVision

🔥 BadWAM: When World-Action Models Dream Right but Act Wrong

💡 This paper introduces BadWAM, a unified framework for modeling and evaluating World-Action Drift Attacks on World-Action Models. World-Action Models are a promising foundation for embodied control, learning representations that couple action generation with future world prediction. However, this paper shows that the assumption that World-Action Models are robust and safe is fragile. The authors introduce a new class of attacks that use small visual perturbations to break the alignment between what a World-Action Model imagines and what it executes. The BadWAM framework characterizes this attack surface along two natural criteria: attack strength and stealthiness. The authors evaluate BadWAM across different variants of World-Action Models and show that their attacks substantially reduce task success rates under closed-loop execution. For example, their action-only attack reduces model performance from 96.5 percent to 43.1 percent success. The results of their imagination-preserving attack further expose a World-Action Model-specific vulnerability, where moderate future-preserving regularization can maintain strong attack performance while reducing future imagination drift. The paper's contributions highlight the potential risks and vulnerabilities of World-Action Models and provide a framework for evaluating and improving their robustness and safety.

📅 Published on Jul 16

🔗 Links:
• GitHub: https://github.com/huggingface
• arXiv: https://arxiv.org/abs/2607.15207
• PDF: https://arxiv.org/pdf/2607.15207
• Project Page: https://liqiiiii.github.io/BadWAM/

📊 Datasets citing this paper:
• https://huggingface.co/datasets/cy0307/awesome-egocentric-atlas

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://xn--r1a.website/PaperNexus

#WorldActionModels #EmbodiedControl #AdversarialAttacks #WorldActionDrift #RobustnessInAI

🔥 SearchOS-V1: Towards Robust Open-Domain Information-Seeking Agent Collaboration

💡 The paper introduces SearchOS, a system level multi agent framework that aims to improve the collaboration of information seeking agents in open domain search tasks. The problem addressed is that current single and multi agent systems struggle to track task progress as interaction histories grow, leading to repetitive loops, wasted search budgets, and compromised output quality.

To address this, the authors formulate open domain information seeking as relational schema completion with grounded citations, where agents discover entities, populate attributes across linked tables, and anchor each value to source evidence. They design Search Oriented Context Management, which externalizes the evolving state into a Frontier Task, an Evidence Graph, a Coverage Map, and a Failure Memory.

Built on this, SearchOS applies a pipeline parallel scheduling mechanism that overlaps the execution of sub agents and continuously refills free slots with tasks targeting unresolved coverage gaps to improve utilization and throughput. The system also introduces a Search Tool Middleware Harness that intercepts model and tool interactions to record grounded evidence and react to stalls or budget exhaustion, and provides a reusable hierarchical skill system comprising strategy and access skills to augment the agents' search process and avoid repeating failed search patterns across runs.

The results show that SearchOS leads all metrics among the evaluated single and multi agent baselines on Wide Search and GISa, paving the way towards robust information seeking collaboration. The paper's contributions include the formulation of open domain information seeking as relational schema completion, the design of Search Oriented Context Management, and the introduction of a pipeline parallel scheduling mechanism and a Search Tool Middleware Harness, all of which work together to improve the collaboration of information seeking agents.

📅 Published on Jul 16

🔗 Links:
• GitHub: https://github.com/huggingface
• arXiv: https://arxiv.org/abs/2607.15257
• PDF: https://arxiv.org/pdf/2607.15257

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://xn--r1a.website/PaperNexus

#InformationSeekingAgents #OpenDomainSearch #MultiAgentSystems #RelationalSchemaCompletion #CollaborativeSearchFrameworks

Boost me and we both win! Sign up on Kimi and we each get a guaranteed benefit — up to 1-Year Membership Credits: https://kimi-bot.com/activities/viral-referral/share?scenario=invite&from=share_poster&invitation_code=PJMK9U

Follow the Ai Tools Daily channel on WhatsApp:
https://whatsapp.com/channel/0029VbChm8XAojYoblmIW60h

🔥 Moonshine: Speech Recognition for Live Transcription and Voice Commands

💡 The paper introduces Moonshine, a speech recognition model designed for live transcription and voice command processing. The problem addressed is the high computational requirements of traditional speech recognition models, which can be a barrier for real-time and resource-constrained applications. To solve this, the authors propose an encoder-decoder transformer architecture that uses Rotary Position Embedding instead of traditional absolute position embeddings. This approach allows the model to be trained on speech segments of various lengths without using zero-padding, making it more efficient during inference time. The results show that the Moonshine model, specifically the Moonshine Tiny version, achieves a 5x reduction in compute requirements for transcribing a 10-second speech segment compared to OpenAI's Whisper tiny.en model, without increasing word error rates on standard evaluation datasets. This makes Moonshine a promising solution for real-time and resource-constrained speech recognition applications.

📅 Published on Oct 21, 2024

🔗 Links:
• GitHub: https://github.com/huggingface
• arXiv: https://arxiv.org/abs/2410.15608
• PDF: https://arxiv.org/pdf/2410.15608

🤖 Models citing this paper:
• https://huggingface.co/UsefulSensors/moonshine
• https://huggingface.co/UsefulSensors/moonshine-base
• https://huggingface.co/UsefulSensors/moonshine-tiny

🚀 Spaces citing this paper:
• https://huggingface.co/spaces/Taf2023/higgs-audio-v3-tts
• https://huggingface.co/spaces/multimodalart/MisoTTS
• https://huggingface.co/spaces/microsoft/paza-bench

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://xn--r1a.website/PaperNexus

#SpeechRecognitionTechnology #LiveTranscriptionSystems #VoiceCommandProcessing #TransformerArchitecture #RotaryPositionEmbedding

🔥 Flavors of Moonshine: Tiny Specialized ASR Models for Edge Devices

💡 The paper challenges the common assumption that multilingual automatic speech recognition models are superior to monolingual models. Instead, the authors demonstrate that training monolingual models on a balanced mix of high-quality human-labeled, pseudo-labeled, and synthetic data can achieve better performance for small model sizes. The proposed approach, called Flavors of Moonshine, involves training tiny specialized models for underrepresented languages. The results show that these models, with only 27 million parameters, outperform larger multilingual models, including the Whisper Tiny, Whisper Small, and Whisper Medium models. On average, the Moonshine models achieve error rates 48 percent lower than the Whisper Tiny model. The authors release models for six languages, including Arabic, Chinese, Japanese, Korean, Ukrainian, and Vietnamese, under an open-source license. The contributions of this paper advance the state of the art for small automatic speech recognition models, enabling accurate on-device speech recognition for languages that previously had limited support.

📅 Published on Sep 2, 2025

🔗 Links:
• GitHub: https://github.com/huggingface
• arXiv: https://arxiv.org/abs/2509.02523
• PDF: https://arxiv.org/pdf/2509.02523

🤖 Models citing this paper:
• https://huggingface.co/UsefulSensors/moonshine-tiny-ja
• https://huggingface.co/UsefulSensors/moonshine-tiny-ar
• https://huggingface.co/UsefulSensors/moonshine-tiny-zh

🚀 Spaces citing this paper:
• https://huggingface.co/spaces/wmoto-ai/moonshine-tiny-ja-demo
• https://huggingface.co/spaces/Haitam03/whisper-tiny-ar-quran
• https://huggingface.co/spaces/Maycsz/video-transcribe-mcp

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://xn--r1a.website/PaperNexus

#AutomaticSpeechRecognition #MultilingualModeling #EdgeDeviceComputing #SpecializedASR #TinyMLModels