✨AIonopedia: an LLM agent orchestrating multimodal learning for ionic liquid discovery

📝 Summary:
AIonopedia is an LLM agent that orchestrates multimodal learning for Ionic Liquid discovery. It enables accurate property predictions and molecular design through hierarchical search, validated by real-world wet-lab experiments, significantly accelerating IL discovery.

🔹 Publication Date: Published on Nov 14

🔹 Paper Links:
• arXiv Page: https://arxiv.org/abs/2511.11257
• PDF: https://arxiv.org/pdf/2511.11257

==================================

For more data science resources:
✓ https://xn--r1a.website/DataScienceT

#LLMAgents #IonicLiquids #MultimodalLearning #MaterialsScience #AIforScience

✨Uni-MoE-2.0-Omni: Scaling Language-Centric Omnimodal Large Model with Advanced MoE, Training and Data

📝 Summary:
Uni-MoE 2.0-Omni is an open-source omnimodal large model improving multimodal understanding, reasoning, and generation. It uses dynamic MoE and progressive training to achieve state-of-the-art results across 85 benchmarks, outperforming leading models like Qwen2.5-Omni.

🔹 Publication Date: Published on Nov 16

🔹 Paper Links:
• arXiv Page: https://arxiv.org/abs/2511.12609
• PDF: https://arxiv.org/pdf/2511.12609
• Project Page: https://idealistxy.github.io/Uni-MoE-v2.github.io/
• Github: https://github.com/HITsz-TMG/Uni-MoE

🔹 Models citing this paper:
• https://huggingface.co/HIT-TMG/Uni-MoE-2.0-Omni
• https://huggingface.co/HIT-TMG/Uni-MoE-2.0-Base
• https://huggingface.co/HIT-TMG/Uni-MoE-2.0-Image

==================================

For more data science resources:
✓ https://xn--r1a.website/DataScienceT

#OmnimodalAI #LLMs #MixtureOfExperts #MultimodalLearning #AIResearch

🔥 PaddleOCR-VL: Boosting Multilingual Document Parsing via a 0.9B Ultra-Compact Vision-Language Model

💡 The paper proposes PaddleOCR-VL, a state-of-the-art and resource-efficient model for document parsing. The problem addressed is the need for a model that can accurately recognize elements in documents, such as text, tables, formulas, and charts, while being efficient in terms of resource consumption. To solve this problem, the authors propose a vision-language model that combines a NaViT-style dynamic resolution visual encoder with the ERNIE language model. The resulting model, PaddleOCR-VL-0.9B, is a compact yet powerful model that can support 109 languages and recognize complex elements with high accuracy. The method used to achieve this is the integration of the visual encoder and language model, which enables the model to efficiently process documents and recognize elements. The results show that PaddleOCR-VL achieves state-of-the-art performance in both page-level document parsing and element-level recognition, outperforming existing solutions and exhibiting strong competitiveness against top-tier vision-language models. The model also delivers fast inference speeds, making it highly suitable for practical deployment in real-world scenarios. The code for the model is available, making it accessible for further research and development. Overall, the paper contributes a highly efficient and accurate model for document parsing, which can be used in a variety of applications.

📅 Published on Oct 16, 2025

🔗 Links:
• arXiv: https://arxiv.org/abs/2510.14528
• PDF: https://arxiv.org/pdf/2510.14528
• GitHub: https://github.com/PaddlePaddle/PaddleOCR ⭐ 77.1k

🤖 Models citing this paper:
• https://huggingface.co/PaddlePaddle/PaddleOCR-VL
• https://huggingface.co/PaddlePaddle/PP-DocLayoutV2
• https://huggingface.co/unsloth/PaddleOCR-VL

📊 Datasets citing this paper:
• https://huggingface.co/datasets/proxectonos/corpus_dominio_cientifico

🚀 Spaces citing this paper:
• https://huggingface.co/spaces/PaddlePaddle/PaddleOCR-VL_Online_Demo
• https://huggingface.co/spaces/eduagarcia/multilingual-tokenizer-leaderboard
• https://huggingface.co/spaces/waytoAGI/PaddleOCR-VL_Online_Demo

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://xn--r1a.website/PaperNexus

#MultilingualDocumentParsing #VisionLanguageModels #DocumentAnalysis #TableRecognition #MultimodalLearning

🔥 RLDX-1 Technical Report

💡 The paper introduces RLDX-1, a general-purpose robotic policy for dexterous manipulation that addresses the limitations of existing vision-language-action models. These models have shown progress in human-like generalist robotic policies but struggle with complex real-world tasks that require broader functional capabilities such as motion awareness, memory-aware decision making, and physical sensing. To overcome this, RLDX-1 uses a Multi-Stream Action Transformer architecture that integrates heterogeneous modalities through modality-specific streams with cross-modal joint self-attention. This architecture is combined with system-level design choices including synthesizing training data for rare manipulation scenarios, learning procedures specialized for human-like manipulation, and inference optimizations for real-time deployment. The results show that RLDX-1 outperforms recent frontier vision-language-action models across both simulation benchmarks and real-world tasks, achieving success rates of 86.8 percent in ALLEX humanoid tasks compared to around 40 percent for other models. This positions RLDX-1 as a promising step toward reliable vision-language-action models for complex and dynamic real-world dexterous manipulation. The method and results demonstrate the ability of RLDX-1 to control a high-degree-of-freedom humanoid robot under diverse functional demands, highlighting its potential for complex real-world tasks.

📅 Published on May 5

🔗 Links:
• arXiv: https://arxiv.org/abs/2605.03269
• PDF: https://arxiv.org/pdf/2605.03269
• Project Page: http://rlwrld.ai/rldx-1
• GitHub: https://github.com/RLWRLD/RLDX-1 ⭐ 75

🤖 Models citing this paper:
• https://huggingface.co/RLWRLD/RLDX-1-PT
• https://huggingface.co/RLWRLD/RLDX-1-FT-ROBOCASA
• https://huggingface.co/RLWRLD/RLDX-1-MT-ALLEX

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://xn--r1a.website/PaperNexus

#RoboticManipulation #DexterousRobotics #VisionLanguageAction #MultiModalLearning #RobotPolicyLearning

🔥 D-OPSD: On-Policy Self-Distillation for Continuously Tuning Step-Distilled Diffusion Models

💡 The paper introduces D-OPSD, a new training approach for diffusion models that enables efficient supervised fine-tuning while preserving few-step inference capabilities. The current landscape of high-performance image generation models is shifting from inefficient multi-step models to efficient few-step models, but these models are challenging to fine-tune using traditional techniques. The problem with traditional fine-tuning methods is that they compromise the model's inherent few-step inference capability.

To address this issue, the authors propose D-OPSD, which leverages on-policy self-distillation with text and multimodal features. The method works by making the model act as both the teacher and the student, where the student is conditioned only on the text feature, and the teacher is conditioned on the multimodal feature of both the text prompt and the target image. The training process minimizes the difference between the predicted distributions over the student's own roll-outs, allowing the model to learn new concepts and styles without sacrificing its original few-step capacity.

The key contribution of D-OPSD is that it enables on-policy learning during supervised fine-tuning, which allows the model to learn from its own trajectory and under its own supervision. This approach enables the model to inherit the in-context capabilities of its encoder, making it possible to fine-tune the model continuously without compromising its few-step inference capability. The results show that D-OPSD enables efficient supervised fine-tuning for diffusion models, making it a promising approach for high-performance image generation models.

📅 Published on May 6

🔗 Links:
• arXiv: https://arxiv.org/abs/2605.05204
• PDF: https://arxiv.org/pdf/2605.05204
• Project Page: https://vvvvvjdy.github.io/d-opsd/
• GitHub: https://github.com/vvvvvjdy/D-OPSD ⭐ 24

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://xn--r1a.website/PaperNexus

#DiffusionModels #SelfDistillation #FewShotLearning #ImageGeneration #MultimodalLearning

🔥 CapVector: Learning Transferable Capability Vectors in Parametric Space for Vision-Language-Action Models

💡 This paper proposes a novel approach called CapVector to improve the performance of vision-language-action models. The problem addressed is that pre-trained models often fail to improve performance and reduce adaptation costs during standard supervised finetuning. Advanced finetuning methods with auxiliary training objectives can improve performance but incur significant computational overhead.

The proposed method decouples the auxiliary training objectives from standard supervised finetuning to enhance model capabilities while reducing computational overhead. This is achieved by training the model to converge on a small-scale task set using two distinct training strategies, resulting in two finetuned models. The parameters difference between the two models is interpreted as capability vectors provided by auxiliary objectives. These vectors are then merged with pre-trained parameters to form a capability-enhanced meta model.

The method also uses a lightweight orthogonal regularization loss to augment standard supervised finetuning, which reduces computational overhead. The results show that the capability vectors are effective and versatile across diverse models, and can generalize to novel environments and embodiments without additional training. The proposed approach achieves performance comparable to auxiliary finetuned baselines with reduced computational overhead, making it a promising solution for improving vision-language-action models.

📅 Published on May 11

🔗 Links:
• arXiv: https://arxiv.org/abs/2605.10903
• PDF: https://arxiv.org/pdf/2605.10903
• Project Page: https://capvector.github.io/
• GitHub: https://github.com/OpenHelix-Team/CapVector ⭐ 26

🤖 Models citing this paper:
• https://huggingface.co/haofuly/capvector_models_collection

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://xn--r1a.website/PaperNexus

#VisionLanguageModels #ParametricSpaceLearning #TransferableCapabilities #VisionLanguageAction #MultimodalLearning

🔥 Semantic Generative Tuning for Unified Multimodal Models

💡 The paper addresses the issue of unified multimodal models where visual understanding and generation are not well aligned due to separate training objectives. The prevailing approach of optimizing understanding through text signals and generation through pixel objectives leads to isolated representation spaces. To bridge this gap, the authors propose a novel approach called Semantic Generative Tuning, which uses semantic segmentation as a generative proxy to align and synergize multimodal capabilities.

The method involves formulating hierarchical visual tasks as generative proxies, with a focus on high-level semantic tasks like image segmentation. The authors find that segmentation provides structural semantics that enhance both vision-centric perception and generative layout fidelity. Unlike low-level tasks, segmentation does not distract models with texture details, making it an optimal proxy.

The results show that Semantic Generative Tuning fundamentally improves feature linear separability and optimizes visual-textual attention allocation patterns. Extensive evaluations demonstrate that this approach consistently improves both multimodal comprehension and generative fidelity across mainstream benchmarks. The authors provide a systematic investigation into generative post-training and introduce a new paradigm that leverages segmentation to align multimodal capabilities. The code for the proposed method is made available for further research and development. Overall, the paper presents a significant contribution to the field of unified multimodal models by introducing a novel approach that enhances multimodal alignment and performance.

📅 Published on May 18

🔗 Links:
• GitHub: https://github.com/huggingface
• arXiv: https://arxiv.org/abs/2605.18714
• PDF: https://arxiv.org/pdf/2605.18714
• Project Page: https://song2yu.github.io/SGT/

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://xn--r1a.website/PaperNexus

#MultimodalLearning #SemanticSegmentation #GenerativeModels #UnifiedMultimodalModels #MultimodalRepresentationLearning

🔥 LLaVA-OneVision-2: Towards Next-Generation Perceptual Intelligence

💡 The paper introduces LLaVA-OneVision-2, a vision-language model that achieves superior performance across various multimodal benchmarks. The problem addressed is the need for a more capable model that can efficiently process and understand video content. The method used to achieve this is codec-stream tokenization, which treats compressed video as a continuous bit-cost stream and allocates a limited token budget to event-bearing content. This approach enables more stable long-video token compression than fixed groups of pictures. The model also incorporates windowed attention for efficient local computation and a shared 3D RoPE to place codec canvases, sampled frames, and images in a unified spatiotemporal coordinate system.

The model was trained using large-scale open supervision, with approximately 8 million re-captioned video samples for pretraining and a 4 million sample spatial corpus for fine-tuning. The paper also introduces JumpScore, a temporal-localization benchmark that targets fine-grained grounding in high-frequency, densely repeated motion. The results show that LLaVA-OneVision-2 outperforms existing models, including Qwen3-VL-8B, by a significant margin. On the JumpScore benchmark, LLaVA-OneVision-2-8B reaches 74.9 JumpScore mAP, surpassing Qwen3-VL-8B by 44.8 points. The model also outperforms Qwen3-VL-8B by 4.3 average points on video tasks, 5.3 on spatial tasks, and 15.6 average J&F on tracking tasks.

The key contributions of the paper are the introduction of codec-stream tokenization, windowed attention, and large-scale open supervision, which enable the model to achieve superior performance across a broad range of multimodal benchmarks. The paper also highlights the importance of unified perception across video understanding, temporal grounding, spatial grounding, and manipulation-trace reasoning. Overall, the paper demonstrates the effectiveness of LLaVA-OneVision-2 in achieving next-generation perceptual intelligence.

📅 Published on May 25

🔗 Links:
• GitHub: https://github.com/huggingface
• arXiv: https://arxiv.org/abs/2605.25979
• PDF: https://arxiv.org/pdf/2605.25979
• Project Page: https://evolvinglmms-lab.github.io/LLaVA-OneVision-2/

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://xn--r1a.website/PaperNexus

#MultimodalLearning #VisionLanguageModels #VideoContentUnderstanding #PerceptualIntelligence #CodecStreamTokenization

🔥 WebWatcher: Breaking New Frontier of Vision-Language Deep Research Agent

💡 The paper introduces WebWatcher, a multimodal agent designed to improve visual-language reasoning in deep research tasks. The problem addressed is that most existing research agents are text-centric and overlook visual information, making multimodal deep research challenging. To solve this, WebWatcher is equipped with enhanced visual-language reasoning capabilities, leveraging synthetic multimodal trajectories for efficient training, utilizing various tools for deep reasoning, and enhancing generalization through reinforcement learning.

The method involves using high-quality synthetic multimodal trajectories for cold start training, which allows the agent to learn from both visual and textual information. The agent is also designed to work with various tools to improve its reasoning abilities. Additionally, the paper proposes a new benchmark called BrowseComp-VL, which is used to evaluate the capabilities of multimodal agents in complex information retrieval tasks involving both visual and textual information.

The results show that WebWatcher significantly outperforms existing baseline agents, including proprietary and open-source agents, in four challenging visual question answering benchmarks. This demonstrates the effectiveness of WebWatcher in solving complex multimodal information-seeking tasks and paves the way for further research in this area. Overall, the paper contributes to the development of multimodal agents with stronger reasoning abilities, which can handle both visual and textual information, and provides a new benchmark for evaluating the performance of such agents.

📅 Published on Aug 7, 2025

🔗 Links:
• GitHub: https://github.com/huggingface
• arXiv: https://arxiv.org/abs/2508.05748
• PDF: https://arxiv.org/pdf/2508.05748
• Project Page: https://tongyi-agent.github.io/blog/introducing-tongyi-deep-research/

🤖 Models citing this paper:
• https://huggingface.co/Alibaba-NLP/WebWatcher-32B
• https://huggingface.co/Alibaba-NLP/WebWatcher-7B

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://xn--r1a.website/PaperNexus

#MultimodalLearning #VisionLanguageReasoning #DeepResearchAgents #SyntheticMultimodalTrajectories #ReinforcementLearningForVision

🔥 M^3Eval: Multi-Modal Memory Evaluation through Cognitively-Grounded Video Tasks

💡 The paper introduces M3Eval, a comprehensive evaluation framework and benchmark for assessing the memory capabilities of multi-modal models in video understanding systems. The problem addressed is that current multi-modal models have significant limitations in their memory capabilities, particularly in maintaining disentangled representations and demonstrating human-like interference patterns. To address this gap, the authors designed M3Eval, which is grounded in cognitive psychology and features carefully constructed tasks that isolate key aspects of memory.

The method involves conducting extensive experiments across representative multi-modal models using the M3Eval framework, which evaluates different memory dimensions such as what models retain, how faithfully information is preserved, and how robust memory remains under interference. The framework includes tasks that test the models' ability to maintain disentangled representations, exhibit human-like interference patterns, and demonstrate symbolic memory.

The results of the experiments reveal consistent weaknesses and distinctive behaviors in the models. The models struggle to maintain disentangled representations when processing parallel video streams, exhibit interference patterns that differ substantially from those observed in human memory, and ground memory sources more reliably in the spatial domain than the temporal domain. Additionally, the models demonstrate limited symbolic memory.

The paper's contributions include providing a valuable resource for future research in the form of the M3Eval benchmark and highlighting memory as a fundamental yet underexplored capability in multi-modal models. The findings offer insights for designing more effective memory mechanisms in multi-modal models, which can advance the field of video understanding systems. The code and dataset are made available to facilitate future research.

📅 Published on Jun 3

🔗 Links:
• GitHub: https://github.com/huggingface
• arXiv: https://arxiv.org/abs/2606.05008
• PDF: https://arxiv.org/pdf/2606.05008
• Project Page: https://pku-value-lab.github.io/m3eval-homepage/

📊 Datasets citing this paper:
• https://huggingface.co/datasets/PKU-VaLuE-Lab/m3eval

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://xn--r1a.website/PaperNexus

#MultimodalLearning #VideoUnderstanding #CognitiveArchitectures #MemoryEvaluation #MultimodalModels

🔥 UniCorn: Towards Self-Improving Unified Multimodal Models through Self-Generated Supervision

💡 The paper introduces UniCorn, a self-improvement framework for unified multimodal models that addresses the generation gap in these models. The generation gap refers to the discrepancy between a model's ability to understand multimodal inputs and its ability to generate high-quality outputs. This gap is formalized as Conduction Aphasia, where models can accurately interpret multimodal inputs but struggle to translate that understanding into faithful and controllable synthesis.

To address this issue, UniCorn proposes a simple yet elegant self-improvement framework that eliminates the need for external data or teacher supervision. The framework partitions a single unified multimodal model into three collaborative roles: Proposer, Solver, and Judge. The Proposer generates initial outputs, the Solver refines these outputs, and the Judge evaluates the quality of the refined outputs. Through self-play and cognitive pattern reconstruction, UniCorn generates high-quality interactions and distills latent understanding into explicit generative signals.

The authors introduce UniCycle, a cycle-consistency benchmark based on a Text to Image to Text reconstruction loop, to validate the restoration of multimodal coherence. The results demonstrate that UniCorn achieves comprehensive and substantial improvements over the base model across six general image generation benchmarks. Notably, it achieves state-of-the-art performance on several benchmarks, including TIIF, DPG, CompBench, and UniCycle, and delivers substantial gains on WISE and OneIG.

The contributions of the paper are significant, as UniCorn enhances text-to-image generation while maintaining robust comprehension, demonstrating the scalability of fully self-supervised refinement for unified multimodal intelligence. The results highlight the effectiveness of the self-improvement framework in addressing the generation gap in unified multimodal models, and the potential of UniCorn to improve the performance of these models in various applications. Overall, the paper presents a novel approach to self-improving unified multimodal models, with significant implications for the development of more advanced and effective multimodal models.

📅 Published on Jan 6

🔗 Links:
• GitHub: https://github.com/huggingface
• arXiv: https://arxiv.org/abs/2601.03193
• PDF: https://arxiv.org/pdf/2601.03193
• Project Page: https://costaliya.github.io/UniCorn.github.io/

🤖 Models citing this paper:
• https://huggingface.co/CostaliyA/UniCorn

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://xn--r1a.website/PaperNexus

#MultimodalLearning #SelfImprovingModels #UnifiedModels #SelfGeneratedSupervision #MultimodalSynthesis

🔥 Robust-U1: Can MLLMs Self-Recover Corrupted Visual Content for Robust Understanding?

💡 The paper proposes a novel framework called Robust-U1 to enhance the robustness of multimodal large language models against visual corruptions. The problem addressed is that existing models perform poorly when faced with real-world visual corruptions such as noise or blur. Current approaches to improve robustness have limitations, either lacking interpretability or being unable to restore lost pixel-level details.

The Robust-U1 framework is designed to equip models with explicit visual self-recovery capability, allowing them to recover corrupted visual content by themselves. The approach consists of three stages: supervised fine-tuning for initial reconstruction, reinforcement learning with dual rewards to align high visual quality, and multimodal reasoning that considers both the corrupted input and the recovered image.

The results show that Robust-U1 achieves state-of-the-art robustness on a real-world corruption benchmark and maintains superior performance under adversarial corruptions on general visual question answering benchmarks. The analysis confirms that high-quality visual recovery directly enhances reasoning performance, establishing self-recovery as a critical mechanism for robust visual understanding. Overall, the paper demonstrates that multimodal large language models can self-recover corrupted visual content, leading to improved robustness and performance in visual understanding tasks.

📅 Published on Jun 6

🔗 Links:
• GitHub: https://github.com/huggingface
• arXiv: https://arxiv.org/abs/2606.08063
• PDF: https://arxiv.org/pdf/2606.08063
• Project Page: https://huggingface.co/spaces/Jiaqi-hkust/Robust-U1

🤖 Models citing this paper:
• https://huggingface.co/Jiaqi-hkust/Robust-U1-SFT
• https://huggingface.co/Jiaqi-hkust/Robust-U1-RL
• https://huggingface.co/Jiaqi-hkust/Robust-U1

🚀 Spaces citing this paper:
• https://huggingface.co/spaces/Jiaqi-hkust/Robust-U1

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://xn--r1a.website/PaperNexus

#MultimodalLearning #VisualContentRecovery #RobustLanguageModels #SelfRecoveryMechanisms #CorruptionResistantAI