💡 The paper introduces OpenSearch-VL, an open-source framework for training advanced multimodal search agents using reinforcement learning. The problem addressed is that current top-tier multimodal search agents are difficult to reproduce due to the lack of open high-quality training data, transparent trajectory synthesis pipelines, and detailed training recipes. To solve this, the authors propose a fully open-source recipe for training frontier multimodal deep search agents.

The method involves curating high-quality training data through a dedicated pipeline that includes Wikipedia path sampling, fuzzy entity rewriting, and source-anchor visual grounding. This pipeline is used to create two training datasets, SearchVL-SFT-36k and SearchVL-RL-8k. The authors also design a diverse tool environment that combines text search, image search, and other tools to enable agents to acquire external knowledge.

A new training algorithm, multi-turn fatal-aware GRPO, is proposed to handle cascading tool failures by masking post-failure tokens while preserving useful pre-failure reasoning. The results show that OpenSearch-VL delivers substantial performance gains, with over 10-point average improvements across seven benchmarks, and achieves results comparable to proprietary commercial models on several tasks. The authors will release all data, code, and models to support open research on multimodal deep search agents.

Overall, the paper contributes to the development of multimodal search agents by providing an open-source framework, high-quality training data, and a novel training algorithm, making it easier for researchers to reproduce and improve upon the results.