💡 The paper presents ProRL Agent, a scalable infrastructure for reinforcement learning training of multi-turn large language model agents. The problem addressed is the difficulty in generating and managing large numbers of sandboxed rollout trajectories required for reinforcement learning, which is a key component for improving the long-horizon behavior of these agents. Existing infrastructures often combine rollout orchestration with the training loop, making systems hard to migrate and maintain.

To solve this problem, the authors propose a rollout-as-a-service approach, where ProRL Agent serves the full agentic rollout lifecycle through an API service. This allows for decoupling rollout orchestration from the training loop, making the system more flexible and easier to maintain. Additionally, ProRL Agent provides standardized and extensible sandbox environments that support diverse agentic tasks in high-performance computing settings.

The authors validate ProRL Agent by applying it to reinforcement learning training on various tasks, including software engineering, math, STEM, and coding. The results demonstrate the effectiveness of ProRL Agent in supporting scalable and efficient reinforcement learning training. Furthermore, ProRL Agent is open-sourced and integrated as part of NVIDIA NeMo Gym, making it accessible to the research community. Overall, the paper contributes a scalable and flexible infrastructure for reinforcement learning training of multi-turn large language model agents, which can facilitate advancements in complex, interactive tasks.

💡 The paper introduces VibeSearchBench, a benchmark for evaluating long-horizon proactive search in real-world scenarios. The problem addressed is the poor performance of large language model-based agents in search tasks that involve multi-turn dialogue and collaborative refinement of user intent. Existing benchmarks rely on over-specified queries, single-turn interactions, and fixed-schema evaluation, which do not reflect real search behavior.

To address this issue, the authors propose VibeSearch, a paradigm that involves multi-turn dialogue and collaborative refinement of vague user intent. The VibeSearchBench benchmark consists of 200 manually curated bilingual tasks across 20 domains, split into professional and daily-life subsets. Each task pairs a user persona with a schema-free ground-truth knowledge graph and is evaluated through a progressive-disclosure user simulator and a graph-matching evaluation framework.

The authors benchmark seven frontier models under two different frameworks and find that all models perform poorly, with the best F1 score being 30.30. This highlights the need for fundamental advances in long-context reasoning, proactive intent elicitation, and structured knowledge construction. The paper's contributions include the introduction of the VibeSearch paradigm, the creation of the VibeSearchBench benchmark, and the evaluation of state-of-the-art models in this new benchmark, which reveals the significant gap between current models and real-world search requirements.