🔥 Self-Distilled Agentic Reinforcement Learning

💡 The paper introduces Self Distilled Agentic Reinforcement Learning, a method that improves reinforcement learning for multi turn agent training. The problem with traditional reinforcement learning is that it provides only coarse supervision for long horizon interaction, which can lead to instability in multi turn agents. On Policy Self Distillation is a technique that complements reinforcement learning by providing dense token level guidance from a teacher branch, but it has limitations when applied to multi turn agents, such as compounding instability and negative teacher rejections.

The proposed method, Self Distilled Agentic Reinforcement Learning, addresses these limitations by treating On Policy Self Distillation as a gated auxiliary objective, while keeping reinforcement learning as the primary optimization backbone. It uses a sigmoid gate to selectively strengthen positive token level guidance and mitigate negative teacher rejections. This allows the method to stabilize supervision and improve the performance of multi turn agents.

The results show that Self Distilled Agentic Reinforcement Learning substantially improves over existing methods, such as GRPO, and avoids the instability of naive combinations of GRPO and On Policy Self Distillation. The method consistently outperforms hybrid reinforcement learning and On Policy Self Distillation baselines across different model scales and datasets, including Qwen2.5 and Qwen3 families on ALFWorld, WebShop, and Search-QA. Overall, the paper contributes a new method that improves the performance and stability of multi turn agents in reinforcement learning.

📅 Published on May 14

🔗 Links:
• GitHub: https://github.com/huggingface
• arXiv: https://arxiv.org/abs/2605.15155
• PDF: https://arxiv.org/pdf/2605.15155

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://xn--r1a.website/PaperNexus

#AgenticReinforcementLearning #MultiTurnAgentTraining #OnPolicySelfDistillation #ReinforcementLearningMethods #SelfDistilledLearning

🔥 QUEST: Training Frontier Deep Research Agents with Fully Synthetic Tasks

💡 The paper introduces QUEST, a family of open deep research agents that can perform well across diverse long horizon search tasks. The problem addressed is that existing open agents often generalize poorly across different task types, while frontier systems remain proprietary. To solve this, the authors propose a training recipe that combines mid training, supervised fine tuning, and reinforcement learning. A key component of this recipe is a curated data synthesis pipeline that applies to different task types and enables synthesizing training data with verifiable rewards without human annotation. The pipeline uses unified rubric trees to generate tasks. The authors also incorporate a built in context management mechanism that enables effective long horizon reasoning and knowledge synthesis. The results show that QUEST approaches or surpasses frontier closed source agents across eight deep research benchmarks using only 8K synthesized tasks. The models, data, and training scripts are released, making it possible for others to use and build upon the work. The contributions of the paper are the proposed training recipe, the data synthesis pipeline, and the release of the QUEST models, which provide a general purpose deep research agent that can handle a wide range of tasks, including fact seeking, citation grounding, and report synthesis.

📅 Published on May 22

🔗 Links:
• GitHub: https://github.com/huggingface
• arXiv: https://arxiv.org/abs/2605.24218
• PDF: https://arxiv.org/pdf/2605.24218
• Project Page: https://osu-nlp-group.github.io/QUEST/

🤖 Models citing this paper:
• https://huggingface.co/osunlp/QUEST-35B-RL
• https://huggingface.co/osunlp/QUEST-35B-MT-Plus-SFT
• https://huggingface.co/osunlp/QUEST-9B

📊 Datasets citing this paper:
• https://huggingface.co/datasets/osunlp/QUEST-RL-Data
• https://huggingface.co/datasets/osunlp/QUEST-SFT-Data-Objective
• https://huggingface.co/datasets/osunlp/QUEST-SFT-Data-Open-ended

🚀 Spaces citing this paper:
• https://huggingface.co/spaces/osunlp/QUEST

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://xn--r1a.website/PaperNexus

#DeepResearchAgents #LongHorizonSearchTasks #SyntheticTaskGeneration #ReinforcementLearningMethods #OpenAgentArchitectures