🔥 Achieving Gold-Medal-Level Olympiad Reasoning via Simple and Unified Scaling

💡 The paper presents a systematic approach to transform post-trained reasoning models into rigorous olympiad-level solvers. The problem addressed is achieving gold-medal-level performance on mathematical and physics competitions. The method involves a simple and unified recipe that includes three main components: a reverse-perplexity curriculum, a two-stage reinforcement learning pipeline, and test-time scaling. The reverse-perplexity curriculum is used to instill rigorous proof-search and self-checking behaviors in the model. The two-stage reinforcement learning pipeline progresses from reinforcement learning with verifiable rewards to more delicate proof-level reinforcement learning, allowing the model to scale its behaviors. Finally, test-time scaling is used to boost the solving performance of the model.

The authors applied this recipe to a 30B-A3B backbone with sequence-to-function transformer training on around 340K sub-8K-token trajectories, followed by 200 reinforcement learning steps. The resulting model, SU-01, demonstrates stable reasoning on difficult problems with trajectories exceeding 100K tokens. The results show that the model achieves gold-medal-level performance on mathematical and physical olympiad competitions, including the International Mathematical Olympiad and the International Physics Olympiad. Additionally, the model demonstrates strong generalization of scientific reasoning to domains beyond mathematics and physics. Overall, the paper contributes a simple and unified approach to achieving gold-medal-level olympiad reasoning, with significant implications for advancing long-horizon mathematical and scientific problem solving.

📅 Published on May 13

🔗 Links:
• GitHub: https://github.com/huggingface
• arXiv: https://arxiv.org/abs/2605.13301
• PDF: https://arxiv.org/pdf/2605.13301
• Project Page: https://simplified-reasoning.github.io/SU-01

🤖 Models citing this paper:
• https://huggingface.co/Simplified-Reasoning/SU-01

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://xn--r1a.website/PaperNexus

#OlympiadReasoning #MathematicalCompetitions #PhysicsCompetitions #ReinforcementLearning #ArtificialIntelligence

🔥 GoLongRL: Capability-Oriented Long Context Reinforcement Learning with Multitask Alignment

💡 The paper introduces GoLongRL, a new approach to long context reinforcement learning that focuses on capability oriented data construction and multitask alignment. The existing methods for long context reinforcement learning often result in homogeneous task coverage and reward formulations that do not accurately reflect real world requirements. To address this issue, the authors propose two main contributions.

First, they introduce a capability oriented data construction method that involves creating a dataset of 23,000 reinforcement learning samples with verifiable rewards, spanning 9 task types, each with its own evaluation metric. The dataset is openly released along with the construction pipeline and training code. The results show that this dataset outperforms a closed source dataset called QwenLong-L1.5 under the same training setup.

Second, the authors propose a new method called TMN-Reweight for heterogeneous multitask optimization. This method combines task level mean normalization for cross task reward scale alignment with difficulty adaptive weighting for more reliable advantage estimation. The results show that TMN-Reweight improves average performance over the vanilla GRPO method, while preserving or improving general capabilities across evaluations.

The authors also train a model called Qwen3-30B-A3B on the new dataset and achieve long context performance comparable to other state of the art models, such as DeepSeek-R1-0528 and Qwen3-235B-A22B-Thinking-2507. This suggests that the new dataset and TMN-Reweight method can substantially improve long context capability. Overall, the paper presents a new approach to long context reinforcement learning that focuses on capability oriented data construction and multitask alignment, and achieves state of the art results.

📅 Published on May 19

🔗 Links:
• GitHub: https://github.com/huggingface
• arXiv: https://arxiv.org/abs/2605.19577
• PDF: https://arxiv.org/pdf/2605.19577
• Project Page: https://huggingface.co/collections/Kwai-Klear/golongrl

🤖 Models citing this paper:
• https://huggingface.co/Kwai-Klear/GoLongRL-4B
• https://huggingface.co/Kwai-Klear/GoLongRL-30B-A3B

📊 Datasets citing this paper:
• https://huggingface.co/datasets/Kwai-Klear/GoLongRL

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://xn--r1a.website/PaperNexus

#ReinforcementLearning #LongContextLearning #MultitaskAlignment #CapabilityOrientedLearning #DeepLearning

🔥 CUA-Gym: Scaling Verifiable Training Environments and Tasks for Computer-Use Agents

💡 The paper addresses the problem of training computer-use agents using reinforcement learning with verifiable rewards, which is limited by the scarcity of scalable training data with deterministic rewards. To solve this, the authors propose CUA-Gym, a scalable pipeline that generates task instructions, environment states, and reward functions. The pipeline consists of a generator agent, a discriminator agent, and an orchestrator agent that work together to create high-quality training data. The generated data is then filtered using a combination of large language model majority voting and agent rollouts to ensure quality.

To further address the scarcity of training environments, the authors create CUA-Gym-Hub, a suite of high-fidelity mock web applications that mimic real-world software-use distributions. Using this pipeline, the authors construct a dataset of 32,112 verified training tuples grounded in 110 environments. They then train two models, CUA-Gym-A3B and CUA-Gym-A17B, using the dataset and achieve state-of-the-art performance on the OSWorld-Verified benchmark, with scores of 62.1% and 72.6% respectively.

The results demonstrate that the proposed pipeline and dataset can be used to train computer-use agents that outperform prior models at comparable scales. Additionally, the models show transferability beyond the training environments, as they also improve on the held-out WebArena benchmark. The authors plan to open-source the full synthesis pipeline, dataset, environments, and models, making it possible for others to build upon their work and further advance the field of computer-use agents. Overall, the paper presents a significant contribution to the field of reinforcement learning and computer-use agents, providing a scalable and effective way to train agents that can perform complex tasks in a variety of environments.

📅 Published on May 25

🔗 Links:
• GitHub: https://github.com/huggingface
• arXiv: https://arxiv.org/abs/2605.25624
• PDF: https://arxiv.org/pdf/2605.25624
• Project Page: https://cua-gym.xlang.ai

📊 Datasets citing this paper:
• https://huggingface.co/datasets/xlangai/CUA-Gym

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://xn--r1a.website/PaperNexus

#ComputerUseAgents #VerifiableRewards #ReinforcementLearning #TaskInstructionGeneration #ScalableTrainingEnvironments

🔥 ProRL Agent: Rollout-as-a-Service for RL Training of Multi-Turn LLM Agents

💡 The paper presents ProRL Agent, a scalable infrastructure for reinforcement learning training of multi-turn large language model agents. The problem addressed is the difficulty in generating and managing large numbers of sandboxed rollout trajectories required for reinforcement learning, which is a key component for improving the long-horizon behavior of these agents. Existing infrastructures often combine rollout orchestration with the training loop, making systems hard to migrate and maintain.

To solve this problem, the authors propose a rollout-as-a-service approach, where ProRL Agent serves the full agentic rollout lifecycle through an API service. This allows for decoupling rollout orchestration from the training loop, making the system more flexible and easier to maintain. Additionally, ProRL Agent provides standardized and extensible sandbox environments that support diverse agentic tasks in high-performance computing settings.

The authors validate ProRL Agent by applying it to reinforcement learning training on various tasks, including software engineering, math, STEM, and coding. The results demonstrate the effectiveness of ProRL Agent in supporting scalable and efficient reinforcement learning training. Furthermore, ProRL Agent is open-sourced and integrated as part of NVIDIA NeMo Gym, making it accessible to the research community. Overall, the paper contributes a scalable and flexible infrastructure for reinforcement learning training of multi-turn large language model agents, which can facilitate advancements in complex, interactive tasks.

📅 Published on Mar 19

🔗 Links:
• GitHub: https://github.com/huggingface
• arXiv: https://arxiv.org/abs/2603.18815
• PDF: https://arxiv.org/pdf/2603.18815

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://xn--r1a.website/PaperNexus

#ReinforcementLearning #LargeLanguageModels #MultiTurnDialogue #RolloutOptimization #RLTrainingInfrastructure

🔥 stable-worldmodel-v1: Reproducible World Modeling Research and Evaluation

💡 The paper introduces stable-worldmodel, a modular and standardized research framework for developing and evaluating world models. World models are a powerful tool for learning compact representations of environment dynamics, enabling agents to reason and generalize beyond direct experience. However, current implementations are often publication-specific, which limits their reusability, increases the risk of bugs, and reduces evaluation standardization.

To address this issue, the authors developed stable-worldmodel, a tested and documented research ecosystem that provides efficient data collection tools, standardized environments, planning algorithms, and baseline implementations. The framework allows for controllable environmental factors, including visual and physical properties, to support robustness and continual learning research.

The authors demonstrate the utility of stable-worldmodel by using it to study zero-shot robustness in DINO-WM. The framework provides a standardized way to evaluate world models, which can help to advance research in this area. The main contributions of the paper are the introduction of a modular and standardized research framework for world models, the provision of efficient data collection tools and standardized environments, and the demonstration of the framework's utility in studying zero-shot robustness.

Overall, the paper aims to provide a reliable and reproducible research framework for world modeling, which can help to accelerate progress in this field. The authors' goal is to enable researchers to focus on developing new world models and evaluating their performance, rather than spending time on implementing and debugging existing models. By providing a standardized framework, the authors hope to facilitate the development of more robust and generalizable world models that can be used in a variety of applications.

📅 Published on Feb 9

🔗 Links:
• GitHub: https://github.com/huggingface
• arXiv: https://arxiv.org/abs/2602.08968
• PDF: https://arxiv.org/pdf/2602.08968
• Project Page: https://galilai-group.github.io/stable-worldmodel/

🤖 Models citing this paper:
• https://huggingface.co/zzsi/swm-dmc-cheetah
• https://huggingface.co/zzsi/swm-dmc-expert-policies

📊 Datasets citing this paper:
• https://huggingface.co/datasets/zzsi/swm-dmc-expert
• https://huggingface.co/datasets/zzsi/swm-dmc-mixed-small
• https://huggingface.co/datasets/zzsi/swm-dmc-mixed-large

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://xn--r1a.website/PaperNexus

#WorldModeling #ReinforcementLearning #ArtificialIntelligence #RoboticsResearch #EnvironmentModeling

🔥 MobileGym: A Verifiable and Highly Parallel Simulation Platform for Mobile GUI Agent Research

💡 The paper introduces MobileGym, a browser-based mobile environment designed for mobile GUI agent research. The main problem addressed is the lack of a verifiable and highly parallel simulation platform for training and evaluating mobile GUI agents. Traditional methods are limited by their inability to provide deterministic outcome signals and scalable reinforcement learning.

The authors propose MobileGym as a solution, which enables deterministic evaluation and scalable reinforcement learning through JSON-based state management and parallel execution. The platform captures the full environment state as structured JSON, allowing for easy configuration, forking, and comparison of states. This approach enables a single server to host hundreds of parallel instances, with low memory requirements and fast startup times.

MobileGym features a layered state model and a declarative task-definition framework, making it practical to create and program tasks at scale. The platform also includes a single programmatic judging mechanism that delivers both deterministic evaluation verdicts and dense RL rewards. To facilitate research, the authors provide MobileGym-Bench, a collection of 416 parameterized task templates across 28 apps, including 256 test and 160 train templates.

The results demonstrate the effectiveness of MobileGym in a Sim-to-Real case study, where a model trained in the simulation environment achieves a 12.8 percentage point gain on a 256-task test set. When executed on real devices, the model retains 95.1% of the simulation-side training gain, indicating the potential of MobileGym for real-world applications. Overall, MobileGym provides a verifiable and highly parallel simulation platform for mobile GUI agent research, enabling scalable reinforcement learning and deterministic evaluation.

📅 Published on May 25

🔗 Links:
• GitHub: https://github.com/huggingface
• arXiv: https://arxiv.org/abs/2605.26114
• PDF: https://arxiv.org/pdf/2605.26114
• Project Page: https://mobilegym.github.io

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://xn--r1a.website/PaperNexus

#MobileGUIAgents #ParallelSimulation #ReinforcementLearning #MobileEnvironmentSimulation #GUIAgentResearch

🔥 LongTraceRL: Learning Long-Context Reasoning from Search Agent Trajectories with Rubric Rewards

💡 The paper LongTraceRL addresses the challenge of long-context reasoning in large language models. Long-context reasoning is a central challenge for these models as they often fail to locate and integrate key information in extensive distracting content. Existing methods using reinforcement learning with verifiable rewards have shown promise but are limited by low-confusability distractors and sparse reward signals that cannot supervise intermediate reasoning steps.

To address these issues, the authors introduce LongTraceRL, a method that uses tiered distractor construction and rubric reward design to improve reasoning quality. For data construction, the authors generate multi-hop questions via knowledge graph random walks and leverage search agent trajectories to build tiered distractors. These distractors include documents the agent read but did not cite, which are high in confusability, and documents that appeared in search results but were never opened, which are low in confusability. This approach produces training contexts that are far more challenging than those built by random sampling or one-shot search.

The authors also propose a rubric reward that uses gold entities along each reasoning chain as fine-grained, entity-level process supervision. This reward is applied only to responses with correct final answers, which distinguishes the reasoning quality among correct responses and prevents reward hacking.

The experiments on three reasoning large language models across five long-context benchmarks demonstrate that LongTraceRL consistently outperforms strong baselines and encourages comprehensive, evidence-grounded reasoning. The results show that LongTraceRL is effective in improving the long-context reasoning capabilities of large language models. The codes, datasets, and models are available for further research and development. Overall, LongTraceRL provides a new approach to addressing the challenge of long-context reasoning in large language models and has the potential to improve the performance of these models in a variety of applications.

📅 Published on May 29

🔗 Links:
• GitHub: https://github.com/huggingface
• arXiv: https://arxiv.org/abs/2605.31584
• PDF: https://arxiv.org/pdf/2605.31584

🤖 Models citing this paper:
• https://huggingface.co/THU-KEG/LongTraceRL-4B
• https://huggingface.co/THU-KEG/LongTraceRL-8B
• https://huggingface.co/THU-KEG/LongTraceRL-30B

📊 Datasets citing this paper:
• https://huggingface.co/datasets/THU-KEG/LongTraceRL

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://xn--r1a.website/PaperNexus

#LongContextReasoning #ReinforcementLearning #LargeLanguageModels #RubricRewards #SearchAgentTrajectories

🔥 Harness-1: Reinforcement Learning for Search Agents with State-Externalizing Harnesses

💡 The paper introduces Harness-1, a 20 billion parameter search agent trained with reinforcement learning to improve retrieval performance across multiple domains. The problem with traditional search agents is that they are trained as policies that must decide how to search while also remembering what they have seen, which can lead to inefficient state management. To address this, the authors propose separating semantic decision-making from environmental bookkeeping by using a stateful search harness that maintains environment-side working memory. This harness includes features such as a candidate pool, importance-tagged curated set, compact evidence links, verification records, and budget-aware context rendering. The policy is then responsible for making semantic decisions, such as what to search, which documents to keep or discard, and when to stop. The results show that Harness-1 achieves an average curated recall of 0.730 across eight retrieval benchmarks, outperforming the next strongest open search subagent by 11.4 points. The gains are especially strong on held-out transfer benchmarks, suggesting that the approach can produce retrieval behaviors that generalize beyond the training domains. Overall, the paper demonstrates that using a stateful search harness to separate state management from semantic decision-making can lead to improved retrieval performance and more generalizable search behaviors.

📅 Published on Jun 1

🔗 Links:
• GitHub: https://github.com/huggingface
• arXiv: https://arxiv.org/abs/2606.02373
• PDF: https://arxiv.org/pdf/2606.02373

🤖 Models citing this paper:
• https://huggingface.co/pat-jj/harness-1

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://xn--r1a.website/PaperNexus

#ReinforcementLearning #SearchAgents #StateExternalizing #ReinforcementLearningAlgorithms #DeepLearningForSearch

🔥 SIA: Self Improving AI with Harness & Weight Updates

💡 The paper proposes a self-improving AI framework called SIA that addresses the bottleneck of human involvement in building and improving AI systems. Currently, two separate research approaches exist to tackle this issue: the harness-update school, which updates the task-specific agent's architecture while keeping the model weights fixed, and the test-time training school, which updates the model weights using reinforcement learning pipelines while keeping the harness fixed. However, these two approaches operate in isolation.

The SIA framework combines these two approaches by introducing a language-model feedback agent that simultaneously updates both the model weights and the task-specific agent's architecture. This is achieved through a self-improving loop where the feedback agent provides updates to both the harness and the weights of the task-specific agent.

The authors evaluate the SIA framework across three diverse domains: Chinese legal charge classification, low-level GPU kernel optimization, and single-cell RNA denoising. The results show that combining both harness and weight updates outperforms using only harness updates. The gains are significant, with improvements of 56.6% on the LawBench benchmark, 91.9% runtime reduction on GPU kernels, and 502% improvement on denoising over the initial baseline.

The SIA framework makes the model more agentic by shaping how it searches and acts, while the weight updates build domain-specific intuition that cannot be instilled through prompts or scaffolds alone. Overall, the paper contributes to the development of self-improving AI systems by proposing a novel framework that integrates both harness and weight updates, demonstrating its effectiveness across multiple domains.

📅 Published on May 26

🔗 Links:
• GitHub: https://github.com/huggingface
• arXiv: https://arxiv.org/abs/2605.27276
• PDF: https://arxiv.org/pdf/2605.27276
• Project Page: https://hexolabs.com/

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://xn--r1a.website/PaperNexus

#SelfImprovingAI #HarnessUpdates #WeightUpdates #ReinforcementLearning #TestTimeTraining

🔥 Tmax: A simple recipe for terminal agents

💡 The paper presents a novel approach to training terminal agents using reinforcement learning, called Tmax. Terminal agents are a popular application of language models, but their training has been hindered by the lack of simple and effective methods, limited data, and challenging benchmarks. The authors address these issues by introducing a simplified recipe for training terminal agents, which achieves superior performance with fewer parameters than previous methods.

The method involves generating a large dataset of terminal environments using a novel taxonomy that combines difficulty control, personas, and verifier diversification. This allows for the cheap generation of large amounts of data, which is then used to train open-weight models using reinforcement learning with a simple outcome-only recipe.

The results show that Tmax achieves 27 percent on Terminal-Bench 2.0 with only 9 billion parameters, outperforming much larger models from prior work. The authors also release their terminal dataset, which is over 2.5 times larger than previously released terminal-agent datasets, as well as their models and code as a strong baseline for future academic work on terminal agents.

The contributions of the paper are threefold. First, it presents a simple and effective recipe for training terminal agents, which can be used as a baseline for future work. Second, it introduces a novel taxonomy for generating terminal environments, which allows for the cheap generation of large amounts of data. Third, it releases a large dataset of terminal environments, models, and code, which can be used by other researchers to advance the field of terminal agents. Overall, the paper provides a significant contribution to the field of terminal agents and reinforcement learning, and has the potential to advance the state of the art in this area.

📅 Published on Jun 22

🔗 Links:
• GitHub: https://github.com/huggingface
• arXiv: https://arxiv.org/abs/2606.23321
• PDF: https://arxiv.org/pdf/2606.23321
• Project Page: https://wai-org.com/blog/tmax/

🤖 Models citing this paper:
• https://huggingface.co/allenai/tmax-27b
• https://huggingface.co/allenai/tmax-9b
• https://huggingface.co/allenai/qwen35-9b-openthoughts

📊 Datasets citing this paper:
• https://huggingface.co/datasets/allenai/TMax-15K
• https://huggingface.co/datasets/allenai/tmax-15k-open-instruct
• https://huggingface.co/datasets/allenai/tmax-sft

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://xn--r1a.website/PaperNexus

#TerminalAgents #ReinforcementLearning #LanguageModels #TmaxAlgorithm #AgentTrainingMethods