🔥 CUA-Gym: Scaling Verifiable Training Environments and Tasks for Computer-Use Agents

💡 The paper addresses the problem of training computer-use agents using reinforcement learning with verifiable rewards, which is limited by the scarcity of scalable training data with deterministic rewards. To solve this, the authors propose CUA-Gym, a scalable pipeline that generates task instructions, environment states, and reward functions. The pipeline consists of a generator agent, a discriminator agent, and an orchestrator agent that work together to create high-quality training data. The generated data is then filtered using a combination of large language model majority voting and agent rollouts to ensure quality.

To further address the scarcity of training environments, the authors create CUA-Gym-Hub, a suite of high-fidelity mock web applications that mimic real-world software-use distributions. Using this pipeline, the authors construct a dataset of 32,112 verified training tuples grounded in 110 environments. They then train two models, CUA-Gym-A3B and CUA-Gym-A17B, using the dataset and achieve state-of-the-art performance on the OSWorld-Verified benchmark, with scores of 62.1% and 72.6% respectively.

The results demonstrate that the proposed pipeline and dataset can be used to train computer-use agents that outperform prior models at comparable scales. Additionally, the models show transferability beyond the training environments, as they also improve on the held-out WebArena benchmark. The authors plan to open-source the full synthesis pipeline, dataset, environments, and models, making it possible for others to build upon their work and further advance the field of computer-use agents. Overall, the paper presents a significant contribution to the field of reinforcement learning and computer-use agents, providing a scalable and effective way to train agents that can perform complex tasks in a variety of environments.

📅 Published on May 25

🔗 Links:
• GitHub: https://github.com/huggingface
• arXiv: https://arxiv.org/abs/2605.25624
• PDF: https://arxiv.org/pdf/2605.25624
• Project Page: https://cua-gym.xlang.ai

📊 Datasets citing this paper:
• https://huggingface.co/datasets/xlangai/CUA-Gym

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://xn--r1a.website/PaperNexus

#ComputerUseAgents #VerifiableRewards #ReinforcementLearning #TaskInstructionGeneration #ScalableTrainingEnvironments