💡 The paper introduces EvoPolicyGym, a benchmark for evaluating autonomous policy evolution in interactive environments. The problem addressed is that existing evaluations of autonomous agents often oversimplify the process of policy improvement through feedback, either by reducing it to a final score or confounding it with software engineering progress. To address this, the authors propose a controlled evaluation setting called Autonomous Policy Evolution, where an agent repeatedly edits an executable policy system under a fixed interaction budget.

The method involves instantiating this setting in EvoPolicyGym, a suite of compact interactive reinforcement learning environments that assess how agents iteratively improve explored policies. The authors evaluate the performance of various agents, including GPT-5.5, on the EvoPolicyGym suite.

The results show that GPT-5.5 achieves the strongest aggregate rank score and top-two performance on all 16 environments, demonstrating its effectiveness in autonomous policy evolution. However, the paper goes beyond leaderboard results and provides trajectory-level diagnostics that reveal how agents allocate budget and convert feedback into parametric tuning. These analyses indicate that strong autonomous policy evolution depends on discovering task-appropriate mechanisms and refining policies under bounded feedback, rather than just achieving isolated task wins.

The paper's contributions include the introduction of EvoPolicyGym as a benchmark for evaluating autonomous policy evolution, and the demonstration that successful policy evolution requires both task-specific mechanisms and feedback-constrained refinement. The results provide insights into the importance of iterative improvement and bounded feedback in autonomous policy evolution, and highlight the need for more nuanced evaluations of autonomous agents beyond simple final scores.