💡 The paper addresses the challenge of evaluating robotic policies, which is a critical bottleneck in the development of embodied robot foundation models. Unlike large language models, robotic policies require real-world rollouts that are slow and costly, limited by hardware and human supervision. To overcome this, the authors propose the use of world models as surrogate policy evaluators. However, the key properties that make a world model reliable for policy assessment are not well understood.

To study this, the authors introduce WMBench, a benchmark constructed from real-robot teleoperation data and matched policy rollouts covering diverse manipulation tasks. They use WMBench to analyze seven video world models, four action representation schemes, and over 324,000 simulated policy rollouts paired with real robot executions. The analysis is further enriched with large-scale community submissions, curated synthetic trajectories, and a large dataset of training videos.

The experiments reveal three core insights: first, evaluator quality is dominated by long-horizon, action-faithful rollout consistency rather than short-term visual realism; second, pretraining gains come not only from data scale but also from balancing general world knowledge with robot-specific controllability; and third, architectural choices such as action encoding, memory design, and evaluator-focused post-training strongly determine alignment with real-world robot behavior.

Based on these results, the authors derive a practical design roadmap and realize it in GigaWorld-1, a world model specially optimized for policy evaluation. The authors fully release their code, models, datasets, and toolkits to advance scalable evaluation research for embodied foundation models. The contributions of the paper include a systematic study of world models for robotic policy evaluation, the introduction of WMBench as a benchmark, and the development of GigaWorld-1 as a reliable world model for policy assessment. Overall, the paper provides a comprehensive framework for evaluating robotic policies and advancing the development of embodied robot foundation models.