🔥 PlayWorld: Benchmarking World Models with Agent Players over Long-Horizon Objectives

💡 The paper introduces PlayWorld, a benchmark for evaluating interactive video world models. The problem addressed is that current methods for comparing these models are inadequate, as they typically rely on fixed action-conditioned evaluation, which is not suitable for cross-model comparison. To address this, the authors propose using multi-modal agents, called Agent Players, to interact with world models towards long-horizon objectives, such as turning around 360 degrees to check environment consistency or walking into water to see realistic water ripples.

The method involves employing these Agent Players to pursue specified objectives in 171 scenarios, and evaluating the models along four core dimensions: geometry consistency, interaction fidelity, out-of-sight evolution, and insight evolution. Additionally, basic ability metrics for video quality and controllability are assessed.

The results of the experiments, which were conducted on nine state-of-the-art world models, show that current models are unreliable on long-horizon interactive objectives, particularly in maintaining spatial consistency and persistent state evolution. The paper contributes a new benchmark, PlayWorld, which provides a comprehensive evaluation framework for interactive video world models, and highlights the need for further research in this area. The code and data are made available for further use.

📅 Published on Aug 13

🔗 Links:
• GitHub: https://github.com/huggingface
• arXiv: https://arxiv.org/abs/2608.13552
• PDF: https://arxiv.org/pdf/2608.13552
• Project Page: https://kxding.github.io/project/PlayWorld/

📊 Datasets citing this paper:
• https://huggingface.co/datasets/jocelynd/playworld-bench

🚀 Spaces citing this paper:
• https://huggingface.co/spaces/jocelynd/PlayWorld-Leaderboard

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://xn--r1a.website/PaperNexus

#WorldModels #LongHorizonObjectives #AgentPlayers #InteractiveVideoModels #BenchmarkingWorldModels