💡 The paper addresses the issue of hallucination in world models, which occurs when the model generates unrealistic futures despite appearing visually fluent. The authors hypothesize that hallucination happens in low-data regions of the state-action space and can be detected and mitigated using data-centric signals and coverage-aware sampling techniques.

To test this hypothesis, the authors created a large dataset called MMBench2, consisting of 427 hours of data and 210 tasks for visual world modeling, with ground-truth actions and rewards. They trained a 350M-parameter world model on this dataset and identified three distinct modes of hallucination: perceptual, action-marginalized, and scene-diverging.

The authors developed three signals that can accurately predict where the model will fail and used these signals to develop a coverage-aware sampling technique to close coverage gaps during training. They also used the hallucination predictors as curiosity rewards for targeted data collection to adapt the pretrained world model to new environments with as few as 50 real environment trajectories.

The results show that hallucination in world models is indeed a data coverage issue and that the same signals used to detect it can also be used for mitigation. The authors provide a data-efficient finetuning recipe that can adapt the pretrained world model to entirely unseen environments, demonstrating the effectiveness of their approach. Overall, the paper contributes to a better understanding of hallucination in world models and provides a practical solution to prevent and mitigate it.