💡 The paper introduces PhiZero, a physical world model that uses a compact discrete representation of world-state transitions, referred to as physical language. Existing physical world models typically predict future videos directly in pixel space, which leaves the underlying world dynamics implicit within high-dimensional visual predictors. In contrast, PhiZero is motivated by humans' ability to abstract predictive structure from visual experience and organize it in natural language for explicit reasoning.

The method used in PhiZero involves learning physical language from in-the-wild videos through self-supervision and using it to explicitly reason about how the physical world evolves. PhiZero adopts a reason-then-render paradigm, where it first infers future world evolution as a physical-language sequence and then renders the inferred transitions into videos.

The results of extensive experiments across generation and understanding benchmarks validate the ability of PhiZero to model physically coherent world evolution. The paper also shows the potential of PhiZero for realistic and interactive world modeling, fine-grained action-conditioned simulation, and zero-shot motion transfer. Overall, PhiZero provides a new approach to physical world modeling that is based on a compact and discrete representation of world-state transitions, and has the potential to enable more efficient and effective modeling of complex physical systems.