🔥 DreamX-World 1.0: A General-Purpose Interactive World Model

💡 DreamX-World 1.0 is a general-purpose interactive text-to-video model that generates long-horizon content with camera control and scene persistence. The problem addressed by this model is the need for a controllable and interactive world model that can generate high-quality video content. To solve this problem, the authors introduced several new methods, including a lightweight variant of projective positional encoding called E-PRoPE, which retains projective camera geometry while applying camera-aware attention to spatially reduced tokens.

The authors also converted a bidirectional video generator into a few-step autoregressive world model using causal forcing, DMD-style distillation, and long-rollout training. This training process exposes the model to its own generated history, reducing style and color drift that accumulates across autoregressive chunks. Additionally, the authors introduced Memory-Conditioned Scene Persistence, which retrieves earlier views through camera-geometry-based retrieval, and residual recycling, which makes the conditioning path less sensitive to imperfect memory latents.

The model also includes Event Instruction Tuning, which adds composable event control, and reinforcement learning alignment, which recovers camera control and visual quality after distillation. To improve efficiency, the authors used mixed-precision DiT execution, residual reuse, 75%-pruned VAE decoding, and asynchronous pipeline parallelism, allowing the model to reach up to 16 FPS on eight RTX 5090 GPUs.

The results show that DreamX-World 1.0 achieves a camera-control score of 73.75 and an overall score of 84.76, outperforming HY-WorldPlay 1.5 and LingBot-World in overall score. The model's ability to generate high-quality video content with camera control and scene persistence makes it a significant contribution to the field of interactive world models. Overall, DreamX-World 1.0 is a powerful tool for generating interactive and controllable video content, with potential applications in a variety of fields, including gaming, simulation, and education.

📅 Published on Jun 15

🔗 Links:
• GitHub: https://github.com/huggingface
• Project Page: https://huggingface.co/papers?q=projective%20positional%20encoding
• arXiv: https://arxiv.org/abs/2606.16993
• PDF: https://arxiv.org/pdf/2606.16993

🤖 Models citing this paper:
• https://huggingface.co/GD-ML/DreamX-World-5B

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://xn--r1a.website/PaperNexus

#TextToVideoSynthesis #InteractiveWorldModels #VideoContentGeneration #ScenePersistence #CameraControlMechanisms