✨Out of Sight but Not Out of Mind: Hybrid Memory for Dynamic Video World Models

📝 Summary:
Hybrid Memory improves video world models by consistently tracking dynamic subjects during occlusion. It combines static background archiving with active dynamic subject tracking. This ensures motion continuity and outperforms existing methods in generation quality.

🔹 Publication Date: Published on Mar 26

🔹 Paper Links:
• arXiv Page: https://arxiv.org/abs/2603.25716
• PDF: https://arxiv.org/pdf/2603.25716
• Project Page: https://kj-chen666.github.io/Hybrid-Memory-in-Video-World-Models/
• Github: https://github.com/H-EmbodVis/HyDRA

🔹 Models citing this paper:
• https://huggingface.co/H-EmbodVis/HyDRA

==================================

For more data science resources:
✓ https://xn--r1a.website/DataScienceT

#VideoWorldModels #ComputerVision #AI #MachineLearning #GenerativeAI

🔥 minWM: A Full-Stack Open-Source Framework for Real-Time Interactive Video World Models

💡 The paper presents a comprehensive framework called minWM for converting bidirectional video diffusion models into real-time interactive video world models. The problem addressed is that recent video diffusion foundation models have achieved high-quality video generation but turning them into real-time interactive world models remains challenging due to the need for controllable, causal, and low-latency capabilities.

The method used in minWM is a full-stack open-source framework that provides an end-to-end pipeline to convert existing bidirectional video foundation models into camera-controllable few-step autoregressive world models. This is achieved through fine-tuning and distillation techniques, including causal forcing, causal consistency distillation, and asymmetric DMD. The framework is modular and architecture-extensible, allowing it to be instantiated on different open backbones and adapted to new data distributions, training recipes, and latency targets.

The results of minWM are a real-time interactive video world model that can be controlled by a camera, with low-latency rollout and high-quality video generation. The framework is released with runnable scripts, checkpoints, documentation, and inference code, along with practical ablations on camera trajectory quality, controllability training steps, and minimal batch-size requirements. Overall, minWM provides a reproducible and extensible recipe for building and adapting real-time interactive video world models, making it a valuable contribution to the field of video generation and interactive world modeling.

📅 Published on May 28

🔗 Links:
• GitHub: https://github.com/huggingface
• arXiv: https://arxiv.org/abs/2605.30263
• PDF: https://arxiv.org/pdf/2605.30263

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://xn--r1a.website/PaperNexus

#VideoDiffusionModels #RealTimeInteractiveSystems #VideoWorldModels #BidirectionalVideoGeneration #InteractiveVideoFrameworks

🔥 Latent Spatial Memory for Video World Models

💡 The paper proposes a novel approach to video world models called latent spatial memory, which stores 3D scene information directly in diffusion latent space. This approach eliminates the need for explicit point cloud memory constructed in RGB space, which is computationally expensive and inherently lossy due to the round trip through pixel space. The authors introduce a framework called Mirage, which constructs the latent spatial memory by lifting latent tokens into 3D via depth-guided back-projection and queries it by synthesizing novel views through direct latent-space warping. This approach avoids pixel-space reconstruction and reduces the computational burden of repeated encoding and rendering. The results show that latent spatial memory achieves significant improvements in video generation speed and memory footprint, with up to 10.57 times faster end-to-end video generation and 55 times reduction in memory footprint compared to explicit 3D baselines. The Mirage framework also attains state-of-the-art performance on WorldScore and strong reconstruction quality on RealEstate10K, demonstrating the effectiveness of the proposed approach. Overall, the paper contributes a new and efficient method for video world models that leverages the geometric prior of the diffusion model to achieve faster and more memory-efficient video generation.

📅 Published on Jun 8

🔗 Links:
• GitHub: https://github.com/huggingface
• Project Page: https://huggingface.co/papers?q=depth-guided%20back-projection
• arXiv: https://arxiv.org/abs/2606.09828
• PDF: https://arxiv.org/pdf/2606.09828

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://xn--r1a.website/PaperNexus

#LatentSpatialMemory #VideoWorldModels #DiffusionLatentSpace #3DSceneUnderstanding #LatentSpaceWarping