🔥 LiveEdit: Towards Real-Time Diffusion-Based Streaming Video Editing

💡 The paper introduces a novel streaming video editing framework called LiveEdit that enables real-time editing of videos with stable backgrounds and non-edited regions over time. The main challenge in streaming video editing is maintaining stable backgrounds and non-edited regions while achieving low latency for real-time interactive scenarios. Existing streaming video generation methods are not suitable for editing due to the strict preservation requirement and region-specific control.

To address this issue, the authors propose a three-stage distillation pipeline that transfers editing capability from a powerful bidirectional foundation model to an efficient unidirectional streaming editor. This pipeline enables stable long-horizon edits without sacrificing visual fidelity. Additionally, the authors introduce an AR-oriented mask cache that reuses region-related computation across frames, reducing redundant processing and accelerating inference.

The results show that the proposed method achieves state-of-the-art visual quality among streaming baselines and drastically boosts inference speed to 12.66 frames per second. This makes it suitable for interactive and augmented reality applications. The authors also establish a dedicated benchmark for streaming video editing to evaluate their method. Overall, the paper presents a significant contribution to the field of streaming video editing by providing a real-time and stable editing framework.

📅 Published on Jun 25

🔗 Links:
• GitHub: https://github.com/huggingface
• arXiv: https://arxiv.org/abs/2606.26740
• PDF: https://arxiv.org/pdf/2606.26740
• Project Page: https://arxiv.org/abs/2606.26740

🚀 Spaces citing this paper:
• https://huggingface.co/spaces/multimodalart/LiveEdit

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://xn--r1a.website/PaperNexus

#VideoEditingTechniques #RealTimeStreaming #DiffusionBasedEditing #StreamingVideoGeneration #LowLatencyEditing