🔥 RAVEN: Real-time Autoregressive Video Extrapolation with Consistency-model GRPO

💡 The paper introduces RAVEN, a real-time autoregressive video extrapolation network, and CM-GRPO, a consistency model-based reinforcement learning approach. The problem addressed is the gap between the history distributions encountered during training and those arising at inference in causal autoregressive video diffusion models, which constrains generation quality over long horizons.

To solve this problem, RAVEN repacks each self rollout into an interleaved sequence of clean historical endpoints and noisy denoising states, aligning training attention with inference-time extrapolation. This formulation allows downstream chunk losses to supervise the history representations on which future predictions depend.

Additionally, CM-GRPO reformulates a consistency sampling step as a conditional Gaussian transition and applies online reinforcement learning directly to this kernel, avoiding the Euler-Maruyama auxiliary process adopted in prior flow-model RL formulations.

The results demonstrate that RAVEN surpasses recent causal video distillation baselines across quality, semantic, and dynamic degree evaluations. Furthermore, CM-GRPO provides further gains when combined with RAVEN, indicating the effectiveness of the proposed methods in improving real-time video generation.

Overall, the paper presents a novel approach to real-time video generation through causal autoregressive extrapolation with improved training alignment and consistency model-based reinforcement learning, achieving state-of-the-art results in video generation quality and performance.

📅 Published on May 14

🔗 Links:
• GitHub: https://github.com/huggingface
• arXiv: https://arxiv.org/abs/2605.15190
• PDF: https://arxiv.org/pdf/2605.15190
• Project Page: https://yanzuo.lu/raven/

🤖 Models citing this paper:
• https://huggingface.co/mvp-lab/RAVEN

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://xn--r1a.website/PaperNexus

#AutoregressiveVideoExtrapolation #VideoDiffusionModels #ReinforcementLearningForVideo #ConsistencyModelBasedRL #RealTimeVideoGeneration

🔥 Vidu S1: A Real-Time Interactive Video Generation Model

💡 The paper introduces Vidu S1, a real-time interactive video generation model that enables voice-controlled digital character animation with high frame rates on consumer hardware. The model addresses the problem of generating high-quality, real-time video content that can be controlled by users through voice instructions. To achieve this, the authors employ TurboDiffusion and TurboServe, which allow Vidu S1 to produce 540p videos at up to 42 frames per second on regular consumer GPUs. The model supports infinite-length video generation without visual distortion and allows users to upload custom images and choose different voice tones for personalized experiences. The results show that Vidu S1 achieves the best performance across all test metrics while meeting real-time inference requirements, demonstrating its effectiveness in generating high-quality, interactive video content. Overall, the paper presents a significant contribution to the field of video generation, enabling real-time, voice-controlled, and personalized video experiences.

📅 Published on Jul 3

🔗 Links:
• GitHub: https://github.com/huggingface
• arXiv: https://arxiv.org/abs/2607.03118
• PDF: https://arxiv.org/pdf/2607.03118
• Project Page: https://vidu.com/vidu-stream

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://xn--r1a.website/PaperNexus

#RealTimeVideoGeneration #InteractiveVideoModel #VoiceControlledAnimation #DigitalCharacterAnimation #TurboDiffusionTechnology

🔥 JoyAI-Video-Edit: Real-Time Open-Ended Video Editing with Autoregressive Diffusion

💡 The paper presents JoyAI-Video-Edit, a real-time open-ended video editing framework that uses a 16 billion parameter autoregressive diffusion model. The goal is to achieve low-latency causal generation of videos while preserving source fidelity and long-term temporal consistency, all within bounded computational resources. The method combines chunk-wise autoregressive adaptation, Source-Anchored Distribution Matching Distillation, and Long-Horizon Autoregressive Distillation to reduce train-inference mismatch, preserve source fidelity during two-step generation, and mitigate accumulated temporal drift.

The approach is designed to work without access to future frames or a predefined video duration, making it suitable for real-time video editing applications. The authors evaluate their system extensively using both automatic and human evaluations, demonstrating that JoyAI-Video-Edit substantially outperforms existing streaming editors and remains competitive with strong offline systems on both short and long videos.

The complete system achieves end-to-end 720p video editing at approximately 30 frames per second on a single Nvidia B200 GPU. The code for the system is made available, allowing for further research and development in the area of real-time video editing. Overall, the paper contributes a novel approach to real-time video editing, addressing the challenges of low-latency generation, source fidelity, and temporal consistency, and demonstrating the effectiveness of the proposed method through extensive evaluations.

📅 Published on Aug 4

🔗 Links:
• GitHub: https://github.com/huggingface
• arXiv: https://arxiv.org/abs/2608.03974
• PDF: https://arxiv.org/pdf/2608.03974

🤖 Models citing this paper:
• https://huggingface.co/jdopensource/JoyAI-Video-Edit

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://xn--r1a.website/PaperNexus

#AutoregressiveDiffusionModels #VideoEditingFrameworks #RealTimeVideoGeneration #CausalVideoSynthesis #DiffusionBasedVideoEditing