🔥 RAVEN: Real-time Autoregressive Video Extrapolation with Consistency-model GRPO

💡 The paper introduces RAVEN, a real-time autoregressive video extrapolation network, and CM-GRPO, a consistency model-based reinforcement learning approach. The problem addressed is the gap between the history distributions encountered during training and those arising at inference in causal autoregressive video diffusion models, which constrains generation quality over long horizons.

To solve this problem, RAVEN repacks each self rollout into an interleaved sequence of clean historical endpoints and noisy denoising states, aligning training attention with inference-time extrapolation. This formulation allows downstream chunk losses to supervise the history representations on which future predictions depend.

Additionally, CM-GRPO reformulates a consistency sampling step as a conditional Gaussian transition and applies online reinforcement learning directly to this kernel, avoiding the Euler-Maruyama auxiliary process adopted in prior flow-model RL formulations.

The results demonstrate that RAVEN surpasses recent causal video distillation baselines across quality, semantic, and dynamic degree evaluations. Furthermore, CM-GRPO provides further gains when combined with RAVEN, indicating the effectiveness of the proposed methods in improving real-time video generation.

Overall, the paper presents a novel approach to real-time video generation through causal autoregressive extrapolation with improved training alignment and consistency model-based reinforcement learning, achieving state-of-the-art results in video generation quality and performance.

📅 Published on May 14

🔗 Links:
• GitHub: https://github.com/huggingface
• arXiv: https://arxiv.org/abs/2605.15190
• PDF: https://arxiv.org/pdf/2605.15190
• Project Page: https://yanzuo.lu/raven/

🤖 Models citing this paper:
• https://huggingface.co/mvp-lab/RAVEN

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://xn--r1a.website/PaperNexus

#AutoregressiveVideoExtrapolation #VideoDiffusionModels #ReinforcementLearningForVideo #ConsistencyModelBasedRL #RealTimeVideoGeneration

🔥 OSP-Next: Efficient High-Quality Video Generation with Sparse Sequence Parallelism, HiF8 Quantization, and Reinforcement Learning

💡 The paper introduces OSP-Next, an efficient text-to-video generation model that addresses the high computational costs of existing models. The problem with current models, such as Diffusion Transformers, is that they achieve strong video generation quality but have quadratic costs due to full attention. To solve this, OSP-Next combines sparse attention, parallelism, quantization, and reinforcement learning.

The method used in OSP-Next is a hybrid full-sparse attention architecture, where the sparse component is implemented with Skiparse-2D Attention. This mechanism applies token-wise and group-wise sparse attention along spatial dimensions, leveraging locality while maintaining compatibility with FlashAttention kernels. The authors also propose Sparse Sequence Parallelism, which partitions subsequences across ranks and switches sparse patterns through a single All-to-All communication. This approach reduces communication volume by 75% compared to Ulysses Sequence Parallelism.

Additionally, OSP-Next incorporates HiF8 quantization to enable stable joint training with 8-bit quantization and sparse fine-tuning. The model also applies Mix-GRPO post-training to improve the performance of the sparse model. The authors evaluate OSP-Next on various settings, including 5-second 720P and 5-second 768P, and achieve significant speedups on NVIDIA H200 GPUs and Ascend 950PR hardware.

The results show that OSP-Next achieves a VBench total score of 83.73%, surpassing the Wan2.1 baseline. The model achieves up to 1.64 times single-GPU speedup and over 1.52 times eight-GPU speedup on NVIDIA H200 GPUs. Furthermore, with only a 0.4% drop in VBench total score, OSP-Next-HiF8 achieves 1.69 times and 2.27 times speedups under the two settings on a single Ascend 950PR, demonstrating the efficiency and performance of OSP-Next across hardware platforms. Overall, the paper contributes to the development of efficient text-to-video generation models with high-quality video synthesis and reduced computational costs.

📅 Published on May 27

🔗 Links:
• GitHub: https://github.com/huggingface
• arXiv: https://arxiv.org/abs/2605.28691
• PDF: https://arxiv.org/pdf/2605.28691

🤖 Models citing this paper:
• https://huggingface.co/yunyangge/OSP-Next

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://xn--r1a.website/PaperNexus

#VideoGenerationModels #SparseSequenceParallelism #HiF8Quantization #ReinforcementLearningForVideo #TextToVideoSynthesis