🔥 Self Gradient Forcing: Native Long Video Extrapolation

💡 The paper proposes a new method called Self Gradient Forcing for native long video extrapolation. Recent autoregressive video diffusion methods are built upon Self Forcing, where the student is trained on histories produced by its own rollout rather than ground-truth video contexts. However, this approach has a limitation known as the historical context-gradient gap, where future losses cannot supervise how earlier generated latents should be written into more useful keys and values for later video-latent generation.

To address this issue, the authors propose a two-pass training strategy called Self Gradient Forcing. The first pass performs a no-gradient autoregressive rollout matching inference and records both the self-generated context and the noisy latents fed to the model at a sampled denoising exit step. The second pass performs parallel context-gradient reconstruction for the recorded exit step. The generated context is used as a stop-gradient clean-latent input, while the model recomputes the context KV representations and future-to-context causal attention.

The proposed method provides the missing memory-writing supervision within the native autoregressive training objective, using losses on future video latents to train the model to encode context into more effective causal memory. The authors evaluate their method across extensive long-horizon frame-wise and chunk-wise experiments under different initializations and achieve stronger native long-video extrapolation than Self Forcing, especially in subject identity, background/layout consistency, and temporal stability. Notably, using only a 5-second training window, Self Gradient Forcing can extrapolate to videos lasting several minutes.

📅 Published on Jul 22

🔗 Links:
• GitHub: https://github.com/huggingface
• arXiv: https://arxiv.org/abs/2607.20368
• PDF: https://arxiv.org/pdf/2607.20368
• Project Page: https://zhuang2002.github.io/SelfGradientForcing/

🤖 Models citing this paper:
• https://huggingface.co/JunhaoZhuang/Self_Gradient_Forcing

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://xn--r1a.website/PaperNexus

#VideoExtrapolation #AutoregressiveVideoDiffusion #SelfGradientForcing #LongVideoGeneration #VideoDiffusionMethods