💡 The paper LongLive-RAG addresses the challenge of generating long videos using autoregressive video diffusion models. The problem with existing methods is that they use sliding-window attention, which can lead to error accumulation and identity drift over time. This is because once the active window accumulates appearance errors, subsequent generations can only condition on this degraded trajectory and drift further away. To overcome this limitation, the authors propose a retrieval-augmented generation framework called LongLive-RAG.

In this framework, previously generated latents are treated as a dynamic and searchable history. At each new block, LongLive-RAG uses a query embedding to retrieve relevant historical latents, allowing the generator to condition on non-local context instead of only the recent window. This retrieval step adds only a small overhead relative to generation and helps reduce error accumulation.

To make retrieval more discriminative, the authors introduce the Window Temporal Delta Loss, which suppresses redundant local similarity and encourages embeddings to capture meaningful temporal changes. The LongLive-RAG framework is general and can be used with multiple autoregressive backbones and generation lengths.

The experiments show that LongLive-RAG improves long video quality and achieves the best average VBench-Long rank. The authors claim that LongLive-RAG is the first method to formulate self-generated latent history as content-addressable retrieval memory, making it a significant contribution to the field of long video generation. The code for LongLive-RAG is available, making it possible for others to build upon and extend this work. Overall, the paper presents a novel approach to long video generation that addresses the limitations of existing methods and achieves state-of-the-art results.