💡 The paper proposes a parallelized autoregressive framework for dense video captioning, which aims to generate temporally grounded descriptions of video events. The existing autoregressive models have limitations in terms of inference efficiency and scalability due to the token-by-token generation paradigm. To address this issue, the authors propose a framework that exploits weak local dependencies across temporally distinct events to enable parallel generation of tokens. The key insight is to restructure the causal dependency graph to allow for lossless parallel decoding, where tokens with weak cross-event dependencies can be decoded in parallel, while tightly coupled tokens within each event retain sequential decoding.

The authors introduce two key components to realize this insight: a latent global planning mechanism and an event-factorized parallel decoding mechanism. The latent global planning mechanism learns the event-level structure and produces compact tokens encoding global inter-event causality, while adaptively aggregating event-level audio-visual semantics. The event-factorized parallel decoding mechanism balances local focus with global inter-event awareness, enabling effective parallel decoding.

The proposed framework improves generation efficiency and enhances temporally grounded captioning performance. The experiments on various benchmarks demonstrate the clear advantage of the approach in both efficiency and performance in omni-modal event grounding and captioning. The results show that the parallelized autoregressive framework can generate dense captions more efficiently and accurately, making it a promising solution for large-scale video captioning tasks. Overall, the paper contributes to the development of more efficient and effective dense video captioning models, which can benefit both event-level video understanding and generation.