💡 The paper proposes SwiftI2V, an efficient framework for high resolution image to video generation. The problem addressed is that existing methods for generating high resolution videos from images are either computationally expensive or lack fidelity to the input image. High resolution image to video generation aims to synthesize realistic temporal dynamics while preserving fine grained appearance details of the input image, but at 2K resolution, this becomes extremely challenging. Existing solutions suffer from weaknesses such as high memory and latency costs, or hallucinating details and drifting from input specific local structures.

The proposed method, SwiftI2V, addresses these weaknesses by using a two stage design. First, it generates a low resolution motion reference to reduce token costs and ease the modeling burden. Then, it performs a strongly image conditioned 2K synthesis guided by the motion to recover input faithful details with controlled overhead. To make generation more scalable, SwiftI2V introduces Conditional Segment wise Generation, which synthesizes videos segment by segment with a bounded per step token budget. It also adopts bidirectional contextual interaction within each segment to improve cross segment coherence and input fidelity.

The results show that SwiftI2V achieves performance comparable to end to end baselines while reducing total GPU time by 202 times on the VBench I2V benchmark at 2K resolution. This enables practical 2K image to video generation on a single datacenter GPU or consumer GPU, making it a significant contribution to the field of image to video generation. Overall, SwiftI2V provides an efficient and scalable solution for high resolution image to video generation, addressing the efficiency fidelity dilemma and achieving state of the art results.