💡 The paper introduces TrackCraft3R, a method for dense 3D tracking from monocular video that adapts video diffusion transformers to follow physical points across frames. The problem of dense 3D tracking is fundamental to dynamic scene understanding, but existing methods either lack real-world motion priors or are inefficient. Pre-trained video diffusion transformers offer rich spatio-temporal priors, but their frame-anchored formulation is mismatched with reference-anchored dense 3D tracking.

To address this, TrackCraft3R uses a dual-latent representation that combines per-frame geometry latents and reference-anchored track latents as dense queries. It also employs temporal RoPE alignment, which specifies the target timestamp of each track latent. These designs convert the per-frame generative paradigm of video diffusion transformers into a reference-anchored tracking formulation.

The method achieves state-of-the-art performance on standard sparse and dense 3D tracking benchmarks, while being 1.3 times faster and using 4.6 times less peak memory than the strongest prior method. Additionally, TrackCraft3R demonstrates robustness to large motions and long videos. The key contribution of the paper is the repurposing of video diffusion transformers as a feed-forward dense 3D tracker, enabling efficient and accurate tracking of physical points across frames in a single forward pass.