💡 The paper introduces PixWorld, a unified approach for 3D scene generation and reconstruction in pixel space. The problem with current methods is that they use separate paradigms for reconstruction and generation, with reconstruction using pixel-based regression and generation using latent diffusion. Recent attempts to unify these tasks in latent space have limitations, including information loss and the need for a pretrained autoencoder.

The method presented in the paper reformulates these tasks under a unified pixel-space diffusion paradigm. PixWorld is a single model that jointly addresses 3D reconstruction and generation by supervising diffusion directly on rendered images. This approach removes the limitations of latent space methods and aligns optimization with 3D scene fidelity. The model also introduces a geometry perception loss that aligns rendered views with their ground truth in the geometry-aware feature space of a pretrained 3D foundation model, providing 3D structural supervision.

The results show that PixWorld consistently outperforms prior latent-space generation methods and matches state-of-the-art reconstruction methods. This demonstrates the superiority of a unified pixel-space approach for 3D scene generation and reconstruction. Overall, the paper presents a novel approach that overcomes the limitations of current methods and achieves superior results, making it a significant contribution to the field of 3D scene generation and reconstruction.