🔥 Pixal3D: Pixel-Aligned 3D Generation from Images

💡 The paper introduces Pixal3D, a new approach to generating 3D models from images that addresses the issue of fidelity, which refers to how accurately the generated 3D model represents the input image. Current 3D generative models often struggle with this due to the implicit correspondence between 2D images and 3D models. Pixal3D solves this problem by generating 3D models in a pixel-aligned way, meaning that each pixel in the input image is directly associated with a corresponding point in the 3D model.

To achieve this, the authors propose a pixel back-projection conditioning scheme that lifts image features into a 3D feature volume, establishing a direct correspondence between pixels and 3D points. This approach allows for high-fidelity 3D asset creation from images and can be scaled up to produce high-quality models. The method also extends to multi-view generation, where feature volumes from multiple views are aggregated to produce a more accurate 3D model.

The results show that Pixal3D substantially improves fidelity and approaches the level of reconstruction-based methods. Additionally, the authors demonstrate that pixel-aligned generation can benefit scene synthesis and propose a modular pipeline for producing high-fidelity, object-separated 3D scenes from images. Overall, Pixal3D provides a new approach to 3D generation that can produce high-fidelity models from single or multi-view images, and has the potential to inspire further research in this area.

📅 Published on May 11

🔗 Links:
• Project Page: https://huggingface.co/papers?q=back-projection%20conditioning
• arXiv: https://arxiv.org/abs/2605.10922
• PDF: https://arxiv.org/pdf/2605.10922
• GitHub: https://github.com/TencentARC/Pixal3D ⭐ 197

🤖 Models citing this paper:
• https://huggingface.co/TencentARC/Pixal3D

🚀 Spaces citing this paper:
• https://huggingface.co/spaces/TencentARC/Pixal3D

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://xn--r1a.website/PaperNexus

#3DModelGeneration #PixelAlignedRendering #ImageTo3D #3DGenerativeModels #DeepLearningForComputerVision