✨Quartet II: Accurate LLM Pre-Training in NVFP4 by Improved Unbiased Gradient Estimation

📝 Summary:
Quartet II improves LLM pre-training in NVFP4 by introducing MS-EDEN for enhanced unbiased gradient estimation, significantly reducing quantization error. This achieves better accuracy and up to 4.2x faster execution on NVIDIA Blackwell GPUs compared to BF16.

🔹 Publication Date: Published on Jan 30

🔹 Paper Links:
• arXiv Page: https://arxiv.org/abs/2601.22813
• PDF: https://arxiv.org/pdf/2601.22813
• Github: https://github.com/IST-DASLab/Quartet-II

==================================

For more data science resources:
✓ https://xn--r1a.website/DataScienceT

#LLM #DeepLearning #Quantization #GPUAcceleration #AIResearch

🔥 TideGS: Scalable Training of Over One Billion 3D Gaussian Splatting Primitives via Out-of-Core Optimization

💡 The paper introduces TideGS, a scalable training framework for 3D Gaussian Splatting with over one billion primitives on a single GPU. The problem with training 3D Gaussian Splatting at a large scale is that it is memory-bound, with each Gaussian primitive having a large attribute vector that quickly exceeds GPU capacity. Prior systems were limited to tens of millions of Gaussians on commodity single-GPU hardware.

The authors observe that 3D Gaussian Splatting training is inherently sparse and trajectory-conditioned, meaning that each iteration only activates the Gaussians visible from the current camera batch. This insight allows the authors to manage parameters across an SSD-CPU-GPU hierarchy using three techniques: block-virtualized geometry for spatial locality, a hierarchical asynchronous pipeline to overlap I/O with computation, and trajectory-adaptive differential streaming that transfers only incremental working-set deltas between iterations.

The TideGS framework enables training with over one billion Gaussians on a single 24 GB GPU, achieving the best reconstruction quality among evaluated single-GPU baselines on large-scale scenes. This is a significant improvement over prior out-of-core baselines, which were limited to approximately 100 million Gaussians, and standard in-memory training, which was limited to approximately 11 million Gaussians. The results demonstrate that TideGS can scale beyond prior systems, making it a promising solution for large-scale 3D Gaussian Splatting applications.

📅 Published on May 19

🔗 Links:
• GitHub: https://github.com/huggingface
• arXiv: https://arxiv.org/abs/2605.20150
• PDF: https://arxiv.org/pdf/2605.20150
• Project Page: https://sponge-lab.github.io/TideGS/

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://xn--r1a.website/PaperNexus

#3DGaussianSplatting #ScalableDeepLearning #OutofCoreOptimization #GPUAcceleration #ComputerVisionTechniques