🔥 Lens: Rethinking Training Efficiency for Foundational Text-to-Image Models

💡 The paper introduces Lens, a compact 3.8 billion parameter text-to-image model that achieves superior performance with reduced training compute. The problem addressed is the high computational cost of training large text-to-image models, which can be a significant barrier to their adoption. To address this, the authors propose two key strategies. First, they maximize data information density per training batch by using a dataset of 800 million densely captioned image-text pairs, where each caption contains approximately 109 words on average, providing richer semantic supervision than conventional short captions. They also construct each batch from images with multiple resolutions and diverse aspect ratios, enlarging the effective visual coverage of each optimization step.

Second, they improve convergence speed through careful architectural choices, including adopting a semantic VAE that provides better latent representations and employing a strong language encoder that accelerates optimization while enabling multilingual generalization from English-only training data. The authors also apply reinforcement learning with taxonomy-driven prompts and structured reward rubrics to suppress artifacts and improve visual quality, and use a reasoner module with training-free system prompt search to better align user requests with the model.

The results show that Lens achieves performance competitive with, and in several cases surpassing, state-of-the-art models with more than 6 billion parameters, while requiring significantly less training compute. For example, Lens requires only about 19.3% of the training compute used by Z-Image. The model generalizes to arbitrary aspect ratios and resolutions up to 1440^2, and supports prompts in several commonly used languages. Thanks to its compact size, Lens generates a 1024^2 image in 3.15 seconds on a single NVIDIA H100 GPU, while its distilled turbo version performs 4-step generation in 0.84 seconds. Overall, the paper demonstrates that Lens is a highly efficient and effective text-to-image model that can be trained with significantly less computational resources than existing models.

📅 Published on May 20

🔗 Links:
• GitHub: https://github.com/huggingface
• arXiv: https://arxiv.org/abs/2605.21573
• PDF: https://arxiv.org/pdf/2605.21573
• Project Page: https://huggingface.co/microsoft/Lens

🤖 Models citing this paper:
• https://huggingface.co/microsoft/Lens-Turbo
• https://huggingface.co/microsoft/Lens
• https://huggingface.co/microsoft/Lens-Base

🚀 Spaces citing this paper:
• https://huggingface.co/spaces/multimodalart/lens

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://xn--r1a.website/PaperNexus

#TextToImageModels #EfficientTrainingMethods #CompactNeuralNetworks #ImageTextPairs #FoundationalModeling

🔥 SmolVLA: A Vision-Language-Action Model for Affordable and Efficient Robotics

💡 The paper introduces SmolVLA, a compact and efficient vision-language-action model designed for affordable and efficient robotics. The problem with existing vision-language-action models is that they are typically very large, with billions of parameters, which leads to high training costs and limited deployability on consumer-grade hardware. These models also rely on academic and industrial datasets, overlooking community-collected data from affordable robotic platforms.

To address this issue, the authors developed SmolVLA, a small and efficient model that can be trained on a single GPU and deployed on consumer-grade GPUs or even CPUs. SmolVLA is designed to be community-driven, using community-collected data from affordable robotic platforms, which reduces both training and inference costs while retaining competitive performance.

The method used to achieve this involves adapting vision-language models into vision-language-action models, but with a much smaller size. The authors also introduced an asynchronous inference stack that decouples perception and action prediction from action execution, allowing for higher control rates with chunked action generation.

The results show that SmolVLA achieves performance comparable to vision-language-action models that are 10 times larger, despite its compact size. The model is evaluated on a range of simulated and real-world robotic benchmarks, and the authors release all code, pretrained models, and training data. Overall, SmolVLA provides a more efficient and affordable solution for robotics, making it possible to deploy vision-language-action models on consumer-grade hardware.

📅 Published on Jun 2, 2025

🔗 Links:
• GitHub: https://github.com/huggingface
• arXiv: https://arxiv.org/abs/2506.01844
• PDF: https://arxiv.org/pdf/2506.01844
• Project Page: https://huggingface.co/blog/smolvla

🤖 Models citing this paper:
• https://huggingface.co/lerobot/smolvla_base
• https://huggingface.co/HuggingFaceVLA/smolvla_libero
• https://huggingface.co/jadechoghari/smolvla_metaworld

📊 Datasets citing this paper:
• https://huggingface.co/datasets/lerobot/community_dataset_v3
• https://huggingface.co/datasets/HuggingFaceVLA/community_dataset_v1
• https://huggingface.co/datasets/HuggingFaceVLA/community_dataset_v2

🚀 Spaces citing this paper:
• https://huggingface.co/spaces/HuggingFaceVLA/libero-vla-leaderboard
• https://huggingface.co/spaces/lerobot/robot-learning-tutorial
• https://huggingface.co/spaces/arpitg1304/lerobot_scripts_simplified

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://xn--r1a.website/PaperNexus

#VisionLanguageAction #EfficientRobotics #CompactNeuralNetworks #RoboticsForAll #AffordableAI