✨HeBA: Heterogeneous Bottleneck Adapters for Robust Vision-Language Models

📝 Summary:
HeBA introduces a heterogeneous bottleneck adapter framework for Vision-Language Models. It uses modality-specific processing like convolutions for images and linear projections for text, combined with a compression bottleneck and active gradient initialization. This design improves few-shot lear...

🔹 Publication Date: Published on Mar 17

🔹 Paper Links:
• arXiv Page: https://arxiv.org/abs/2603.16653
• PDF: https://arxiv.org/pdf/2603.16653
• Project Page: https://huggingface.co/papers?q=dense%20linear%20projections
• Github: https://github.com/Jahid12012021/VLM-HeBA

==================================

For more data science resources:
✓ https://xn--r1a.website/DataScienceT

#VisionLanguageModels #DeepLearning #AIResearch #ModelAdapters #FewShotLearning

🔥 D-OPSD: On-Policy Self-Distillation for Continuously Tuning Step-Distilled Diffusion Models

💡 The paper introduces D-OPSD, a new training approach for diffusion models that enables efficient supervised fine-tuning while preserving few-step inference capabilities. The current landscape of high-performance image generation models is shifting from inefficient multi-step models to efficient few-step models, but these models are challenging to fine-tune using traditional techniques. The problem with traditional fine-tuning methods is that they compromise the model's inherent few-step inference capability.

To address this issue, the authors propose D-OPSD, which leverages on-policy self-distillation with text and multimodal features. The method works by making the model act as both the teacher and the student, where the student is conditioned only on the text feature, and the teacher is conditioned on the multimodal feature of both the text prompt and the target image. The training process minimizes the difference between the predicted distributions over the student's own roll-outs, allowing the model to learn new concepts and styles without sacrificing its original few-step capacity.

The key contribution of D-OPSD is that it enables on-policy learning during supervised fine-tuning, which allows the model to learn from its own trajectory and under its own supervision. This approach enables the model to inherit the in-context capabilities of its encoder, making it possible to fine-tune the model continuously without compromising its few-step inference capability. The results show that D-OPSD enables efficient supervised fine-tuning for diffusion models, making it a promising approach for high-performance image generation models.

📅 Published on May 6

🔗 Links:
• arXiv: https://arxiv.org/abs/2605.05204
• PDF: https://arxiv.org/pdf/2605.05204
• Project Page: https://vvvvvjdy.github.io/d-opsd/
• GitHub: https://github.com/vvvvvjdy/D-OPSD ⭐ 24

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://xn--r1a.website/PaperNexus

#DiffusionModels #SelfDistillation #FewShotLearning #ImageGeneration #MultimodalLearning