✨Image-Free Timestep Distillation via Continuous-Time Consistency with Trajectory-Sampled Pairs

📝 Summary:
TBCM is a self-contained method that distills diffusion models by extracting latent representations directly from the teacher model trajectory. This eliminates external data, greatly improving efficiency and quality for few-step generation with reduced resources.

🔹 Publication Date: Published on Nov 25

🔹 Paper Links:
• arXiv Page: https://arxiv.org/abs/2511.20410
• PDF: https://arxiv.org/pdf/2511.20410
• Github: https://github.com/hustvl/TBCM

==================================

For more data science resources:
✓ https://xn--r1a.website/DataScienceT

#DiffusionModels #ModelDistillation #GenerativeAI #AIResearch #MachineLearning

✨Decoupled DMD: CFG Augmentation as the Spear, Distribution Matching as the Shield

📝 Summary:
This study challenges the understanding of Distribution Matching Distillation DMD for text-to-image generation. It reveals that CFG Augmentation is the primary driver of few-step distillation, while distribution matching acts as a regularizer. This new insight enables improved distillation method...

🔹 Publication Date: Published on Nov 27

🔹 Paper Links:
• arXiv Page: https://arxiv.org/abs/2511.22677
• PDF: https://arxiv.org/pdf/2511.22677
• Project Page: https://tongyi-mai.github.io/Z-Image-blog/
• Github: https://github.com/Tongyi-MAI/Z-Image/tree/main

==================================

For more data science resources:
✓ https://xn--r1a.website/DataScienceT

#TextToImage #GenerativeAI #DiffusionModels #ModelDistillation #AIResearch

✨Flash-DMD: Towards High-Fidelity Few-Step Image Generation with Efficient Distillation and Joint Reinforcement Learning

📝 Summary:
Flash-DMD accelerates generative diffusion models via efficient timestep-aware distillation and joint reinforcement learning. This framework achieves faster convergence, high-fidelity few-step generation, and stabilizes RL training using distillation as a regularizer, all with reduced computation...

🔹 Publication Date: Published on Nov 25

🔹 Paper Links:
• arXiv Page: https://arxiv.org/abs/2511.20549
• PDF: https://arxiv.org/pdf/2511.20549

==================================

For more data science resources:
✓ https://xn--r1a.website/DataScienceT

#DiffusionModels #ImageGeneration #ReinforcementLearning #ModelDistillation #GenerativeAI

✨Zooming without Zooming: Region-to-Image Distillation for Fine-Grained Multimodal Perception

📝 Summary:
MLLMs struggle with fine-grained perception due to latency from iterative zooming. Region-to-Image Distillation internalizes zooming into a single forward pass by training a model on region-grounded data. This significantly improves fine-grained perception without tool calls, achieving leading pe...

🔹 Publication Date: Published on Feb 12

🔹 Paper Links:
• arXiv Page: https://arxiv.org/abs/2602.11858
• PDF: https://arxiv.org/pdf/2602.11858
• Github: https://github.com/inclusionAI/Zooming-without-Zooming

🔹 Models citing this paper:
• https://huggingface.co/inclusionAI/ZwZ-8B
• https://huggingface.co/inclusionAI/ZwZ-4B
• https://huggingface.co/inclusionAI/ZwZ-7B

✨ Datasets citing this paper:
• https://huggingface.co/datasets/inclusionAI/ZwZ-RL-VQA
• https://huggingface.co/datasets/inclusionAI/ZoomBench

==================================

For more data science resources:
✓ https://xn--r1a.website/DataScienceT

#MultimodalAI #ComputerVision #FineGrainedPerception #DeepLearning #ModelDistillation

✨jina-embeddings-v5-text: Task-Targeted Embedding Distillation

📝 Summary:
This paper introduces a novel training regimen for compact text embedding models. It combines distillation with task-specific contrastive loss to achieve state-of-the-art performance for small models. The resulting jina-embeddings-v5-text models support long contexts and robust quantization.

🔹 Publication Date: Published on Feb 17

🔹 Paper Links:
• arXiv Page: https://arxiv.org/abs/2602.15547
• PDF: https://arxiv.org/pdf/2602.15547

🔹 Models citing this paper:
• https://huggingface.co/jinaai/jina-embeddings-v5-text-small
• https://huggingface.co/jinaai/jina-embeddings-v5-text-nano

==================================

For more data science resources:
✓ https://xn--r1a.website/DataScienceT

#TextEmbeddings #MachineLearning #NLP #ModelDistillation #DeepLearning

✨NanoVDR: Distilling a 2B Vision-Language Retriever into a 70M Text-Only Encoder for Visual Document Retrieval

📝 Summary:
NanoVDR improves visual document retrieval by distilling a large VLM teacher into a small 70M text-only query encoder. This decouples document indexing from query processing, achieving 50x lower latency and 32x fewer parameters with nearly identical quality.

🔹 Publication Date: Published on Mar 13

🔹 Paper Links:
• arXiv Page: https://arxiv.org/abs/2603.12824
• PDF: https://arxiv.org/pdf/2603.12824
• Project Page: https://huggingface.co/nanovdr

🔹 Models citing this paper:
• https://huggingface.co/nanovdr/NanoVDR-L
• https://huggingface.co/nanovdr/NanoVDR-S-Multi
• https://huggingface.co/nanovdr/NanoVDR-S

✨ Spaces citing this paper:
• https://huggingface.co/spaces/nanovdr/NanoVDR-Demo

==================================

For more data science resources:
✓ https://xn--r1a.website/DataScienceT

#VisualDocumentRetrieval #ModelDistillation #VLM #InformationRetrieval #DeepLearning

✨Effective Distillation to Hybrid xLSTM Architectures

📝 Summary:
Distilling quadratic LLMs to sub-quadratic models typically loses performance. We introduce an xLSTM distillation pipeline with an expert merging stage, enabling students to recover or exceed teacher performance for efficient LLMs.

🔹 Publication Date: Published on Mar 16

🔹 Paper Links:
• arXiv Page: https://arxiv.org/abs/2603.15590
• PDF: https://arxiv.org/pdf/2603.15590

==================================

For more data science resources:
✓ https://xn--r1a.website/DataScienceT

#LLMs #xLSTM #ModelDistillation #AIResearch #EfficientAI

✨A Survey of On-Policy Distillation for Large Language Models

📝 Summary:
On-Policy Distillation OPD lets LLMs learn from self-generated outputs and teacher feedback, addressing off-policy exposure bias. This survey unifies OPD with an f-divergence framework, organizing methods by feedback, teacher access, and loss.

🔹 Publication Date: Published on Apr 1

🔹 Paper Links:
• arXiv Page: https://arxiv.org/abs/2604.00626
• PDF: https://arxiv.org/pdf/2604.00626

==================================

For more data science resources:
✓ https://xn--r1a.website/DataScienceT

#LLMs #OnPolicyDistillation #ModelDistillation #DeepLearning #MachineLearning

✨LinguDistill: Recovering Linguistic Ability in Vision- Language Models via Selective Cross-Modal Distillation

📝 Summary:
LinguDistill enables recovery of linguistic capabilities in vision-language models through adapter-free distillation using frozen language models as teachers, achieving performance close to pre-adapta...

🔹 Publication Date: Published on Apr 1

🔹 Paper Links:
• arXiv Page: https://arxiv.org/abs/2604.00829
• PDF: https://arxiv.org/pdf/2604.00829

==================================

For more data science resources:
✓ https://xn--r1a.website/DataScienceT

#VisionLanguageModels #NLP #ModelDistillation #ArtificialIntelligence #MachineLearning