✨SkillFactory: Self-Distillation For Learning Cognitive Behaviors

📝 Summary:
SkillFactory fine-tunes models to learn cognitive skills using self-generated data before reinforcement learning. This self-distillation method enhances robustness and generalization post-RL, enabling models to effectively utilize acquired cognitive skills.

🔹 Publication Date: Published on Dec 3

🔹 Paper Links:
• arXiv Page: https://arxiv.org/abs/2512.04072
• PDF: https://arxiv.org/pdf/2512.04072

==================================

For more data science resources:
✓ https://xn--r1a.website/DataScienceT

#SelfDistillation #ReinforcementLearning #CognitiveAI #MachineLearning #AIResearch

✨Reinforcement Learning via Self-Distillation

📝 Summary:
Self-Distillation Policy Optimization SDPO leverages rich textual feedback to address the credit-assignment bottleneck in reinforcement learning. SDPO treats the model as a self-teacher, distilling feedback-informed predictions to improve sample efficiency and accuracy. It significantly enhances ...

🔹 Publication Date: Published on Jan 28

🔹 Paper Links:
• arXiv Page: https://arxiv.org/abs/2601.20802
• PDF: https://arxiv.org/pdf/2601.20802

==================================

For more data science resources:
✓ https://xn--r1a.website/DataScienceT

#ReinforcementLearning #SelfDistillation #MachineLearning #AI #PolicyOptimization

✨Self-Distilled RLVR

📝 Summary:
RLSD combines reinforcement learning with verifiable rewards RLVR and self-distillation to overcome sparse feedback. It uses self-distillation for fine-grained update magnitudes and RLVR for reliable update directions. This achieves superior training stability and convergence.

🔹 Publication Date: Published on Apr 3

🔹 Paper Links:
• arXiv Page: https://arxiv.org/abs/2604.03128
• PDF: https://arxiv.org/pdf/2604.03128

==================================

For more data science resources:
✓ https://xn--r1a.website/DataScienceT

#ReinforcementLearning #SelfDistillation #RLVR #MachineLearning #AI

🔥 D-OPSD: On-Policy Self-Distillation for Continuously Tuning Step-Distilled Diffusion Models

💡 The paper introduces D-OPSD, a new training approach for diffusion models that enables efficient supervised fine-tuning while preserving few-step inference capabilities. The current landscape of high-performance image generation models is shifting from inefficient multi-step models to efficient few-step models, but these models are challenging to fine-tune using traditional techniques. The problem with traditional fine-tuning methods is that they compromise the model's inherent few-step inference capability.

To address this issue, the authors propose D-OPSD, which leverages on-policy self-distillation with text and multimodal features. The method works by making the model act as both the teacher and the student, where the student is conditioned only on the text feature, and the teacher is conditioned on the multimodal feature of both the text prompt and the target image. The training process minimizes the difference between the predicted distributions over the student's own roll-outs, allowing the model to learn new concepts and styles without sacrificing its original few-step capacity.

The key contribution of D-OPSD is that it enables on-policy learning during supervised fine-tuning, which allows the model to learn from its own trajectory and under its own supervision. This approach enables the model to inherit the in-context capabilities of its encoder, making it possible to fine-tune the model continuously without compromising its few-step inference capability. The results show that D-OPSD enables efficient supervised fine-tuning for diffusion models, making it a promising approach for high-performance image generation models.

📅 Published on May 6

🔗 Links:
• arXiv: https://arxiv.org/abs/2605.05204
• PDF: https://arxiv.org/pdf/2605.05204
• Project Page: https://vvvvvjdy.github.io/d-opsd/
• GitHub: https://github.com/vvvvvjdy/D-OPSD ⭐ 24

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://xn--r1a.website/PaperNexus

#DiffusionModels #SelfDistillation #FewShotLearning #ImageGeneration #MultimodalLearning