🔥 Beyond Mode Collapse: Distribution Matching for Diverse Reasoning

💡 The paper addresses the problem of mode collapse in on-policy reinforcement learning, where methods like GRPO concentrate probability mass on a single solution and cease exploring alternative strategies. This is due to the reverse KL minimization method used, which reinforces the first high-reward trajectory found rather than maintaining a distribution over multiple diverse solutions. To solve this problem, the authors propose DMPO, a distribution-matching policy optimization method that uses forward KL minimization to maintain solution diversity and improve performance in combinatorial optimization and reasoning tasks. DMPO constructs a target distribution over sampled trajectories proportional to their rewards and aligns the policy distribution to this target, providing mode-covering behavior without requiring sampling from the intractable global target distribution. The authors validate DMPO on NP-hard combinatorial optimization tasks and achieve significant improvements over GRPO, with a 43.9 percent quality ratio on text-based tasks and 43.1 percent on vision-based tasks. These gains generalize to mathematical reasoning and out-of-domain tasks, demonstrating that diversity-preserving training enhances general reasoning capabilities across modalities. The results show that DMPO achieves consistent quality improvements and sustained exploration across diverse reasoning tasks, establishing distribution matching as a practical approach to preventing mode collapse in on-policy reinforcement learning.

📅 Published on May 19

🔗 Links:
• GitHub: https://github.com/huggingface
• arXiv: https://arxiv.org/abs/2605.19461
• PDF: https://arxiv.org/pdf/2605.19461

📊 Datasets citing this paper:
• https://huggingface.co/datasets/OliverLee/NP_MM

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://xn--r1a.website/PaperNexus

#ModeCollapseMitigation #DistributionMatching #OnPolicyReinforcementLearning #DiverseReasoningTasks #CombinatorialOptimizationTechniques

🔥 Representation Distribution Matching for One-Step Visual Generation

💡 The paper introduces Representation Distribution Matching, a method for one-step visual generation that matches feature distributions under pretrained encoders. The goal is to generate high-quality images by comparing the distributions of generated and reference features. The authors identify two key design axes: how the distributions are compared and the representations they are compared in. They conduct controlled studies and find three main results.

First, they show that the Maximum Mean Discrepancy, a classical method that was previously ineffective, becomes a strong and scalable objective when estimated correctly. Second, they find that the batch size of the generated images has a significant impact on performance, with an optimum batch size above 2048, which is much larger than typical batch sizes. Third, they demonstrate that using a single representation can be gamed, resulting in low scores despite visibly fake images, and instead propose using a balanced set of encoders and evaluating with a Sliced-Wasserstein distance over 14 encoders.

The authors combine these findings to develop an improved Representation Distribution Matching method, which they call iRDM. They evaluate iRDM on the ImageNet dataset and achieve state-of-the-art results, with a Sliced-Wasserstein distance of 1.30. Additionally, they use a human-preference proxy, called PickScore, which shows that iRDM is preferred over the previous best one-step generator on 71.2% of matched samples. They also apply the same method to post-train a four-step generator, called FLUX.2, and achieve better results than the original four-step version, with improved performance on GenEval and PickScore, and requiring only 90 GPU-hours. Overall, the paper presents a new method for one-step visual generation that achieves state-of-the-art results and can be used to improve existing generators.

📅 Published on Jul 2

🔗 Links:
• GitHub: https://github.com/huggingface
• arXiv: https://arxiv.org/abs/2607.02375
• PDF: https://arxiv.org/pdf/2607.02375
• Project Page: https://alan-lanfeng.github.io/rdm/

🤖 Models citing this paper:
• https://huggingface.co/epfl-vita/flux2-klein-1step-rdm

🚀 Spaces citing this paper:
• https://huggingface.co/spaces/epfl-vita/flux2-klein-1step-demo

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://xn--r1a.website/PaperNexus

#VisualGeneration #RepresentationLearning #DistributionMatching #ImageSynthesis #DeepLearning

🔥 DistillAlign: Coordinating Mode Covering and Mode Seeking in Autoregressive Video Distillation

💡 This paper addresses the issue of existing autoregressive video distillation methods which typically decouple the initialization and distribution matching stages, leading to suboptimal results. The authors argue that a good initialization should match the mode coverage of the target distribution, rather than merely pursuing high quality. To analyze this, they introduce a distributional evaluation protocol that measures precision and coverage between student and teacher distributions in a shared latent space.

The authors find that some initializations reach high precision but low coverage, leading to suboptimal refinement, while mode-covering ones preserve broader support. Furthermore, even when the target distributions are aligned, the reverse-KL objective of distribution matching can still drive the student towards high-probability teacher regions in late training, reducing coverage and diversity.

To address this, the authors propose joint distillation, which combines the mode-seeking objective of distribution matching with a consistency distillation-based mode-covering constraint. The experiments show that their method improves generation quality, coverage, and diversity. Notably, even with a smaller teacher model, their method outperforms baselines refined with a larger teacher model, underscoring the importance of distributional alignment in autoregressive video distillation.

The main contributions of this paper are the introduction of a distributional evaluation protocol and the proposal of joint distillation, which coordinates mode covering and mode seeking in autoregressive video distillation. The results demonstrate the effectiveness of the proposed method in improving the quality and diversity of generated videos.

📅 Published on Jul 29

🔗 Links:
• GitHub: https://github.com/huggingface
• arXiv: https://arxiv.org/abs/2607.26811
• PDF: https://arxiv.org/pdf/2607.26811
• Project Page: https://lijiaxing0213.github.io/DistillAlign/

📊 Datasets citing this paper:
• https://huggingface.co/datasets/LiJiaxing/DistillAlign_1p3b_25K

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://xn--r1a.website/PaperNexus

#AutoregressiveVideoDistillation #ModeCovering #DistributionMatching #VideoDistillationMethods #LatentSpaceAnalysis