💡 The paper addresses limitations in existing Flow Matching text-to-image models, which suffer from two main issues: reward sparsity and gradient interference. These problems lead to poor generation quality and alignment metrics. To overcome these challenges, the authors propose Flow-OPD, a two-stage alignment approach that combines on-policy distillation and manifold anchor regularization.

In the first stage, the authors fine-tune domain-specialized teacher models using single-reward GRPO fine-tuning, allowing each expert to reach its performance ceiling. Then, they establish a robust initial policy through a Flow-based Cold-Start scheme and consolidate heterogeneous expertise into a single student model.

The authors also introduce Manifold Anchor Regularization, which leverages a task-agnostic teacher to provide full-data supervision and anchors generation to a high-quality manifold. This helps mitigate aesthetic degradation commonly observed in purely RL-driven alignment.

The results show that Flow-OPD significantly improves generation quality and alignment metrics, raising the GenEval score from 63 to 92 and the OCR accuracy from 59 to 94. This represents an overall improvement of roughly 10 points over vanilla GRPO, while preserving image fidelity and human-preference alignment. The authors also observe an emergent teacher-surpassing effect, where the student model outperforms the teacher models. Overall, Flow-OPD establishes a scalable alignment paradigm for building generalist text-to-image models.