💡 The paper introduces AnyFlow, a novel framework for any-step video diffusion distillation that improves upon existing consistency distillation methods. The problem with consistency distillation is that its performance degrades as more sampling steps are used at test time, limiting its effectiveness for any-step video diffusion. This is because consistency distillation replaces the original probability-flow ODE trajectory with a consistency-sampling trajectory, which weakens the desirable test-time scaling behavior of ODE sampling.

To address this limitation, AnyFlow optimizes the full ODE sampling trajectory instead of distilling a model for only a few fixed sampling steps. The method involves shifting the distillation target from endpoint consistency mapping to flow-map transition learning over arbitrary time intervals. Additionally, the authors propose Flow Map Backward Simulation, which decomposes a full Euler rollout into shortcut flow-map transitions, enabling efficient on-policy distillation that reduces test-time errors.

The results of the paper show that AnyFlow achieves performance that matches or surpasses consistency-based counterparts in the few-step regime, while also scaling with sampling step budgets. The experiments were conducted across both bidirectional and causal architectures, at scales ranging from 1.3B to 14B parameters. Overall, the paper contributes a new framework for any-step video diffusion distillation that improves upon existing methods and achieves state-of-the-art results.

💡 The paper proposes a framework called On-Policy Skill Distillation, or OPID, which aims to improve the efficiency and performance of language agent training in reinforcement learning. The problem addressed is that outcome-based reinforcement learning provides sparse rewards that do not offer sufficient guidance on intermediate decisions, while existing self-distillation methods often rely on external skill memories that can be costly to maintain and may not match the current policy.

OPID addresses this issue by extracting skill supervision directly from completed on-policy trajectories, representing trajectory hindsight as hierarchical skills that capture both global and local decision knowledge. The framework uses a critical-first routing mechanism to select the most relevant skill and inject it into the interaction history, allowing the old policy to re-score responses under both original and skill-augmented contexts. This yields a token-level self-distillation advantage that is combined with the outcome advantage for policy optimization.

The results of the experiments demonstrate that OPID generally improves agent performance, sample efficiency, and robustness over outcome-only reinforcement learning and existing skill-distillation baselines. The framework preserves reinforcement learning as the primary training objective while introducing dense, distribution-matched hindsight supervision. The experiments were conducted on several datasets, including ALFWorld, WebShop, and Search-based QA, and the code is available for further research. Overall, OPID offers a novel approach to skill distillation that can enhance the training of language agents in reinforcement learning.