💡 The paper introduces LingBot-Video, a video pretraining framework designed for embodied intelligence applications, such as robot control. The current video generative models are not suitable for these applications due to their focus on content creation, prioritizing visual fidelity and creativity over computational efficiency and physical realism. To address this issue, the authors propose a framework that combines a Mixture-of-Experts architecture with specialized data augmentation and a multi-dimensional reward system.

The Mixture-of-Experts architecture is used instead of a dense framework to achieve a better balance between modeling capacity and inference efficiency. The authors also construct a data profiling engine that augments standard internet videos with robot-oriented footage, including manipulation, navigation, and egocentric perspectives, to equip the base model with an understanding of actions and world dynamics.

The training process involves a multi-dimensional reward system that enforces physical rationality and task completion, in addition to standard criteria such as aesthetics and motion consistency. The authors evaluate the performance and efficiency of LingBot-Video and validate its effectiveness as a video foundation model.

The main contributions of the paper are the introduction of LingBot-Video, a large-scale, open-source Mixture-of-Experts video foundation model, and the development of a framework that bridges digital creativity and physical actuation. The authors provide a pioneering effort to address the domain mismatch between video generative models and embodied intelligence applications, and their work has the potential to improve the performance of robots in various tasks.