🔥 Lance: Unified Multimodal Modeling by Multi-Task Synergy

💡 The paper introduces Lance, a unified multimodal model that combines understanding, generation, and editing capabilities for images and videos. The goal is to develop a model that can handle multiple tasks without relying on large model capacity or focusing on specific modalities like text or images. Lance achieves this through a dual-stream architecture and collaborative multi-task training, which enables joint context learning while separating the pathways for understanding and generation.

The model uses a mixture-of-experts architecture on shared multimodal sequences, allowing it to learn from both images and videos simultaneously. To address interference among different visual tokens, the model employs modality-aware rotary positional encoding, which helps to align tasks across different modalities.

During training, Lance uses a staged multi-task training paradigm with capability-oriented objectives and adaptive data scheduling. This approach strengthens both semantic comprehension and visual generation performance. The results show that Lance outperforms existing unified models in image and video generation while maintaining strong multimodal understanding capabilities.

Overall, Lance presents a practical approach to unified multimodal modeling, demonstrating that collaborative multi-task training and a dual-stream architecture can lead to improved performance in multiple tasks without requiring large model capacity. The model has the potential to be applied to various applications that require multimodal understanding, generation, and editing capabilities.

📅 Published on May 18

🔗 Links:
• GitHub: https://github.com/huggingface
• arXiv: https://arxiv.org/abs/2605.18678
• PDF: https://arxiv.org/pdf/2605.18678
• Project Page: https://lance-project.github.io/

🤖 Models citing this paper:
• https://huggingface.co/bytedance-research/Lance

🚀 Spaces citing this paper:
• https://huggingface.co/spaces/Nayefleb/Lance

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://xn--r1a.website/PaperNexus

#MultimodalModeling #MultitaskLearning #DualStreamArchitecture #MixtureOfExperts #UnifiedModelingApproach

🔥 SFT Conflicts, RL Coexists: A Theoretical and Empirical Analysis of Multi-Task Learning for LLMs

💡 The paper explores the differences between supervised fine-tuning and reinforcement learning in enhancing multi-task reasoning for large language models. The authors found that supervised fine-tuning suffers from severe task conflicts under multi-stage training, whereas reinforcement learning enables stable coexistence across diverse tasks. They attribute this to the parameter level, where reinforcement learning induces sparse and approximately orthogonal updates across tasks.

The authors provide a theoretical explanation for this mechanism by analyzing multi-task gradient interference. They found that interference in supervised fine-tuning is norm-limited, scaling with the absolute gradient magnitude, whereas interference in reinforcement learning is variance-limited, bounded by the gradient variance induced by advantage normalization and on-policy optimization. This small variance bound yields near-orthogonal optimization directions across tasks.

Leveraging this insight, the authors propose Parallel-RL, a paradigm that decouples multi-task training, significantly improving efficiency and flexibility. The results reveal a distinction between supervised fine-tuning and reinforcement learning, highlighting the benefits of reinforcement learning in multi-task learning for large language models. The proposed Parallel-RL approach has the potential to improve the efficiency and flexibility of multi-task training, making it a promising direction for future research.

📅 Published on Aug 6

🔗 Links:
• GitHub: https://github.com/huggingface
• arXiv: https://arxiv.org/abs/2608.03573
• PDF: https://arxiv.org/pdf/2608.03573

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://xn--r1a.website/PaperNexus

#LargeLanguageModels #MultiTaskLearning #ReinforcementLearning #SupervisedFineTuning #TaskConflicts