🔥 VibeThinker-3B: Exploring the Frontier of Verifiable Reasoning in Small Language Models

💡 The paper introduces VibeThinker-3B, a compact language model with 3 billion parameters, that achieves state-of-the-art performance on verifiable reasoning tasks, challenging the conventional assumption that large models are necessary for such tasks. The model was developed using a specialized training pipeline that includes curriculum-based supervised fine-tuning, multi-domain reinforcement learning, and offline self-distillation. The model was evaluated on several highly demanding verifiable tasks and achieved impressive results, including a score of 94.3 on AIME26, 80.2 Pass@1 on LiveCodeBench v6, and a 96.1 percent acceptance rate on recent unseen LeetCode contests. These results place VibeThinker-3B in the performance band of first-tier reasoning systems, matching or exceeding the performance of much larger models. The paper also shows that the model's performance does not compromise its instruction controllability, with a score of 93.4 on IFEval. The results of this study support the Parametric Compression-Coverage Hypothesis, which suggests that verifiable reasoning can be compressed into compact reasoning cores, while open-domain knowledge and general-purpose competence require larger models with broader parameter coverage. Overall, the paper demonstrates that compact models can be a complementary path to achieving frontier-level performance on verifiable reasoning tasks, and that they are not just efficient substitutes for larger models.

📅 Published on Jun 15

🔗 Links:
• GitHub: https://github.com/huggingface
• arXiv: https://arxiv.org/abs/2606.16140
• PDF: https://arxiv.org/pdf/2606.16140
• Project Page: https://github.com/WeiboAI/VibeThinker

🤖 Models citing this paper:
• https://huggingface.co/WeiboAI/VibeThinker-3B
• https://huggingface.co/KakTakOne/VibeThinker-3B-GGUF
• https://huggingface.co/ffkbblu/pepekberbulu

🚀 Spaces citing this paper:
• https://huggingface.co/spaces/Mike0021/vibethinker-3b-zerogpu
• https://huggingface.co/spaces/ffkbblu/trst

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://xn--r1a.website/PaperNexus

#VerifiableReasoning #SmallLanguageModels #CompactModelArchitecture #ReinforcementLearningForNLP #EfficientLanguageModeling