🔥 MOSS-TTS Technical Report

💡 The MOSS-TTS technical report presents a speech generation model that utilizes discrete audio tokens and autoregressive modeling to achieve voice cloning, pronunciation control, and long-form generation across multiple languages. The model is built on a scalable recipe that includes a causal Transformer tokenizer, which compresses 24 kHz audio to 12.5 fps with variable-bitrate RVQ and unified semantic-acoustic representations. The report releases two complementary generators: MOSS-TTS, which emphasizes structural simplicity, scalability, and long-context/control-oriented deployment, and MOSS-TTS-Local-Transformer, which introduces a frame-local autoregressive module for higher modeling efficiency, stronger speaker preservation, and a shorter time to first audio.

The problem addressed by the report is the need for a speech generation model that can handle multilingual and open-domain settings, and support various features such as voice cloning, pronunciation control, and long-form generation. The method used to address this problem is the development of the MOSS-TTS model, which is built on a combination of discrete audio tokens, autoregressive modeling, and large-scale pretraining.

The results of the report show that the MOSS-TTS model supports zero-shot voice cloning, token-level duration control, phoneme-/pinyin-level pronunciation control, smooth code-switching, and stable long-form generation across multilingual and open-domain settings. The report also summarizes the design, training recipe, and empirical characteristics of the released models, providing a comprehensive overview of the MOSS-TTS model and its capabilities. Overall, the MOSS-TTS model presents a significant contribution to the field of speech generation, offering a scalable and efficient solution for a wide range of applications.

📅 Published on Mar 18

🔗 Links:
• GitHub: https://github.com/huggingface
• arXiv: https://arxiv.org/abs/2603.18090
• PDF: https://arxiv.org/pdf/2603.18090
• Project Page: https://mosi.cn/models/moss-tts

🤖 Models citing this paper:
• https://huggingface.co/OpenMOSS-Team/MOSS-TTS
• https://huggingface.co/OpenMOSS-Team/MOSS-TTS-Nano-100M
• https://huggingface.co/OpenMOSS-Team/MOSS-TTS-Realtime

📊 Datasets citing this paper:
• https://huggingface.co/datasets/somu9/mls_eng_tokens

🚀 Spaces citing this paper:
• https://huggingface.co/spaces/OpenMOSS-Team/MOSS-TTS-v1.5
• https://huggingface.co/spaces/OpenMOSS-Team/MOSS-TTS-Nano
• https://huggingface.co/spaces/OpenMOSS-Team/MOSS-TTS

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://xn--r1a.website/PaperNexus

#SpeechGeneration #VoiceCloning #AutoregressiveModeling #DiscreteAudioTokens #TransformerTokenizer

🔥 Speaking the Language of Science: Toward a General-Purpose Generative Foundation Model for the Natural Sciences

💡 The paper introduces LOGOS, a scientific generative language model that unifies various tasks across the natural sciences within a single autoregressive framework. The model encodes diverse scientific objects and their spatial interactions as token sequences over a common vocabulary, allowing it to capture complex structural interactions in a purely sequential manner. This approach enables a wide range of downstream tasks to be formulated consistently as next-token prediction in the same grammar space, creating strong alignment between continued multi-domain pre-training and downstream objectives.

The researchers trained LOGOS models at different scales, including 1B, 3B, and 8B parameters, and found a consistent positive correlation between model size and performance. The model consistently matches or outperforms domain-specific baselines across diverse tasks, providing preliminary evidence for the feasibility of a single model that can perform well across multiple domains in the natural sciences.

The paper's main contribution is the demonstration of a unified scientific generative language model that can be applied to various tasks in the natural sciences, including those that involve spatial interactions and complex structural relationships. The results suggest that the future of AI for science may lie in deeply aligning scientific foundation models with large language models, rather than building separate technical stacks. The release of the model weights and associated resources is intended to facilitate further research in this area.

The problem addressed by the paper is the lack of a unified framework for modeling various tasks in the natural sciences, which often require separate domain-specific models. The method used to address this problem is the development of a scientific generative language model that can encode diverse scientific objects and spatial interactions as token sequences, allowing for a wide range of downstream tasks to be formulated consistently as next-token prediction.

The results of the paper demonstrate the effectiveness of the LOGOS model in performing various tasks across the natural sciences, including those that involve spatial interactions and complex structural relationships. The positive correlation between model size and performance suggests that larger models may be able to achieve even better results, and the release of the model weights and associated resources is intended to facilitate further research in this area. Overall, the paper contributes to the development of a unified framework for modeling various tasks in the natural sciences, and demonstrates the potential of scientific generative language models for advancing AI research in this area.

📅 Published on Jun 15

🔗 Links:
• GitHub: https://github.com/huggingface
• arXiv: https://arxiv.org/abs/2606.16905
• PDF: https://arxiv.org/pdf/2606.16905

🤖 Models citing this paper:
• https://huggingface.co/LOGOS-Hub/LOGOS-8B
• https://huggingface.co/LOGOS-Hub/LOGOS-pretrain-1B
• https://huggingface.co/LOGOS-Hub/LOGOS-pretrain-8B

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://xn--r1a.website/PaperNexus

#NaturalScienceLanguageModels #GenerativeFoundationModels #ScientificLanguageProcessing #AutoregressiveModeling #MultidomainLearning

🔥 JetSpec: Breaking the Scaling Ceiling of Speculative Decoding with Parallel Tree Drafting

💡 The paper introduces JetSpec, a speculative decoding framework designed to improve the inference speed and acceptance rates of large language models. The problem addressed is the scaling limitation of speculative decoding, which accelerates autoregressive large language models by drafting multiple tokens and verifying them in parallel. However, increasing the draft budget only improves speed when acceptance remains high and drafting overhead stays low, creating a scaling ceiling.

The proposed JetSpec framework combines efficient forward drafting with causal conditioning to break this ceiling. It trains a causal parallel draft head over fused hidden states from the frozen target model, producing candidate trees whose scores align with the target model's autoregressive factorization. This approach enables JetSpec to convert larger draft budgets into longer accepted prefixes and higher end-to-end speedup.

The method is compared to bidirectional-head and tree-based speculative decoding baselines across various benchmarks, including math, coding, and chat tasks on dense and MoE models. The results show that JetSpec consistently outperforms these baselines, achieving significant speedup on different workloads. Specifically, JetSpec achieves up to 9.64x speedup on math tasks and 4.58x on open-ended conversational workloads, with further latency gains demonstrated through integration with virtual large language models under realistic serving loads.

Overall, the paper contributes a novel speculative decoding framework that breaks the scaling ceiling of prior methods, enabling faster and more efficient large language model inference. The code and models are made available for further research and development.

📅 Published on Jun 25

🔗 Links:
• GitHub: https://github.com/huggingface
• arXiv: https://arxiv.org/abs/2606.18394
• PDF: https://arxiv.org/pdf/2606.18394
• Project Page: https://jetspec-project.github.io/jetspec-web/

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://xn--r1a.website/PaperNexus

#SpeculativeDecoding #LargeLanguageModels #AutoregressiveModeling #ParallelTreeDrafting #CausalConditioning