🔥 WavTTS: Towards High-Quality Zero-Shot TTS via Direct Raw Waveform Modeling

💡 The paper introduces WavTTS, a novel text-to-speech model that directly generates raw waveforms, addressing the limitations of existing latent-space diffusion models. Current state-of-the-art models operate on compressed representations such as mel-spectrograms or VAE latents, which leads to information loss and non-end-to-end training. Directly modeling raw waveforms is theoretically beneficial but has been underexplored due to the long sequence length of audio signals.

To overcome this challenge, WavTTS employs a flow matching approach with a Diffusion Transformer architecture and a simple patchification strategy to directly model speech waveforms. The model also incorporates multi-scale mel-spectrogram supervision to provide perceptual guidance during training. Additionally, the authors investigate the impact of prediction targets and noise scheduling in waveform diffusion and develop an effective schedule design to improve generation quality.

The results show that WavTTS significantly narrows the performance gap with latent-space generative models and outperforms previous end-to-end speech generation models. Evaluations on open-source benchmarks demonstrate that WavTTS closely approaches the performance of current state-of-the-art latent generative zero-shot text-to-speech models. The findings of this paper demonstrate the feasibility of scaling diffusion-based text-to-speech models directly in the waveform space, opening a new direction for end-to-end speech generation.

📅 Published on Jun 2

🔗 Links:
• GitHub: https://github.com/huggingface
• arXiv: https://arxiv.org/abs/2606.03455
• PDF: https://arxiv.org/pdf/2606.03455
• Project Page: https://wavtts.github.io/

🤖 Models citing this paper:
• https://huggingface.co/worstchan/WavTTS
• https://huggingface.co/drbaph/WavTTS

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://xn--r1a.website/PaperNexus

#TextToSpeechSynthesis #RawWaveformModeling #DiffusionTransformer #ZeroShotTTS #SpeechSynthesisTechniques