Speech Synthesis

insight - Speech Synthesis

Improving Zero-Shot Text-to-Speech Synthesis with Multi-Scale Acoustic Prompts

A novel zero-shot text-to-speech model that utilizes multi-scale acoustic prompts, including a style prompt to capture personal speaking style and a timbre prompt to preserve the target speaker's voice characteristics, outperforming state-of-the-art language model-based approaches in terms of naturalness and speaker similarity.

RALL-E: A Robust Codec Language Model with Chain-of-Thought Prompting for Improved Text-to-Speech Synthesis

RALL-E improves the robustness of large language model-based text-to-speech synthesis by incorporating prosody tokens as chain-of-thought prompting and using duration-guided masking to enhance the alignment between phonemes and speech tokens.

Enhancing Korean Text-to-Speech Synthesis through Integrated Modeling of Syntactic and Acoustic Cues for Improved Pause Formation

Leveraging the interplay between syntactic and acoustic cues to enhance pause prediction and placement for more natural Korean text-to-speech synthesis, even for longer and more complex sentences.

Generating Emotionally Expressive and Disfluent Speech for Conversational AI Systems

A novel speech synthesis pipeline that generates emotional and disfluent speech patterns in a zero-shot manner using a large language model, enabling more natural and relatable interactions for conversational AI systems.

Enhancing Real-Time Text-to-Speech Synthesis Efficiency through Consistency Models and Weighted Samplers

CM-TTS, a novel architecture based on consistency models, achieves high-quality speech synthesis in fewer steps without adversarial training or pre-trained model dependencies. Weighted samplers are introduced to mitigate biases during model training.

LipVoicer: Generating High-Quality and Intelligible Speech from Silent Videos Using Lip Reading and Diffusion Models

LipVoicer is a novel method that generates high-quality and intelligible speech from silent videos by leveraging a lip-reading model to guide a diffusion-based speech generation model.

Training Generative Adversarial Network-Based Vocoder with Limited Data Using Augmentation-Conditional Discriminator

AugCondD improves speech quality under limited data conditions while maintaining comparable quality under sufficient data conditions.

Mega-TTS 2: Boosting Prompting Mechanisms for Zero-Shot Speech Synthesis at ICLR 2024

Zero-shot TTS systems face challenges in prompting mechanisms, Mega-TTS 2 introduces a generic mechanism to tackle these challenges effectively.

EM-TTS: Lightweight Mongolian Text-to-Speech System Based on CNN

提案されたEM-TTSモデルは、訓練時間とモデルパラメータを大幅に削減しながら、一定の合成品質を維持します。

Empirical Study of Speech Language Models for Prompt-Conditioned Speech Synthesis

Prompt and content characteristics significantly impact speech synthesis quality.

About

Products | Resources

Insights