EMPHASIS: An Emotional Phoneme-based Acoustic Model for Speech Synthesis System

Li, Hao; Kang, Yongguo; Wang, Zhenyu

doi:10.21437/interspeech.2018-1511

Cited by 15 publications

(7 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Later, as the development of statistics machine learning, statistical parametric speech synthesis (SPSS) is proposed [409,350,418,351], which predicts parameters such as spectrum, fundamental frequency and duration for speech synthesis. From 2010s, neural network-based speech synthesis [419,278,76,417,369,187,248,376] has gradually become the dominant methods and achieved much better voice quality.…”

Section: History Of Tts Technologymentioning

confidence: 99%

A Survey on Neural Speech Synthesis

Tan,

Qin,

Soong

et al. 2021

Preprint

View full text Add to dashboard Cite

Text to speech (TTS), or speech synthesis, which aims to synthesize intelligible and natural speech given text, is a hot research topic in speech, language, and machine learning communities and has broad applications in the industry. As the development of deep learning and artificial intelligence, neural network-based TTS has significantly improved the quality of synthesized speech in recent years. In this paper, we conduct a comprehensive survey on neural TTS, aiming to provide a good understanding of current research and future trends. We focus on the key components in neural TTS, including text analysis, acoustic models, and vocoders, and several advanced topics, including fast TTS, low-resource TTS, robust TTS, expressive TTS, and adaptive TTS, etc. We further summarize resources related to TTS (e.g., datasets, opensource implementations) and discuss future research directions. This survey can serve both academic researchers and industry practitioners working on TTS.

show abstract

Section: History Of Tts Technologymentioning

confidence: 99%

A Survey on Neural Speech Synthesis

Tan,

Qin,

Soong

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…In recent years, emotional speech synthesis based on deep learning has become an emergent area of interest. LSTM-based emotional statistical parametric speech synthesis method is proposed by [2], and [25] further tried to synthesize questionable and exclamatory speech rather than specific emotional categories with a real time synthesis system. With the rapid development of deep learning, convolutional neural networks been proven effective in the field of audio processing.…”

Section: Related Workmentioning

confidence: 99%

ET-GAN: Cross-Language Emotion Transfer Based on Cycle-Consistent Generative Adversarial Networks

Jia,

Tai,

Zhou

et al. 2019

Preprint

View full text Add to dashboard Cite

Despite the remarkable progress made in synthesizing emotional speech from text, it is still challenging to provide emotion information to existing speech segments. Previous methods mainly rely on parallel data, and few works have studied the generalization ability for one model to transfer emotion information across different languages. To cope with such problems, we propose an emotion transfer system named ET-GAN, for learning language-independent emotion transfer from one emotion to another without parallel training samples. Based on cycle-consistent generative adversarial network, our method ensures the transfer of only emotion information across speeches with simple loss designs. Besides, we introduce an approach for migrating emotion information across different languages by using transfer learning. The experiment results show that our method can efficiently generate high-quality emotional speech for any given emotion category, without aligned speech pairs.

show abstract

“…synthesis Li et al, 2018), to neural network based parametric synthesis (Arik et al, 2017), and to currently end-to-end neural models. The end-to-end models directly map input text or phonetic characters to output speech, which greatly simplifies the training pipeline and reduces the requirements for linguistic and acoustic knowledge.…”

Section: Introductionmentioning

confidence: 99%

HiFiSinger: Towards High-Fidelity Neural Singing Voice Synthesis

Chen¹,

Tan²,

Luan³

et al. 2020

Preprint

View full text Add to dashboard Cite

High-fidelity singing voices usually require higher sampling rate (e.g., 48kHz, compared with 16kHz or 24kHz in speaking voices) with large range of frequency to convey expression and emotion. However, higher sampling rate causes the wider frequency band and longer waveform sequences and throws challenges for singing modeling in both frequency and time domains in singing voice synthesis (SVS). Conventional SVS systems that adopt moderate sampling rate (e.g., 16kHz or 24kHz) cannot well address the above challenges. In this paper, we develop HiFiSinger, an SVS system towards high-fidelity singing voice using 48kHz sampling rate. HiFiSinger consists of a FastSpeech based neural acoustic model and a Parallel WaveGAN based neural vocoder to ensure fast training and inference and also high voice quality. To tackle the difficulty of singing modeling caused by high sampling rate (wider frequency band and longer waveform), we introduce multi-scale adversarial training in both the acoustic model and vocoder to improve singing modeling. Specifically, 1) To handle the larger range of frequencies caused by higher sampling rate (e.g., 48kHz vs. 24kHz), we propose a novel sub-frequency GAN (SF-GAN) on mel-spectrogram generation, which splits the full 80-dimensional mel-frequency into multiple sub-bands (e.g. low, middle and high frequency bands) and models each sub-band with a separate discriminator. 2) To model longer waveform sequences caused by higher sampling rate, we propose a multi-length GAN (ML-GAN) for waveform generation to model different lengths of waveform sequences with separate discriminators. 3) We also introduce several additional designs and findings in HiFiSinger that are crucial for high-fidelity voices, such as adding F0 (pitch) and V/UV (voiced/unvoiced flag) as acoustic features, choosing an appropriate window/hop size for mel-spectrogram, and increasing the receptive field in vocoder for long vowel modeling in singing voices. Experiment results show that HiFiSinger synthesizes high-fidelity singing voices with much higher quality: 0.32/0.44 MOS gain over 48kHz/24kHz baseline and 0.83 MOS gain over previous SVS systems. Audio samples are available at https://speechresearch.github.io/hifisinger/.

show abstract

EMPHASIS: An Emotional Phoneme-based Acoustic Model for Speech Synthesis System

Cited by 15 publications

References 12 publications

A Survey on Neural Speech Synthesis

A Survey on Neural Speech Synthesis

ET-GAN: Cross-Language Emotion Transfer Based on Cycle-Consistent Generative Adversarial Networks

HiFiSinger: Towards High-Fidelity Neural Singing Voice Synthesis

Contact Info

Product

Resources

About