Sequence-to-Sequence Acoustic-to-Phonetic Conversion Using Spectrograms and Deep Learning

Abstract

Distinctive phonetic features (DPFs) abstractedly describe the place, manner of articulation, and voicing of the language phonemes. While DPFs are powerful features of speech signals that capture the unique articulatory characteristics of each phoneme, the task of DPF extraction is challenged by the need for efficient computational model. Unlike the ordinary acoustic features that can be directly determined form speech waveform using closed-form expressions, DPF elements are extracted from acoustic features using machine learning (ML) techniques. Therefore, for the objective of developing an acoustic-to-phonetic converter of high accuracy and low complexity, it is important to select the input acoustic features that are simple, yet carry adequate information. This paper examines the effectiveness of using spectrogram as the acoustic feature with DPFs modeled using two deep learning techniques: the deep belief network (DBN) and the convolutional recurrent neural network (CRNN). The proposed method is applied on Modern Standard Arabic (MSA). Multi-label modeling is considered in the proposed acoustic-to-phonetic converter. The learning techniques were evaluated by proper evaluation measures that accommodate the imbalanced nature of DPF elements. The results showed that the CRNN is more accurate in extracting the DPFs than the DBN.INDEX TERMS Distinctive phonetic features, Spectrograms, speech processing, convolutional recurrent neural network, deep belief networks, KAPD corpus, Arabic, MSA.

Mispronunciation Detection and Diagnosis with Articulatory-Level Feedback Generation for Non-Native Arabic Speech

“…The process of representing the phonemes by their corresponding AFs for an end-to-end automatic speech recognition system was performed by some researchers, such as in [49] for the English language. In the literature, few researchers have investigated extracting Arabic phonetics distinctive features (PDF), such as [15,50,51]. In this research, we focused on selected AFs that are important to teach the correct pronunciation to non-native Arabic learners [52].…”

Spatial Distribution of Solar Energy (SE) and Comprehensive Potential Evaluation of Regional Development and Utilization Based on Deep Learning

“…In the current study, the SE potential is divided into three levels: solar physical potential, solar geographic potential, and solar technical potential [17][18]. These three levels of solar potential represent SE from natural resources to energy that can be utilized.…”