Phoneme segmentation using spectrogram reading knowledge

Hatazaki, Kaichiro; Komori, Yasuhiro; Kawabata, Takeshi; Shikano, Kiyohiro

doi:10.1109/icassp.1989.266448

Cited by 14 publications

(8 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…S PECTROGRAMS, as time-frequency representations of audio signals, have been used as input for neural network models since the 1980s [1][2][3]. Different types of spectrograms are tailored to different applications.…”

Section: Introductionmentioning

confidence: 99%

nnAudio: An on-the-Fly GPU Audio to Spectrogram Conversion Toolbox Using 1D Convolutional Neural Networks

Cheuk

Anderson²,

Agres

et al. 2020

IEEE Access

View full text Add to dashboard Cite

In this paper, we present nnAudio, a new neural network-based audio processing framework with graphics processing unit (GPU) support that leverages 1D convolutional neural networks to perform time domain to frequency domain conversion. It allows on-the-fly spectrogram extraction due to its fast speed, without the need to store any spectrograms on the disk. Moreover, this approach also allows backpropagation on the waveforms-to-spectrograms transformation layer, and hence, the transformation process can be made trainable, further optimizing the waveform-to-spectrogram transformation for the specific task that the neural network is trained on. All spectrogram implementations scale as Big-O of linear time with respect to the input length. nnAudio, however, leverages the compute unified device architecture (CUDA) of 1D convolutional neural network from PyTorch, its short-time Fourier transform (STFT), Mel spectrogram, and constant-Q transform (CQT) implementations are an order of magnitude faster than other implementations using only the central processing unit (CPU). We tested our framework on three different machines with NVIDIA GPUs, and our framework significantly reduces the spectrogram extraction time from the order of seconds (using a popular python library librosa) to the order of milliseconds, given that the audio recordings are of the same length. When applying nnAudio to variable input audio lengths, an average of 11.5 hours are required to extract 34 spectrogram types with different parameters from the MusicNet dataset using librosa. An average of 2.8 hours is required for nnAudio, which is still four times faster than librosa. Our proposed framework also outperforms existing GPU processing libraries such as Kapre and torchaudio in terms of processing speed.

show abstract

Section: Introductionmentioning

confidence: 99%

nnAudio: An on-the-Fly GPU Audio to Spectrogram Conversion Toolbox Using 1D Convolutional Neural Networks

Cheuk

Anderson²,

Agres

et al. 2020

IEEE Access

View full text Add to dashboard Cite

show abstract

“…It was found that FFT spectrogram was found to be a more robust feature for segmentation and classification even in noisy environments [7][8][9][10][11][12][13].…”

Section: Introductionmentioning

confidence: 99%

Continuous Arabic Speech Segmentation using FFT Spectrogram

Awais

Ahmad

Masud

et al. 2006

2006 Innovations in Information Technology

View full text Add to dashboard Cite

This paper describes a phoneme segmentation algorithm that uses Fast Fourier Transform (FFT) spectrogram. The algorithm has been implemented and tested for utterances of continuous Arabic speech of 10 male speakers that contain almost 2346 phonemes in total. The recognition system determines the phoneme boundaries and identifies them as pauses, vowels and consonants. The system uses intensity and phoneme duration for separating pauses from consonants. Intensity in particular is used to detect two specific consonants (/r/ , /h/) when they are not detected through the spectrographic information. Segmentation accuracy of 95.39% for the overall system has been achieved.

show abstract

“…The spectrum of the signal is shown in every small vertical slice of the spectrogram. The robustness and convenience of the spectrogram in speech application is approved through reading (Laprie and Berger, 1996;Hatazaki et al, 1989;Silverman and Lee, 1987;Hunt, 1987;Connolly and Edmonds, 1986;Zue and Lamel, 1986;Zue and Cole, 1979;Mporas et al, 2007).…”

Section: Introductionmentioning

confidence: 99%