Non-Contact Speech Recovery Technology Using a 24 GHz Portable Auditory Radar and Webcam

Abstract

Language has been one of the most effective ways of human communication and information exchange. To solve the problem of non-contact robust speech recognition, recovery, and surveillance, this paper presents a speech recovery technology based on a 24 GHz portable auditory radar and webcam. The continuous-wave auditory radar is utilized to extract the vocal vibration signal, and the webcam is used to obtain the fitted formant frequency. The traditional formant speech synthesizer is selected to synthesize and recover speech, using the vocal vibration signal as the sound source excitation and the fitted formant frequency as the vocal tract resonance characteristics. Experiments on reading single English characters and words are carried out. Using microphone records as a reference, the effectiveness of the proposed speech recovery technology is verified. Mean opinion scores show a relatively high consistency between the synthesized speech and original acoustic speech.

94 GHz Asymmetric Antenna Radar for Speech Signal Detection and Enhancement via Variational Mode Decomposition and Improved Threshold Strategy

“…At present, the present technologies for obtaining speech signals can be divided into air conduction and non-air conduction detection. However, the shortcomings limit the development of these techniques for speech detection [1][2][3][4][5]. In recent years, bio-radar technology has been developed for using in a variety of remote sensing applications [6,7].…”

EPG2S: Speech Generation and Speech Enhancement Based on Electropalatography and Audio Signals Using Multimodal Learning

“…Based on recent advances in machine learning-based technologies, the conversion of biosignals to speech signals has been reported in several studies [3], [4], [5]. Various signals have been considered for speech generation and enhancement, including surface electromyography (sEMG) [3], [6], electromagnetic articulography (EMA) [4], [7], permanent magnetic articulography (PMA) [5], [8], ultrasound tongue imaging [9], [10], Doppler signals [11], [12], visual cues [13], [14], and bone-conducted microphone signals [15]. Further, multimodal learning has been leveraged to integrate information from complementary data, such as text [16], videos [13], boneconducted microphone signals [15], and articulatory movements [4].However, the transformation of articulatory movements to facilitate communication has not yet been adequately researched.…”