Exploring Silent Speech Interfaces Based on Frequency-Modulated Continuous-Wave Radar

Abstract

Speech is our most natural and efficient form of communication and offers a strong potential to improve how we interact with machines. However, speech communication can sometimes be limited by environmental (e.g., ambient noise), contextual (e.g., need for privacy), or health conditions (e.g., laryngectomy), preventing the consideration of audible speech. In this regard, silent speech interfaces (SSI) have been proposed as an alternative, considering technologies that do not require the production of acoustic signals (e.g., electromyography and video). Unfortunately, despite their plentitude, many still face limitations regarding their everyday use, e.g., being intrusive, non-portable, or raising technical (e.g., lighting conditions for video) or privacy concerns. In line with this necessity, this article explores the consideration of contactless continuous-wave radar to assess its potential for SSI development. A corpus of 13 European Portuguese words was acquired for four speakers and three of them enrolled in a second acquisition session, three months later. Regarding the speaker-dependent models, trained and tested with data from each speaker while using 5-fold cross-validation, average accuracies of 84.50% and 88.00% were respectively obtained from Bagging (BAG) and Linear Regression (LR) classifiers, respectively. Additionally, recognition accuracies of 81.79% and 81.80% were also, respectively, achieved for the session and speaker-independent experiments, establishing promising grounds for further exploring this technology towards silent speech recognition.

A comprehensive multimodal dataset for contactless lip reading and acoustic analysis

“…The result of 95% approves that the mouth motion produces informative signals for UWB sensing 7 . FMCW radar is also an optional choice which has been proven in the result of the paper 8 . The mentioned work adopts point clouds of human mouth while speaking as data feature for classification work of 13 words with 4 speakers.…”

IR-UWB Radar-Based Contactless Silent Speech Recognition of Vowels, Consonants, Words, and Phrases

“…FMCW radar utilizes continuous wave signals, whereas IR-UWB radar employs pulse signals for its functionality. Thus, the signal processing algorithm used in the FMCW radar-based SSR studies of [29], [30], and [31] cannot be directly applied to our IR-UWB radar-based SSR study.…”

“…As indicated in Table 2, with the same vowel corpus as that used in our study, the method proposed in [28] achieved an accuracy of only 51.59%, whereas our method (FERASEC + DNN-HMM) achieved a significantly higher accuracy of 86.47%. Ferreira et al [30] reported an accuracy of 88.3% for a 13-word European Portuguese classification task with four participants using an FMCW radar.…”

“…Surface electromyography [16], [17], vision [18], [19], [20], [21], [22], [23], ultrasound imaging [24], [25], and radar [26], [27], [28], [29], [30], [31] are techniques for capturing nonacoustic speech-related biosignals without the need to place sensors inside the oral cavity. Although these techniques are more convenient than the aforementioned ones, they have some shortcomings.…”

“…Ferreira et al [30] performed a speech recognition task with 13 isolated European Portuguese words using an FMCW radar. They utilized velocity dispersion data as speech features and successfully demonstrated that these features were capable of classifying distinguishable words.…”