Towards Contactless Silent Speech Recognition Based on Detection of Active and Visible Articulators Using IR-UWB Radar

Abstract

People with hearing or speaking disabilities are deprived of the benefits of conventional speech recognition technology because it is based on acoustic signals. Recent research has focused on silent speech recognition systems that are based on the motions of a speaker’s vocal tract and articulators. Because most silent speech recognition systems use contact sensors that are very inconvenient to users or optical systems that are susceptible to environmental interference, a contactless and robust solution is hence required. Toward this objective, this paper presents a series of signal processing algorithms for a contactless silent speech recognition system using an impulse radio ultra-wide band (IR-UWB) radar. The IR-UWB radar is used to remotely and wirelessly detect motions of the lips and jaw. In order to extract the necessary features of lip and jaw motions from the received radar signals, we propose a feature extraction algorithm. The proposed algorithm noticeably improved speech recognition performance compared to the existing algorithm during our word recognition test with five speakers. We also propose a speech activity detection algorithm to automatically select speech segments from continuous input signals. Thus, speech recognition processing is performed only when speech segments are detected. Our testbed consists of commercial off-the-shelf radar products, and the proposed algorithms are readily applicable without designing specialized radar hardware for silent speech processing.

Silent Speech Recognition as an Alternative Communication Device for Persons With Laryngectomy

“…These include ultrasound [47] impulse radio ultra-wide band (IR-UWB) radar [48], and permanent magnet articulography (PMA) [50]. While these other modalities have shown promise, they either involve the use of large external sensors (ultrasound and IR-UWB) or intra-oral sensors that may not be durable in the long term (PMA).…”

Biosignal Sensors and Deep Learning-Based Speech Recognition: A Review

“…Anxiety and depression are considered psychogenic causes of voice disorder [ 1 ]. Although there are several approaches for the treatment of voice disorder, such as PMA using articulatory data captured from the lips and tongue [ 2 ] or a contactless silent speech recognition system using an impulse radio ultra-wideband (IR-UWB) radar [ 3 ], the communication options are limited for the patients whose larynx (voice box) is surgically removed following throat cancer or trauma [ 4 ]. To overcome this problem, the external device, called electrolarynx, has been developed as a form of artificial larynx communication where an electronic vibration is produced and resulted in a monotonic sound that will be formed to speech [ 4 ].…”

Silent Speech Command Word Recognition Using Stepped Frequency Continuous Wave Radar

“…Non-invasive measuring techniques are surface electromyography (sEMG) [18][19][20][21][22] , ultrasound (US) doppler 23 , US imaging 24 , video imaging 25,26 (or a combination of both 27 ) and radar-based sensing (RBS) [28][29][30][31][32][33] . For sEMG, the electrodes are placed on specific locations on the neck, face and chin (above the muscles involved in speech production 21 ), whereas for US, the ultrasound probe is either placed below the chin, facing upwards, to capture tongue movements 27 , or in front of the mouth 23 .…”

“…For sEMG, the electrodes are placed on specific locations on the neck, face and chin (above the muscles involved in speech production 21 ), whereas for US, the ultrasound probe is either placed below the chin, facing upwards, to capture tongue movements 27 , or in front of the mouth 23 . Video imaging captures the lips and for RBS, one or several antennas are placed either on the facial skin 31 or in front of the mouth, as well [28][29][30] .…”