Towards a Multimodal Silent Speech Interface for European Portuguese

Freitas, João; Teixeira, António; Dias, Miguel Sales; Bastos, Carlos A. C.

doi:10.5772/16935

Cited by 14 publications

(9 citation statements)

References 51 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Investigations regarding the EMG-based approach include the application of electromyography in special circumstances, e.g. for firefighters who may be prevented from speaking because they wear a breathing apparatus [7], recognition of disordered speech [8], language-dependent challenges [9], and within the context of our array-based recording setup, signal source decomposition to retrace signal sources and remove artifacts [10].…”

Section: Related Workmentioning

confidence: 99%

Compensation of recording position shifts for a myoelectric Silent Speech Recognizer

Wand

Schulte

Janke

et al. 2014

2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

A myoelectric Silent Speech Recognizer is a system which recognizes speech by capturing the electrical activity of the human articulatory muscles, thus enabling the user to communicate silently. We recently devised a recording setup based on electrode arrays with multiple measuring points. In this study we show that this allows to compensate for shifts of the recording position, which happen when the array is removed and reattached between system training and application. We present a method which determines the amount of recording position shift; compensation is performed by linear interpolation. We evaluate our method by running recognition experiments across recording sessions and obtain a Word Error Rate improvement of 14.3% relative on the development set and 12.9% relative on the evaluation set, compared to using classical session adaptation.

show abstract

Section: Related Workmentioning

confidence: 99%

Compensation of recording position shifts for a myoelectric Silent Speech Recognizer

Wand

Schulte

Janke

et al. 2014

2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

show abstract

“…The use of silent speech interfaces (SSIs) (Denby et al, 2010) provides an alternative solution to the conventional methods by enabling oral communication in the absence of audible speech by exploiting other non-audible signals generated during speech, such as electrical activity in the brain (Herff et al, 2015) or in the articulator muscles (Jou et al, 2006;Schultz and Wand, 2010;Wand et al, 2014) or the movement of the speech articulators themselves (Petajan, 1984;Toda et al, 2008;Denby et al, 2010;Hueber et al, 2010;Gilbert et al, 2010;Freitas et al, 2011;Hofe et al, 2013). Because of this unique feature, SSIs can be suitable for applications other than post-laryngectomy voice rehabilitation, such as communication in noisy environments or in situations where privacy/confidentiality is important.…”

Section: Introductionmentioning

confidence: 99%

Direct Speech Generation for a Silent Speech Interface based on Permanent Magnet Articulography

González

Cheah

Gilbert

et al. 2016

Proceedings of the 9th International Joint Conference on Biomedical Engineering Systems and Technologies

View full text Add to dashboard Cite

Patients with larynx cancer often lose their voice following total laryngectomy. Current methods for postlaryngectomy voice restoration are all unsatisfactory due to different reasons: requires frequent replacement due to biofilm growth (tracheo-oesoephageal valve), speech sounds gruff and masculine (oesophageal speech) or robotic (electro-larynx) and, in general, are difficult to master (oesophageal speech and electro-larynx). In this work we investigate an alternative approach for voice restoration in which speech articulator movement is converted into audible speech using a speaker-dependent transformation learned from simultaneous recordings of articulatory and audio signals. To capture articulator movement, small magnets are attached to the speech articulators and the magnetic field generated while the user 'mouths' words is captured by a set of sensors. Parallel data comprising articulatory and acoustic signals recorded before laryngectomy are used to learn the mapping between the articulatory and acoustic domains, which is represented in this work as a mixture of factor analysers. After laryngectomy, the learned transformation is used to restore the patient's voice by transforming the captured articulator movement into an audible speech signal. Results reported for normal speakers show that the proposed system is very promising.

show abstract

“…This method allows speech to be recognized even when it is produced silently, i. e. mouthed without any vocal effort, as demonstrated by (Jorgensen et al, 2003). Most current approaches to EMG based speech recognition use single electrodes for the recording of facial electromyographic signals (Deng et al, 2012), (Jorgensen and Dusan, 2010), (Freitas et al, 2012). However our EMG acquisition system uses electrode arrays, as used by (Wand et al, 2013b) for continuous speech recognition and by (Kubo et al, 2013) for vowel discrimination.…”

Section: Introductionmentioning

confidence: 99%

Spatial Artifact Detection for Multi-channel EMG-based Speech Recognition

Heistermann

Janke

Wand

et al. 2014

Proceedings of the International Conference on Bio-Inspired Systems and Signal Processing

View full text Add to dashboard Cite

We introduce a spatial artifact detection method for a surface electromyography (EMG) based speech recognition system. The EMG signals are recorded using grid-shaped electrode arrays affixed to the speakers face. Continuous speech recognition is performed on the basis of these signals. As the EMG data are highdimensional, Independent Component Analysis (ICA) can be applied to separate artifact components from the content-bearing signal. The proposed artifact detection method classifies the ICA components by their spatial shape, which is analyzed using the spectra of the spatial patterns of the independent components. Components identified as artifacts can then be removed. Our artifact detection method reduces the word error rates (WER) of the recognizer significantly. We observe a slight advantage in terms of WER over the temporal signal based artifact detection method by (Wand et al., 2013a).

show abstract

Towards a Multimodal Silent Speech Interface for European Portuguese

Cited by 14 publications

References 51 publications

Compensation of recording position shifts for a myoelectric Silent Speech Recognizer

Compensation of recording position shifts for a myoelectric Silent Speech Recognizer

Direct Speech Generation for a Silent Speech Interface based on Permanent Magnet Articulography

Spatial Artifact Detection for Multi-channel EMG-based Speech Recognition

Contact Info

Product

Resources

About