Updating the Silent Speech Challenge benchmark with deep learning

Abstract

The 2010 Silent Speech Challenge benchmark is updated with new results obtained in a Deep Learning strategy, using the same input features and decoding strategy as in the original article. A Word Error Rate of 6.4% is obtained, compared to the published value of 17.4%. Additional results comparing new auto-encoder-based features with the original features at reduced dimensionality, as well as decoding scenarios on two different language models, are also presented. The Silent Speech Challenge archive has been updated to contain both the original and the new auto-encoder features, in addition to the original raw data. Index Terms-silent speech interface, multimodal speech recognition, deep learning, language model INTRODUCTION 1.1.Silent speech interfaces and challengesA Silent Speech Interface, or SSI, is defined as a device enabling speech processing in the absence of an exploitable audio signal -for example, speech recognition obtained exclusively from video images of the mouth, or from electromyographic sensors (EMA) glued to the tongue. Classic applications targeted by SSIs include: 1)Voice-replacement for persons who have lost the ability to vocalize through illness or an accident, yet who retain the ability to articulate;2) Speech communication in environments where silence is either necessary or desired: responding to cellphone in meetings or public places without disturbing others; avoiding interference in call centers, conferences and classrooms; private communications by police, military, or business personnel. 7)Cortical implants for a "thought-driven" SSI. Figure 1: Overview of an SSI, showing non-acoustic sensors and non-acoustic automatic speech recognition, ASR, which can be followed by speech synthesis, or retained as a phonetic, text, or other digital representation, depending on the application.As a non-acoustic technology, SSIs initially stood somewhat apart from the main body of speech processing, where the standard techniques are intrinsically associated with an audio signal. Nevertheless, the novelty of the SSI concept and their exciting range of applications -perhaps aided by an accrued interest in multi-modal speech processing -are gradually allowing SSI technology to join the speech processing main stream. Activity in SSI research has remained strong since the publication of [1], which received the ISCA/Eurasip Best Paper Award in 2015. A recent survey of the literature reveals dozens of publications on SSI systems, using not only on the original seven non-acoustic technologies mentioned above, but also two additional ones, namely, low frequency air-borne ultrasound; and micropower radar .Despite this activity, SSIs today remain for the most part specialized laboratory instruments. The performance of any automatic speech recognition (ASR) system is most often characterized by a WordError Rate, or WER, expressed as a percentage of the total number of words appearing in a corpus. To date, no SSI ASR system has been able to achieve WER parity with state-of-the-art acoustic ASR.Indeed, a numbe...

Biosignal Sensors and Deep Learning-Based Speech Recognition: A Review

“…As the area of speech technology such as speech recognition and speech synthesis using deep learning has become wider, recent studies are attempting to solve the issue of articulatory-to-acoustic conversion [ 76 ]. In implementing SSI or silent speech recognition (SSR) technologies, such as sensor handling, interference, and feature extraction, using deep learning are also increasing to improve recognition performance [ 7 ]. Recently, DNN has been conducted more frequently than traditional systems, such as Gaussian mixture model (GMM) in speech recognition research, and CNN is also widely used because it proved to be effective in recognizing patterns in the speech signal and image processing [ 7 ].…”

“…Thus, a novel concept must be developed for voice recognition and production technologies which also can include brain-computer interfaces (BCIs) and silent-speech interfaces (SSIs). SSI is considered as a plausible approach to producing natural-sounding speech by capturing biosignals from the articulators, neural pathways, or the brain itself in brain-computer interfaces (BCIs) [ 5 , 6 , 7 , 8 ]. Recently, various biosignals captured by techniques such as ultrasound, optical imagery, EPG, EEG, and surface electromyography (sEMG) have been investigated in terms of developing silent speech communication systems [ 8 , 9 , 10 ].…”

Silent Speech Interfaces for Speech Restoration: A Review

“…This model achieved a recognition accuracy of 80.4% when tested over the database developed in [106], which validated it for visual speech recognition. Deep autoencoders were used in [273], [274] to extract features from ultrasound images, achieving significant gains in both silent ASR and direct synthesis. In [275], multitask learning of speech recognition and synthesis parameters was evaluated in the context of an ultrasoundbased SSI system designed to enhance the performance of individual tasks.…”

SMO-DNN: Spider Monkey Optimization and Deep Neural Network Hybrid Classifier Model for Intrusion Detection

“…The spider monkey optimization algorithm (SMO) is one of the metaheuristic methods [41,[44][45][46]] based on the spider monkey's social behavior, adopting the fission and fusion swarm intelligence tactic for foraging [47]. Spider monkeys usually live in a swarm of 40 to 50 members.…”