Human-machine interface for tele-robotic operation: mapping of tongue movements based on aural flow monitoring

2006 IEEE 12th Digital Signal Processing Workshop &Amp;amp; 4th IEEE Signal Processing Education Workshop

2006

Self Cite

Speech collected through a microphone placed in front of the mouth has been the primary source of data collection for speech recognition. However, this set-up also picks up any ambient noise present at the same time. As a result, locations which may provide shielding from surrounding noise have also been considered. This study considers an ear-insert microphone which collects speech from the ear canal to take advantage of the ear canal noise shielding properties to operate in noisy environments. Speech segmentation is achieved using short-time signal magnitude and short-time energy-entropy features. Cepstral coefficients extracted from each segmented utterance are used as input features to a back-propagation neural network for the seven isolated word recognizer implemented. Results show that a backpropagation neural network configuration may be a viable choice for this recognition task and that the best average recognition rate (94.73%) is obtained with melfrequency cepstral coefficients for a two-layer network.Index Terms -speech processing, in-ear microphone, backpropagation neural network

Section: Introductionmentioning

confidence: 89%

In-Ear Microphone Speech Data Segmentation and Recognition using Neural Networks

Bulbuller

Fargues²,

2006 IEEE 12th Digital Signal Processing Workshop &Amp;amp; 4th IEEE Signal Processing Education Workshop

2006

Self Cite

“…The system makes use of changes in air pressure or sound waves (vibrations) in the ear to characterize measured parameters. Research has shown that initiating actions, in particular movements of the tongue [3,4] and speech, produce detectable pressure waves with strength corresponding to the direction, speed and/or intensity of the action*..…”

Section: Machine Interface Systemmentioning

confidence: 99%

“…In this work, we introduce a method for detecting both tongue movement and speech, and generating a control instruction corresponding to that action that can be applied to any teleoperated or semi-autonomous robot. We have previously reported on the development of a non-intrusive tonguemovement based machine interface without the need for insertion of any device within the oral cavity [3,4]. This interface consists of tracking tongue movement by monitoring changes in airflow that occur in the ear canal.…”

Section: Introductionmentioning

confidence: 99%

A Dual-Mode Human-Machine Interface for Robotic Control based on Acoustic Sensitivity of the Aural Cavity

The First IEEE/RAS-EMBS International Conference on Biomedical Robotics and Biomechatronics, 2006. BioRob 2006.

Fargues

Gupta³

et al.

Self Cite

.Abstract -We introduce an unobtrusive sensor-based control system for human-machine interface to control robotic and rehabilitative devices. The interface is capable of directing robotic or assist devices in response to tongue movement and/or speech without insertion of any device in the vicinity of the oral cavity. The interface is centered on the unique properties of the human ear as an acoustic output device.Our work has shown that various movements within the oral cavity create unique, traceable pressure changes in the human ear, which can be measured with a simple sensor (such as a microphone) and analysed to produce commands signals, which can in turn be used to control robotic devices.In this work, we present: 1) an analysis of the sensitivity of human ear canals as acoustic output device, 2) the design of a new sensor for monitoring airflow in the aural canal, 3) pattern recognition procedures for recognition of both speech and tongue movement by monitoring aural flow across several human test subjects, and 4) a conceptual design and simulation of the machine interface system.

“…In past work [1,2] we have introduced a non-intrusive tongue-movement based machine interface without the need for insertion of any device within the oral cavity. This interface consists of tracking tongue movement by monitoring changes in airflow that occur in the ear canal.…”

Section: Introductionmentioning

confidence: 99%

“…Once recognized, said movements can in turn be used in for robotic tele-operation. While previous studies by our research team [1,2] have demonstrated the feasibility of recognizing tongue movements by monitoring flow in the aural cavity, a specific comparison of signal processing and pattern recognition strategies for this unique interface across several test subjects is necessary to begin to properly quantify the phenomenon.…”

Section: Introductionmentioning

confidence: 99%

A decision fusion classification architecture for mapping of tongue movements based on aural flow monitoring

Proceedings 2006 IEEE International Conference on Robotics and Automation, 2006. ICRA 2006.

Gupta²,

Kook³

et al.

Self Cite

A complete signal processing strategy is presented to detect and precisely recognize tongue movement by monitoring changes in airflow that occur in the ear canal. Tongue movements within the human oral cavity create unique, subtle pressure signals in the ear that can be processed to produce command signals in response to that movement. The strategy developed for the human machine interface architecture includes energy-based signal detection and segmentation to extract ear pressure signals due to tongue movements, signal normalization to decrease the trial-to-trial variations in the signals, and pairwise cross-correlation signal averaging to obtain accurate estimates from ensembles of pressure signals. A new decision fusion classification algorithm is formulated to assign the pressure signals to their respective tongue-movement classes. The complete strategy of signal detection and segmentation, estimation, and classification is tested on 4 tongue movements of 4 subjects. Through extensive experiments, it is demonstrated that the ear pressure signals due to the tongue movements are distinct and that the 4 pressure signals can be classified with over 96% classification accuracies across the 4 subjects using the decision fusion classification algorithm.I.