Vowel generation for children with cerebral palsy using myocontrol of a speech synthesizer

Abstract

For children with severe cerebral palsy (CP), social and emotional interactions can be significantly limited due to impaired speech motor function. However, if it is possible to extract continuous voluntary control signals from the electromyograph (EMG) of limb muscles, then EMG may be used to drive the synthesis of intelligible speech with controllable speed, intonation and articulation. We report an important first step: the feasibility of controlling a vowel synthesizer using non-speech muscles. A classic formant-based speech synthesizer is adapted to allow the lowest two formants to be controlled by surface EMG from skeletal muscles. EMG signals are filtered using a non-linear Bayesian filtering algorithm that provides the high bandwidth and accuracy required for speech tasks. The frequencies of the first two formants determine points in a 2D plane, and vowels are targets on this plane. We focus on testing the overall feasibility of producing intelligible English vowels with myocontrol using two straightforward EMG-formant mappings. More mappings can be tested in the future to optimize the intelligibility. Vowel generation was tested on 10 healthy adults and 4 patients with dyskinetic CP. Five English vowels were generated by subjects in pseudo-random order, after only 10 min of device familiarization. The fraction of vowels correctly identified by 4 naive listeners exceeded 80% for the vowels generated by healthy adults and 57% for vowels generated by patients with CP. Our goal is a continuous “virtual voice” with personalized intonation and articulation that will restore not only the intellectual content but also the social and emotional content of speech for children and adults with severe movement disorders.

Learning and adaptation in speech production without a vocal tract

“…More recently, other studies have produced alternate modalities for adults to learn to produce speech in the absence of the vocal tract. However, these modalities either provided visual feedback to facilitate production 35 or required extensive, prolonged training to successfully synthesize speech sounds 36 .…”

Short-term effects of “Yijinjing Wohu Pushi” posture-voice therapy on speech in Parkinson’s disease after STN-DBS

“…We plotted the vector change graph on a two-dimensional plane, with the starting and ending points of the vector being the formant frequency before and after training, respectively(Fig. 7) (Niu et al, 2015). The black bold symbols represent the average formant frequencies (Hz) for U.S. English vowels produced by 45 males (Hillenbrand et al, 1995).…”

Can spatial filtering separate voluntary and involuntary components in children with dyskinetic cerebral palsy?

“…Myocontrol, the control of devices using electromyographic (EMG) signals, may be used to allow children with dyskinetic CP to control assistive devices in a flexible manner. Myoelectric signals have been used for biofeedback and functional electrical stimulation for rehabilitation [ 3 – 9 ], as well as the control of other external devices such as exoskeletons and speech synthesizers [ 10 , 11 ]. Unlike in spinal cord injury, in CP there is no disconnect between the brain and the spinal cord.…”