A silent speech system based on permanent magnet articulography and direct synthesis

González, José A.; Cheah, Lam Aun; Gilbert, James M.; Bai, Jie; Ell, Stephen R.; Green, Phil; Moore, Roger K.

doi:10.1016/j.csl.2016.02.002

Cited by 51 publications

(47 citation statements)

References 41 publications

(60 reference statements)

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Apart from that, another reason the MCD results are worse on the Arctic database is the limitations of the current PMA device for modelling some areas of the vocal tract (e.g. sounds articulated at the back of the mouth) [20,21], as discussed below. Since the Arctic sentences contain more phones articulated in those areas than the digits vocabulary, this harms the overall reconstruction performance achieved on the Arctic database.…”

Section: Results On the Tidigits Databasementioning

confidence: 99%

“…When considering the place of articulation, it can seen that sounds articulated at the middle and back of the mouth (palatal, velar, and glottal consonants) are, on average, less well reconstructed than other phones. This is due to the limitations of the current PMA prototype for modelling those areas of the vocal tract [20,21].…”

Section: Results On the Arctic Databasementioning

confidence: 99%

“…Results reported for isolated-digit synthesis showed the superiority of this approach for modelling the PMA-to-acoustic mapping. Finally, in [21], we proposed a conversion system based on mixture of factor analysers, similar to the one described in this work, and showed the viability of voice reconstruction from PMA data for continuous speech.…”

Section: Related Workmentioning

confidence: 93%

“…Many different SSIs have been proposed so far, mainly differing in the type of biosignal they rely on. Thus, we can find SSIs that exploit the electrical signals generated by the neurons in the brain [23] or in the articulator muscles [31,42,49] or the movement of the speech articulators themselves [40,44,9,29,18,14,26,21]. In our work we use a magnetic sensing technique known as Permanent Magnet Articulography (PMA) [13,18] for capturing the movement of the speech articulators.…”

Section: Introductionmentioning

confidence: 99%

“…This approach, however, is not exempt from problems [21]. Firstly, due to the variable delay introduced by the ASR and TTS systems, speech articulation and the corresponding acoustic feedback produced by the SSI are disconnected (i.e.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Voice Restoration After Laryngectomy Based on Magnetic Sensing of Articulator Movement and Statistical Articulation-to-Speech Conversion

González

Cheah

Gilbert

et al. 2017

Biomedical Engineering Systems and Technologies

Self Cite

View full text Add to dashboard Cite

ReuseUnless indicated otherwise, fulltext items are protected by copyright with all rights reserved. The copyright exception in section 29 of the Copyright, Designs and Patents Act 1988 allows the making of a single copy solely for the purpose of non-commercial research or private study within the limits of fair dealing. The publisher or other rights-holder may allow further reproduction and re-use of this version -refer to the White Rose Research Online record for this item. Where records identify the publisher as the copyright holder, users can verify any specific terms of use on the publisher's website. TakedownIf you consider content in White Rose Research Online to be in breach of UK law, please notify us by emailing eprints@whiterose.ac.uk including the URL of the record and the reason for the withdrawal request. Abstract. In this work, we present a silent speech system that is able to generate audible speech from captured movement of speech articulators. Our goal is to help laryngectomy patients, i.e. patients who have lost the ability to speak following surgical removal of the larynx most frequently due to cancer, to recover their voice. In our system, we use a magnetic sensing technique known as Permanent Magnet Articulography (PMA) to capture the movement of the lips and tongue by attaching small magnets to the articulators and monitoring the magnetic field changes with sensors close to the mouth. The captured sensor data is then transformed into a sequence of speech parameter vectors from which a time-domain speech signal is finally synthesised. The key component of our system is a parametric transformation which represents the PMA-to-speech mapping. Here, this transformation takes the form of a statistical model (a mixture of factor analysers, more specifically) whose parameters are learned from simultaneous recordings of PMA and speech signals acquired before laryngectomy. To evaluate the performance of our system on voice reconstruction, we recorded two PMA-and-speech databases with different phonetic complexity for several non-impaired subjects. Results show that our system is able to synthesise speech that sounds as the original voice of the subject and also is intelligible. However, more work still need to be done to achieve a consistent synthesis for phonetically-rich vocabularies.

show abstract

Section: Results On the Tidigits Databasementioning

confidence: 99%

Section: Results On the Arctic Databasementioning

confidence: 99%

Section: Related Workmentioning

confidence: 93%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Voice Restoration After Laryngectomy Based on Magnetic Sensing of Articulator Movement and Statistical Articulation-to-Speech Conversion

González

Cheah

Gilbert

et al. 2017

Biomedical Engineering Systems and Technologies

Self Cite

View full text Add to dashboard Cite

show abstract

Flexible Throat Patch for Phonetic Recognition and Ingestion Monitoring via Sensory Transition based on Machine Learning

Hong

2024

Adv Materials Technologies

View full text Add to dashboard Cite

The throat is an organ that includes the larynx, esophagus, and bronchi. Vocalization or deglutition triggers micro‐movements of muscles around the throat. In this study, based on machine learning using a flexible patch, pronunciation is identified by analyzing the vibration and movement of the laryngeal muscles, and the type of food consumed is determined through peristalsis analysis. Previously, voices are recognized through cochlea hair cells, and the type of food intake is identified through taste receptors. However, this study converted auditory or gustatory senses to tactile stage through sensory transition and finally visualized. Using a customized throat patch with segmented nodes, the pronunciation of [a], [e], [i], [o], and [u], representative vowels of languages worldwide, are distinguished with an accuracy of 85.9% based on the polynomial kernel algorithm. In addition, cookies, water, gummies, and chocolate are distinguished with 91.1% accuracy by detecting differences in the esophagus's peristalsis. Transformation of auditory or gustatory sensations to tactile information and visualization is expected to be used for communication in noisy environments or soundless environments such as outer space. This approach can also provide significant potential for determining the type of ingestion for users who have lost their sense of taste.

show abstract

Introduction

Freitas¹,

Teixeira²,

Dias³

et al. 2016

An Introduction to Silent Speech Interfaces

View full text Add to dashboard Cite

A silent speech system based on permanent magnet articulography and direct synthesis

Cited by 51 publications

References 41 publications

Voice Restoration After Laryngectomy Based on Magnetic Sensing of Articulator Movement and Statistical Articulation-to-Speech Conversion

Voice Restoration After Laryngectomy Based on Magnetic Sensing of Articulator Movement and Statistical Articulation-to-Speech Conversion

Flexible Throat Patch for Phonetic Recognition and Ingestion Monitoring via Sensory Transition based on Machine Learning

Introduction

Contact Info

Product

Resources

About