Optimization of Tongue Gesture Processing Algorithm for Standalone Multimodal Tongue Drive System

Sahadat, Nazmus; Sebkhi, Nordine; Anderson, David V.; Ghovanloo, Maysam

doi:10.1109/jsen.2018.2887257

“…Each tongue position is mapped to a specific command for the target application. More details about the data processing for tongue-generated commands can be found in [15]. An inertial measure unit (IMU) is embedded in the control unit mounted on the top of the headset which captures head movements using an accelerometer and a gyroscope.…”

Section: A Mtds: System Overviewmentioning

confidence: 99%

Evaluation of Multimodal Tongue Drive System by People with Tetraplegia for Computer Access

Sebkhi¹,

Sahadat²,

Walling³

et al. 2021

Preprint

Self Cite

0

View full text Add to dashboard Cite

The multimodal Tongue Drive System (mTDS) is an assistive technology for people with tetraplegia that provides an alternative method to interact with a computer by combining tongue control, head gesture, and speech. This multimodality is designed to facilitate the completion of complex computer tasks (e.g. drag-and-drop) that cannot be easily performed by existing uni-modal assistive technologies. Previous studies with able-bodied participants showed promising performance of the mTDS on complex tasks when compared to other input methods such as keyboard and mouse. In this three-session pilot study, the primary objective is to show the feasibility of using mTDS to facilitate human-computer interactions by asking fourteen participants with tetraplegia to complete five computer access tasks with increased level of complexity: maze navigation, center-out tapping, playing bubble shooter and peg solitaire, and sending an email. Speed and accuracy are quantified by key metrics that are found to be generally increasing from the first to third session, indicating the potential existence of a learning phase that could result in improved performance over time.

show abstract

“…2. mTDS headset composed of four magnetometers to capture the tongue tracer's magnetic field, a microphone to record speech, and a control unit to measure head motion and transmit processed data wirelessly using bluetooth [15]. Fig.…”

Section: A Mtds: System Overviewmentioning

confidence: 99%

“…1. The standalone multimodal tongue drive system (mTDS) converts tongue motion, head gesture, and speech, into commands that are transmitted wirelessly to control many devices such as computers, smartphones, powered wheelchair (PWC), and smart-home devices [15] . powered wheelchair.…”

Section: Introductionmentioning

confidence: 99%

“…Thus, the efficacy of an AT can be significantly improved by leveraging multiple remaining abilities. The multimodal Tongue Drive System (mTDS) is an example of such AT in which tongue movements are translated into discrete commands, head movement into proportional input, and speech is used for typing [15]- [18]. By combining these modalities, complex tasks can be performed with more ease and, as illustrated in Fig.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Evaluation of Multimodal Tongue Drive System by People with Tetraplegia for Computer Access

Sebkhi¹,

Sahadat²,

Walling³

et al. 2021

Preprint

Self Cite

1

0

View full text Add to dashboard Cite

The multimodal Tongue Drive System (mTDS) is an assistive technology for people with tetraplegia that provides an alternative method to interact with a computer by combining tongue control, head gesture, and speech. This multimodality is designed to facilitate the completion of complex computer tasks (e.g. drag-and-drop) that cannot be easily performed by existing uni-modal assistive technologies. Previous studies with able-bodied participants showed promising performance of the mTDS on complex tasks when compared to other input methods such as keyboard and mouse. In this three-session pilot study, the primary objective is to show the feasibility of using mTDS to facilitate human-computer interactions by asking fourteen participants with tetraplegia to complete five computer access tasks with increased level of complexity: maze navigation, center-out tapping, playing bubble shooter and peg solitaire, and sending an email. Speed and accuracy are quantified by key metrics that are found to be generally increasing from the first to third session, indicating the potential existence of a learning phase that could result in improved performance over time.

show abstract

“…[12] for potentially environment control. Compared with other non-contact tracking technologies [13], such as optical sensing or ultrasonic sensing, the noninvasive magnetic tracking begins to attract the interests of researchers in medicine applications recently [14,15], taking advantages of similar magnetic permeability of human body with the free space and high safety without side-effects or biomechanical reaction.…”

Section: Introductionmentioning

confidence: 99%

Passive Magnetic Localization Based on Connotative Pre-Calibration for Tongue-Machine-Interface

Shen

¹

,

Ge

²

,

Lian

³

et al. 2020

View full text Add to dashboard Cite

In human-machine-interface study, recognition of conscious human motion by contactless passive magnetic marked method can provide abundant information, such as PM marked tongue-machineinterface (TMI). However, solution of this nonlinear magnetic inverse problem heavily relies on initialization. This paper takes advantage of the motion characteristics constrained by physiological structure, and develops an enhanced algorithm for real-time full-pose passive magnetic localization in TMI application. In the proposed algorithm, a predetermined-discretized database is introduced, and provides reliable initializations for the nonlinear magnetic localization problem. This database presents the connotative relationships between the location and orientation of passive magnetic source, termed location-orientation mapping database (LOMD). The influence of initialization and LOMD constitution on solving the nonlinear inverse magnetic problem are studied through extensive simulations. Test bench evaluations are conducted on the designed experiment system. The experiment results show that about 38.5 % localization improvement can be achieved with a real-time tracking frequency up to 56.5 Hz.

show abstract

Optimization of Tongue Gesture Processing Algorithm for Standalone Multimodal Tongue Drive System

Cited by 18 publications

References 28 publications

Evaluation of Multimodal Tongue Drive System by People with Tetraplegia for Computer Access

Evaluation of Multimodal Tongue Drive System by People with Tetraplegia for Computer Access

Evaluation of Multimodal Tongue Drive System by People with Tetraplegia for Computer Access

Passive Magnetic Localization Based on Connotative Pre-Calibration for Tongue-Machine-Interface

Contact Info

Product

Resources

About