A Dual-Mode Human Computer Interface Combining Speech and Tongue Motion for People with Severe Disabilities

Abstract

We are presenting a new wireless and wearable human computer interface called the dual-mode Tongue Drive System (dTDS), which is designed to allow people with severe disabilities to use computers more effectively with increased speed, flexibility, usability, and independence through their tongue motion and speech. The dTDS detects users’ tongue motion using a magnetic tracer and an array of magnetic sensors embedded in a compact and ergonomic wireless headset. It also captures the users’ voice wirelessly using a small microphone embedded in the same headset. Preliminary evaluation results based on 14 able-bodied subjects and three individuals with high level spinal cord injuries at level C3–C5 indicated that the dTDS headset, combined with a commercially available speech recognition (SR) software, can provide end users with significantly higher performance than either unimodal forms based on the tongue motion or speech alone, particularly in completing tasks that require both pointing and text entry.

Two decades of assistive technologies to empower people with disability: a systematic mapping study

“…[63, [163][164][165][166][167] [ 162,163,[168][169][170] [ [171][172][173] [92, 164,167,174] [170] [175] [41, [176][177][178][179] HCI-NEURO [119] [100,161,183] [184]…”

A Novel Asynchronous Brain Signals-Based Driver–Vehicle Interface for Brain-Controlled Vehicles

“…Brain signals-based driver–vehicle interfaces (DVIs), which are responsible for translating brain signals into driving commands, are the core part of these brain-controlled vehicles. Compared to DVIs based on eye tracking [ 2 , 3 , 4 ] and speech recognition [ 5 , 6 , 7 ], brain signals-based DVIs do not need speech input and have low or no requirements in terms of neuromuscular control capabilities. Therefore, they are especially desirable for severely disabled people (or wounded soldiers on battlefields) to operate a vehicle.…”

Facile Fabrication of a Highly Sensitive and Robust Flexible Pressure Sensor with Batten Microstructures

“…Wearable sensors can monitor the status of users without interrupting and restricting their actions, so they are of great significance for various potential applications in the future [ 1 , 2 , 3 , 4 , 5 ], including real-time physiological information collection [ 6 , 7 , 8 , 9 ], human–machine interaction interfaces [ 10 , 11 , 12 ], and electronic skin [ 13 , 14 , 15 ]. In addition, these various practical applications propose various requirements for sensors.…”