Hand Gesture Recognition Across Various Limb Positions Using a Multimodal Sensing System Based on Self-Adaptive Data-Fusion and Convolutional Neural Networks (CNNs)

Zhang, Shen; Zhou, Hao; Tchantchane, Rayane; Alici, Gursel

doi:10.1109/jsen.2024.3389963

Cited by 4 publications

References 43 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

Advancing Human–Machine Interface (HMI) through Development of a Conductive‐Textile Based Capacitive Sensor

Tchantchane,

Zhou,

Zhang

et al. 2024

Adv Materials Technologies

View full text Add to dashboard Cite

Smart wearable sensors in human–machine interfaces (HMI) facilitate communication and interface between humans and robots, as well as among humans. Conventional wearables face significant limitations, including performance degradation under various deformations (e.g., strain and pressure), limited stretchability and flexibility, poor comfort, and breathability, complicating their integration into HMI applications. In response to these limitations, a smart, sewable, high‐precision HMI device based on a soft, textile‐based sensor with machine learning (ML)‐assisted data processing is proposed. The (Ag)‐based fabric electrode integrated into polymeric composite dielectric layer, exhibiting hysteresis error of (≤4%), a tensile strain sensitivity of gauge factor (GF) ≃ 1.03 at 100% elongation, a pressure sensitivity of ≈1.583 × 10−2 kPa−1 at pressures (<50 kPa) and excellent stability (>1000 cycles). To demonstrate its versatility in HMI, the textile‐based sensor is integrated into a glove for real‐time control of a commercially available prosthetic hand and a drone, as well as for sign language recognition, achieving 95.4% classification accuracy using the random forest (RF) classifier across 10 gestures. Beyond gesture recognition, the sensor measures a range of subtle physiological activities (>0.35 kPa), functioning as a force myograph, esophageal manometry, and in gait analysis, among other strain and pressure‐related applications, highlighting exceptional capabilities for advanced wearable systems.

show abstract