MoCapaci: Posture and gesture detection in loose garments using textile cables as capacitive antennas

Bello, Hymalai; Zhou, Bo; Suh, Sungho; Lukowicz, Paul

doi:10.1145/3460421.3480418

Cited by 24 publications

(12 citation statements)

References 40 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Recently, capacitive sensors have been embedded in glasses in [39] to recognize 12 facial gestures and head movements. They explored two ways to deploy the electrodes; one involves injecting a 12V AC signal into the body to elevate the body's potential and increase the signal-to-noise ratio, and the second one by generating a tremendous electric field close to the person's face/eyes to remove the ground dependency in capacitive sensing [43,44] and get good signals. Another active sensing approach in glasses is presented in [22], using 17 photo-reflective sensors to classify eight facial expressions with results between 78.00-92.00% per user.…”

Section: Facial Monitoring With Wearablesmentioning

confidence: 99%

“…The FSR, PEF, and inertial sensors (quaternions and acceleration) data were normalized by subtracting the average of the gesture's first (starting point) and last values (ending point). Since each facial event is a temporal series with variable lengths, a dynamic resample procedure to 400 samples was applied [43,44]. Then, the resampled signals were fed to a firstdegree Butterworth low pass filter with a cut frequency of 5Hz to remove the ringing peak in the signal's edges coming from the resample procedure and to highlight the low-frequency range.…”

Section: Multimodal Ensemble Late Sensor Fusionmentioning

confidence: 99%

See 1 more Smart Citation

InMyFace: Inertial and Mechanomyography-Based Sensor Fusion for Wearable Facial Activity Recognition

Bello¹,

Marin²,

Suh³

et al. 2023

Preprint

View full text Add to dashboard Cite

Recognizing facial activity is a well-understood (but non-trivial) computer vision problem. However, reliable solutions require a camera with a good view of the face, which is often unavailable in wearable settings. Furthermore, in wearable applications, where systems accompany users throughout their daily activities, a permanently running camera can be problematic for privacy (and legal) reasons. This work presents an alternative solution based on the fusion of wearable inertial sensors, planar pressure sensors, and acoustic mechanomyography (muscle sounds). The sensors were placed unobtrusively in a sports cap to monitor facial muscle activities related to facial expressions. We present our integrated wearable sensor system, describe data fusion and analysis methods, and evaluate the system in an experiment with thirteen subjects from different cultural backgrounds (eight countries) and both sexes (six women and seven men). In a one-model-per-user scheme and using a late fusion approach, the system yielded an average F1 score of 85.00% for the case where all sensing modalities are combined. With a cross-user validation and a one-model-for-all-user scheme, an F1 score of 79.00% was obtained for thirteen participants (six females and seven males). Moreover, in a hybrid fusion (cross-user) approach and six classes, an average F1 score of 82.00% was obtained for eight users. The results are competitive with state-of-the-art non-camera-based solutions for a cross-user study. In addition, our unique set of participants demonstrates the inclusiveness and generalizability of the approach.

show abstract

Section: Facial Monitoring With Wearablesmentioning

confidence: 99%

Section: Multimodal Ensemble Late Sensor Fusionmentioning

confidence: 99%

InMyFace: Inertial and Mechanomyography-Based Sensor Fusion for Wearable Facial Activity Recognition

Bello¹,

Marin²,

Suh³

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…Therefore, we could argue that a reliable method for BPG recognition with loose garments remains a largely open problem. The present study explores further the simple and novel method for BPG detection in Bello et al (2021), which proposes a loose garment solution based on non-contact capacitive sensing with off-the-shelf components. In addition to our previous study in Bello et al (2021), a set of dance movements is evaluated to demonstrate the potential use case of our design as a sophisticated/elegant game controller.…”

Section: Introductionmentioning

confidence: 99%

“…The present study explores further the simple and novel method for BPG detection in Bello et al (2021), which proposes a loose garment solution based on non-contact capacitive sensing with off-the-shelf components. In addition to our previous study in Bello et al (2021), a set of dance movements is evaluated to demonstrate the potential use case of our design as a sophisticated/elegant game controller. The inspiration came from the Nintendo Wii Rayman Raving Rabbids R : TV Party-ShakeTV (Wii, 2009).…”

Section: Introductionmentioning

confidence: 99%

“…A distinct aspect of our approach is that the theremin's antennas move with the wearer's body motion, changing the signals. Our main contribution is to expand our wearable and loose-fitting solution for BPG recognition in Bello et al (2021). The prototype is based on off-theshelf components, such as a modified electronic musical instrument, the "OpenTheremin" (Gaudenz, 2016), which in conjunction with textile antennas, is embedded into a loose men's jacket.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Move With the Theremin: Body Posture and Gesture Recognition Using the Theremin in Loose-Garment With Embedded Textile Cables as Antennas

Bello

Zhou

Suh

et al. 2022

Front. Comput. Sci.

Self Cite

View full text Add to dashboard Cite

We present a novel intelligent garment design approach for body posture/gesture detection in the form of a loose-fitting blazer prototype, “the MoCaBlazer.” The design is realized by leveraging conductive textile antennas with the capacitive sensing modality, supported by an open-source electronic theremin system (OpenTheremin). The use of soft textile antennas as the sensing element allows flexible garment design and seamless tech-garment integration for the specific structure of different clothes. Our novel approach is evaluated through two experiments involving defined movements (20 arm/torso gestures and eight dance movements). In cross-validation, the classification model yields up to 97.18% average accuracy and 92% f1-score, respectively. We have also explored real-time inference enabled by a radio frequency identification (RFID) synchronization method, yielding an f1-score of 82%. Our approach opens a new paradigm for designing motion-aware smart garments with soft conductive textiles beyond traditional approaches that rely on tight-fitting flexible sensors or rigid motion sensor accessories.

show abstract

Lessons Learned in Developing Sensorised Textiles to Capture Body Shapes

García-García

Valsamakis

Münzenrieder

et al. 2022

Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering

View full text Add to dashboard Cite

MoCapaci: Posture and gesture detection in loose garments using textile cables as capacitive antennas

Cited by 24 publications

References 40 publications

InMyFace: Inertial and Mechanomyography-Based Sensor Fusion for Wearable Facial Activity Recognition

InMyFace: Inertial and Mechanomyography-Based Sensor Fusion for Wearable Facial Activity Recognition

Move With the Theremin: Body Posture and Gesture Recognition Using the Theremin in Loose-Garment With Embedded Textile Cables as Antennas

Lessons Learned in Developing Sensorised Textiles to Capture Body Shapes

Contact Info

Product

Resources

About