Move With the Theremin: Body Posture and Gesture Recognition Using the Theremin in Loose-Garment With Embedded Textile Cables as Antennas

Abstract

We present a novel intelligent garment design approach for body posture/gesture detection in the form of a loose-fitting blazer prototype, “the MoCaBlazer.” The design is realized by leveraging conductive textile antennas with the capacitive sensing modality, supported by an open-source electronic theremin system (OpenTheremin). The use of soft textile antennas as the sensing element allows flexible garment design and seamless tech-garment integration for the specific structure of different clothes. Our novel approach is evaluated through two experiments involving defined movements (20 arm/torso gestures and eight dance movements). In cross-validation, the classification model yields up to 97.18% average accuracy and 92% f1-score, respectively. We have also explored real-time inference enabled by a radio frequency identification (RFID) synchronization method, yielding an f1-score of 82%. Our approach opens a new paradigm for designing motion-aware smart garments with soft conductive textiles beyond traditional approaches that rely on tight-fitting flexible sensors or rigid motion sensor accessories.

InMyFace: Inertial and mechanomyography-based sensor fusion for wearable facial activity recognition

“…Then the inertial model triggers the gesture recognition with the capacitive information (nine classes). This is similar to the approaches applied in [4] and in [2], where the Radio Frequency Identification (RFID) signal is used as a trigger to begin gesture detection, reducing power consumption and model complexity while improving accuracy.…”

“…CaptAinGlove incorporates textile capacitive electrodes as sensing channels on the fingers and an IMU sensor on the wrist. Textile capacitive sensing has demonstrated its effectiveness as a low-power consumption, cost-effective, and scalable technology for movement tracking in gesture and activity recognition [3,4,24]. Moreover, IMU sensors have been extensively used to monitor wrist movements by researchers [7,11,19].…”

Embedding textile capacitive sensing into smart wearables as a versatile solution for human motion capturing

“…The basis for including the proposed capacitive sensing functionality in a garment rests on the conductive fabrics and their characteristics. Such material can be used as the antenna to form the capacitor, which can further be applied in downstream tasks such as motion capturing or gesture recognition similar to a Theremin 20 . To connect the conductive patches with the data acquisition unit (DAU) for sensing the capacitance, conductive traces from the same material can be used to route through the garment 19 .…”

InMyFace: Inertial and Mechanomyography-Based Sensor Fusion for Wearable Facial Activity Recognition

“…Recently, capacitive sensors have been embedded in glasses in [39] to recognize 12 facial gestures and head movements. They explored two ways to deploy the electrodes; one involves injecting a 12V AC signal into the body to elevate the body's potential and increase the signal-to-noise ratio, and the second one by generating a tremendous electric field close to the person's face/eyes to remove the ground dependency in capacitive sensing [43,44] and get good signals. Another active sensing approach in glasses is presented in [22], using 17 photo-reflective sensors to classify eight facial expressions with results between 78.00-92.00% per user.…”

“…The FSR, PEF, and inertial sensors (quaternions and acceleration) data were normalized by subtracting the average of the gesture's first (starting point) and last values (ending point). Since each facial event is a temporal series with variable lengths, a dynamic resample procedure to 400 samples was applied [43,44]. Then, the resampled signals were fed to a firstdegree Butterworth low pass filter with a cut frequency of 5Hz to remove the ringing peak in the signal's edges coming from the resample procedure and to highlight the low-frequency range.…”