Capacitive sensor-based hand gesture recognition in ambient intelligence scenarios

Abstract

Input devices based on arrays of capacitive proximity sensors allow the tracking of a user's hands in three dimensions. They can be hidden behind materials such as wood, wool or plastics without limiting their functionality, making them ideal for application in Ambient Intelligence (AmI) scenarios. Most gesture recognition frameworks are targeted towards classical input devices and interpret two-dimensional data. In this work, we present a concept for adapting classical gesture recognition methods for capacitive input devices by realizing an extension of the feature set to three dimensional input data. This allows more robust gesture recognition for free-space interaction and training specific to capacitive input devices. We have implemented this concept in a prototypical setup and tested the device in various Ambient Intelligence scenarios, ranging from manipulating home appliances to controlling multimedia applications

Capacitive proximity sensing in smart environments

“…The most common positive remarks gathered in this study can be roughly put into three groups:  The device is very intuitive to use  The idea of interacting this way is novel and interesting  It is easy to control applications with those gestures Likewise we identified three main groups for negative comments about the prototype:  The device is not very precise  The interaction speed is slow  It can be tiring for the arm Later iterations have improved on some of the weaknesses by using a more sophisticated gesture recognition system and higher sensor refresh rates. Accordingly, there were fewer complaints about interaction speed and precision [77]. However, the final complaint about the device being tiring for the arm, requires a different approach, that we investigate in the final prototype presented in this work.…”

“…An example interpolation is shown in Figure 27. Figure 28 Gesture overview module (left) and gesture recorder (right) [77] An addition of the MagicBox is a generic gesture recognition module, based on methods similar to mouse gesture recognition adapted for three dimensional locations [77]. Using the developed debug software, we can definie an arbitrary set of potential gestures and adding training data, as shown in Figure 28.…”

Proximity Sensing Electronic Skin: Principles, Characteristics, and Applications

“…[ 114 ] Amid the pandemic, touchless HMI has garnered heightened interest due to its flexibility and hygiene advantages. It demonstrates substantial potential in various domains, encompassing gesture recognition [ 115 , 116 , 117 , 118 ] and touchless door locks. [ 119 ] Vision‐based gesture recognition is an effective method for non‐contact HMI.…”

3.4.1 - Voruntersuchungen zur berührungslosen Gestenerkennung mittels kapazitiver Sensorik

“…Ein Großteil der wissenschaftlichen Publikationen, die die berührungslose Erkennung von menschlichen Gesten mittels kapazitiver Sensorik behandeln, beruhen auf Machine Learning Ansätzen [1,2]. Zur Implementierung solcher Algorithmen genügen oft empirische Näherungen für den Kapazitätsverlauf [3] und ein Satz von Trainingsdaten.…”