Hand Tracking and Gesture Recognition Using Lensless Smart Sensors

Abstract

The Lensless Smart Sensor (LSS) developed by Rambus, Inc. is a low-power, low-cost visual sensing technology that captures information-rich optical data in a tiny form factor using a novel approach to optical sensing. The spiral gratings of LSS diffractive grating, coupled with sophisticated computational algorithms, allow point tracking down to millimeter-level accuracy. This work is focused on developing novel algorithms for the detection of multiple points and thereby enabling hand tracking and gesture recognition using the LSS. The algorithms are formulated based on geometrical and mathematical constraints around the placement of infrared light-emitting diodes (LEDs) on the hand. The developed techniques dynamically adapt the recognition and orientation of the hand and associated gestures. A detailed accuracy analysis for both hand tracking and gesture classification as a function of LED positions is conducted to validate the performance of the system. Our results indicate that the technology is a promising approach, as the current state-of-the-art focuses on human motion tracking that requires highly complex and expensive systems. A wearable, low-power, low-cost system could make a significant impact in this field, as it does not require complex hardware or additional sensors on the tracked segments.

A Postural Approach to the Pelvic Diameters of Obstetrics: The Dynamic External Pelvimetry Test

“…Research with optical-electronic instruments evaluates the change of the position in the space of the sensors placed on the external surface of the body [5,11,[13][14], but some studies show that external landmarks move with body movement due to the soft tissue artifact that would affect external markers located above those points during movement analysis [15]. The human hand is a very precise and reliable sensor; the hand is one of the most complex and beautiful pieces of natural engineering and it allows us to manipulate small objects with great precision [16]. Furthermore, technical instruments (MR, CT, optoelectronic devices, 3D-reconstruction software) are not available in every part of the world and cannot be used in routine obstetric practice on every single pregnant woman.…”

A Piezoresistive Array Armband With Reduced Number of Sensors for Hand Gesture Recognition

“…Such remote sensing is very attractive, but its performances are heavily influenced by many factors such as camera field of view, challenging image processing, illumination conditions, objects overlapping, etc. (Chakraborty et al, 2017;Abraham et al, 2018). Devices based on surface electromyography (sEMG or simply EMG) recordings (Geng et al, 2016;Du et al, 2017) need electrodes in steady contact with the skin, and they are prone to motion artifacts, electromagnetic noise, and crosstalk with other biopotentials.…”

A Gesture Elicitation Study of Nose-Based Gestures

“…Sensors have become so miniaturized that they can be integrated into virtually any wearable device or everyday object, such as smart watches and glasses, thus offering new forms of interaction [ 1 ]: a sensor is able to recognize human movements performed on some dedicated parts of the human body [ 2 , 3 , 4 ].…”