A depth camera-based system to enable touch-less interaction using hand gestures

Damindarov, R. R.; Fam, C. A.; Boby, Riby Abraham; Fahim, Muhammad; Klimchik, Alexandr; Matsumaru, Takafumi

doi:10.1109/nir52917.2021.9666090

Cited by 7 publications

(3 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In particular, the data acquisition was performed using a ZED 2 depth camera (Stereolabs Inc., San Francisco, CA, USA) mounted on a UGV (Thorvald, SAGA Robotics SA, Oslo, Norway). The open-source MediaPipe framework [56] was used for object detection and tracking in conjunction with an algorithm developed for the present study. For the preprocessing of the data, the SMOTE algorithm [57] was implemented.…”

Section: Discussion and Main Conclusionmentioning

confidence: 99%

“…The predefined hand gestures, which are depicted in Figure 1a-e, are commonly known as "fist", "okay", "victory", "flat", and "rock". For the purpose of identifying the position of hands, the MediaPipe framework was implemented [54], similarly to recent studies such as [55][56][57]. MediaPipe is an open-source framework created by Google that offers various ML solutions for object detection and tracking.…”

Section: Data Acquisitionmentioning

confidence: 99%

See 1 more Smart Citation

An Integrated Real-Time Hand Gesture Recognition Framework for Human–Robot Interaction in Agriculture

et al. 2022

View full text Add to dashboard Cite

Incorporating hand gesture recognition in human–robot interaction has the potential to provide a natural way of communication, thus contributing to a more fluid collaboration toward optimizing the efficiency of the application at hand and overcoming possible challenges. A very promising field of interest is agriculture, owing to its complex and dynamic environments. The aim of this study was twofold: (a) to develop a real-time skeleton-based recognition system for five hand gestures using a depth camera and machine learning, and (b) to enable a real-time human–robot interaction framework and test it in different scenarios. For this purpose, six machine learning classifiers were tested, while the Robot Operating System (ROS) software was utilized for “translating” the gestures into five commands to be executed by the robot. Furthermore, the developed system was successfully tested in outdoor experimental sessions that included either one or two persons. In the last case, the robot, based on the recognized gesture, could distinguish which of the two workers required help, follow the “locked” person, stop, return to a target location, or “unlock” them. For the sake of safety, the robot navigated with a preset socially accepted speed while keeping a safe distance in all interactions.

show abstract

Section: Discussion and Main Conclusionmentioning

confidence: 99%

Section: Data Acquisitionmentioning

confidence: 99%

An Integrated Real-Time Hand Gesture Recognition Framework for Human–Robot Interaction in Agriculture

et al. 2022

View full text Add to dashboard Cite

show abstract

“…Since such an OMC is expensive and beyond the affordable ability of this project; the Mediapipe Pose was resorted to as a comparison reference for motion tracking. Although Damindarov et al [26] reported that the relative errors between the real size (1050 × 620 mm 2 ) and the computed size of the screen using Mediapipe Pose are 5.06% for the width and 1.11% for the height with a standard deviation in measurements within 10 mm (except for one point), these figures would just give us some idea about the performance of Mediapipe Pose.…”

Section: The Limitation Of the Studymentioning

confidence: 97%

Development of AI Algorithm for Weight Training Using Inertial Measurement Units

Lin

et al. 2022

Applied Sciences

View full text Add to dashboard Cite

Thanks to the rapid development of Wearable Fitness Trackers (WFTs) and Smartphone Pedometer Apps (SPAs), people are keeping an eye on their health through fitness and heart rate tracking; therefore, home weight training exercises have received a lot of attention lately. A multi-procedure intelligent algorithm for weight training using two inertial measurement units (IMUs) is proposed in this paper. The first procedure is for motion tracking that estimates the arm orientation and calculates the positions of the wrist and elbow. The second procedure is for posture recognition based on deep learning, which identifies the type of exercise posture. The final procedure is for exercise prescription variables, which first infers the user’s exercise state based on the results of the previous two procedures, triggers the corresponding event, and calculates the key indicators of the weight training exercise (exercise prescription variables), including exercise items, repetitions, sets, training capacity, workout capacity, training period, explosive power, etc.). This study integrates the hardware and software as a complete system. The developed smartphone App is able to receive heart rate data, to analyze the user’s exercise state, and to calculate the exercise prescription variables automatically in real-time. The dashboard in the user interface of the smartphone App can display exercise information through Unity’s Animation System (avatar) and graphics, and records are stored by the SQLite database. The designed system was proven by two types of experimental verification tests. The first type is to control a stepper motor to rotate the designed IMU and to compare the rotation angle obtained from the IMU with the rotation angle of the controlled stepper motor. The average mean absolute error of estimation for 31 repeated experiments is 1.485 degrees. The second type is to use Mediapipe Pose to calculate the position of the wrist and the angles of upper arm and forearm between the Z-axis, and these calculated data are compared with the designed system. The root-mean-square (RMS) error of positions of the wrist is 2.43 cm, and the RMS errors of two angles are 5.654 and 4.385 degrees for upper arm and forearm, respectively. For posture recognition, 12 participants were divided into training group and test group. Eighty percent and 20% of 24,963 samples of 10 participants were used for the training and validation of the LSTM model, respectively. Three-thousand-three-hundred-and-fifty-nine samples of two participants were used to evaluate the performance of the trained LSTM model. The accuracy reached 99%, and F1 score was 0.99. When compared with the other LSTM-based variants, the accuracy of one-layer LSTM presented in this paper is still promising. The exercise prescription variables provided by the presented system are helpful for weight trainers/trainees to closely keep an eye on their fitness progress and for improving their health.

show abstract

Finger Gesture Recognition using Recurrent Neural Network and Convolution Neural Network

Kizhekethottam,

Sherekar,

Thakur

et al. 2023

2023 7th International Conference on Computing, Communication, Control and Automation (ICCUBEA)

View full text Add to dashboard Cite

A depth camera-based system to enable touch-less interaction using hand gestures

Cited by 7 publications

References 14 publications

An Integrated Real-Time Hand Gesture Recognition Framework for Human–Robot Interaction in Agriculture

An Integrated Real-Time Hand Gesture Recognition Framework for Human–Robot Interaction in Agriculture

Development of AI Algorithm for Weight Training Using Inertial Measurement Units

Finger Gesture Recognition using Recurrent Neural Network and Convolution Neural Network

Contact Info

Product

Resources

About