A Framework for Fall Detection Based on OpenPose Skeleton and LSTM/GRU Models

Abstract

Falling is one of the causes of accidental death of elderly people over 65 years old in Taiwan. If the fall incidents are not detected in a timely manner, it could lead to serious injury or even death of those who fell. General fall detection approaches require the users to wear sensors, which could be cumbersome for the users to put on, and misalignment of sensors could lead to erroneous readings. In this paper, we propose using computer vision and applied machine-learning algorithms to detect fall without any sensors. We applied OpenPose real-time multi-person 2D pose estimation to detect movement of a subject using two datasets of 570 × 30 frames recorded in five different rooms from eight different viewing angles. The system retrieves the locations of 25 joint points of the human body and detects human movement through detecting the joint point location changes. The system is able to effectively identify the joints of the human body as well as filtering ambient environmental noise for an improved accuracy. The use of joint points instead of images improves the training time effectively as well as eliminating the effects of traditional image-based approaches such as blurriness, light, and shadows. This paper uses single-view images to reduce equipment costs. We experimented with time series recurrent neural network, long- and short-term memory, and gated recurrent unit models to learn the changes in human joint points in continuous time. The experimental results show that the fall detection accuracy of the proposed model is 98.2%, which outperforms the baseline 88.9% with 9.3% improvement.

Anthropometric Ratios for Lower-Body Detection Based on Deep Learning and Traditional Methods

“…The results showed the high accuracy of body part detection of the human for multi-person detection. Lin et al [46] also deployed the OpenPose method to detect human movement through detecting the keypoints of human joint changes. They applied the series recurrent neural network, long-and short-term memory (LSTM), and gated recurrent unit (GRU) models to detect a human fall.…”

Toward Modeling Psychomotor Performance in Karate Combats Using Computer Vision Pose Estimation

“…From our own review, we conclude that OpenPose is the one that has generated more literature works and seems to have the largest community of developers. OpenPose has been used in multiple areas: sports [87], telerehabilitation [88], HAR [89][90][91], artistic disciplines [92], identification of multi-person groups [93], and VR [94]. Thus, we have selected OpenPose algorithm for this work due to the following reasons: (i) it is open source, (ii) it can be applied in real situations with new video inputs [84], (iii) there is a large number of projects available with code and examples, (iv) it is widely reported in scientific papers, (v) there is a strong developers community, and (vi) the API gives users the flexibility of selecting source images from camera fields, webcams, and others.…”

Optimal Training Configurations of a CNN-LSTM-Based Tracker for a Fall Frame Detection System

“…Previous studies have revealed that an RNN has been implemented in the development of fall classification systems. The work by Lin et al [ 21 ] employed the RNN architecture using LSTM as the underlying model to capture temporal relationships of the body joints, which were pre-extracted using OpenPose. LSTM classifies the fall features from the non-fall features based on the change trajectory of the extracted body joint points.…”

“…Recent research trends have grouped fall event detection techniques based on the type of sensor. The three categories are as follows: wearable devices [12][13][14][15][16], ambientbased devices [17][18][19], and vision-based sensors [20][21][22][23]. Both wearable and ambient-based devices require the sensors to be in close proximity to the targeted users.…”