A Fall Detection Network by 2D/3D Spatio-temporal Joint Models with Tensor Compression on Edge

Abstract

Falling is ranked highly among the threats in elderly healthcare, which promotes the development of automatic fall detection systems with extensive concern. With the fast development of the Internet of Things (IoT) and Artificial Intelligence (AI), camera vision-based solutions have drawn much attention for single-frame prediction and video understanding on fall detection in the elderly by using Convolutional Neural Network (CNN) and 3D-CNN, respectively. However, these methods hardly supervise the intermediate features with good accurate and efficient performance on edge devices, which makes the system difficult to be applied in practice. This work introduces a fast and lightweight video fall detection network based on a spatio-temporal joint-point model to overcome these hurdles. Instead of detecting fall motion by the traditional CNNs, we propose a Long Short-Term Memory (LSTM) model based on time-series joint-point features extracted from a pose extractor . We also introduce the increasingly mature RGB-D camera and propose 3d pose estimation network to further improve the accuracy of the system. We propose to apply tensor train decomposition on the model to reduce storage and computational consumption, so that the deployment on edge devices can to realized. Experiments are conducted to verify the proposed framework. For fall detection task, the proposed video fall detection framework achieves a high sensitivity of \(98.46\% \) on Multiple Cameras Fall, \(100\% \) on UR Fall and \(98.01\% \) on NTU RGB-D 120. For pose estimation task, our 2D model attains 73.3 mAP in the COCO keypoint challenge, which outperforms the OpenPose by \(8\% \) . Our 3D model attains \(78.6\% \) mAP on NTU RGB-D dataset with 3.6 × faster speed than OpenPose.

AI-Driven Privacy in Elderly Care: Developing a Comprehensive Solution for Camera-Based Monitoring of Older Adults

“…Fall detection models rely heavily on object detection or pose estimation models. These models typically utilize a combination of spatial and temporal data processed through advanced machine learning algorithms like CNNs and Long Short-Term Memory networks (LSTMs) [26,27]. The spatial aspect involves analyzing the posture and position of an individual, while the temporal component examines movement patterns over time.…”

“…However, while sensors and wearables offer a less intrusive means of observation, they may not always provide the comprehensive visual context necessary for certain caregiving tasks. Furthermore, several vision-based monitoring systems [10,26,33,50,51] have proven to be more capable of detecting emergency situations, such as falls, making them more popular among caregivers and less inconvenient for elderly people than wearables.…”