Methods for the Real-World Evaluation of Fall Detection Technology: A Scoping Review

Abstract

Falls in older adults present a major growing healthcare challenge and reliable detection of falls is crucial to minimise their consequences. The majority of development and testing has used laboratory simulations. As simulations do not cover the wide range of real-world scenarios performance is poor when retested using real-world data. There has been a move from the use of simulated falls towards the use of real-world data. This review aims to assess the current methods for real-world evaluation of fall detection systems, identify their limitations and propose improved robust methods of evaluation. Twenty-two articles met the inclusion criteria and were assessed with regard to the composition of the datasets, data processing methods and the measures of performance. Real-world tests of fall detection technology are inherently challenging and it is clear the field is in its infancy. Most studies used small datasets and studies differed on how to quantify the ability to avoid false alarms and how to identify non-falls, a concept which is virtually impossible to define and standardise. To increase robustness and make results comparable, larger standardised datasets are needed containing data from a range of participant groups. Measures that depend on the definition and identification of non-falls should be avoided. Sensitivity, precision and F-measure emerged as the most suitable robust measures for evaluating the real-world performance of fall detection systems.

How We Found Our IMU: Guidelines to IMU Selection and a Comparison of Seven IMUs for Pervasive Healthcare Applications

“…Gait speed has been termed the sixth vital sign and has been reported to correlate with mortality [14,17]. Stride-to-stride variability of stride time, stride length, as well as swing time are examples of valid indicators for falls risk [18][19][20]. Recent studies showed that the gait characteristics in controlled lab settings and daily life settings are different [21,22].…”

A Study on the Application of Convolutional Neural Networks to Fall Detection Evaluated with Multiple Public Datasets

“…In any case, this issue is just another example of the existence of a global problem associated with the research on fall detection systems: the lack of commonly accepted methodology to evaluate and benchmark the new proposals on FDSs existing in the literature. This lack of consensus, which has been highlighted and discussed by different authors [70][71][72][73], does not only affect the employed datasets, but also other important operational aspects of the evaluation policy (performance metrics, availability of the code of the proposed algorithms, etc.). Thus, the problem of the proper selection (number and typology of movements, participants, etc.)…”

Data Analytics and Applications of the Wearable Sensors in Healthcare: An Overview

“…Two case studies were used to demonstrate the usefulness of this open source software. Broadley et al [34] presented a systematic review to assess existing methods of evaluating fall detection systems, identify their limitations, and propose improved evaluation methods in the literature. The search results of articles that met the inclusion criteria identified few issues, such as use of small population datasets and inconsistency for performance quantification for these systems.…”