Moving the Lab into the Mountains: A Pilot Study of Human Activity Recognition in Unstructured Environments

Abstract

Goal: To develop and validate a field-based data collection and assessment method for human activity recognition in the mountains with variations in terrain and fatigue using a single accelerometer and a deep learning model. Methods: The protocol generated an unsupervised labelled dataset of various long-term field-based activities including run, walk, stand, lay and obstacle climb. Activity was voluntary so transitions could not be determined a priori. Terrain variations included slope, crossing rivers, obstacles and surfaces including road, gravel, clay, mud, long grass and rough track. Fatigue levels were modulated between rested to physical exhaustion. The dataset was used to train a deep learning convolutional neural network (CNN) capable of being deployed on battery powered devices. The human activity recognition results were compared to a lab-based dataset with 1,098,204 samples and six features, uniform smooth surfaces, non-fatigued supervised participants and activity labelling defined by the protocol. Results: The trail run dataset had 3,829,759 samples with five features. The repetitive activities and single instance activities required hyper parameter tuning to reach an overall accuracy 0.978 with a minimum class precision for the one-off activity (climbing gate) of 0.802. Conclusion: The experimental results showed that the CNN deep learning model performed well with terrain and fatigue variations compared to the lab equivalents (accuracy 97.8% vs. 97.7% for trail vs. lab). Significance: To the authors knowledge this study demonstrated the first successful human activity recognition (HAR) in a mountain environment. A robust and repeatable protocol was developed to generate a validated trail running dataset when there were no observers present and activity types changed on a voluntary basis across variations in terrain surface and both cognitive and physical fatigue levels.

Predicting Fatigue in Long Duration Mountain Events with a Single Sensor and Deep Learning Model

“…Time at a waypoint was determined when the subject was closest. Walk and Run activity labels were defined by cadence from vertical axis accelerometery zero crossings (100 < Walk < 150 < Run steps per minute) as described in Russel et al for human activity recognition [ 43 ]. Identification of crossing obstacles was based on geographic location and manual observation of the acceleration waveforms Figure 2 .…”

“…Gait has been shown to change physical performance with increased mental fatigue [ 9 , 16 , 40 ], goals [ 41 ], and reduced executive function [ 42 ]. Terrain has been shown to influence gait and accelerometry readings [ 43 ].…”

“…Traditional machine learning with feature extraction has been used in applications such human activity recognition [ 43 , 44 ], however this approach assumes the features of interest are known and calculatable. Deep learning uses models which automatically determine feature morphology and significance in the data which may not be observable with traditional statistics and data analysis.…”

“…Recurrent neural network (RNN) and long short-term memory (LSTM) are often cited as the preferred models for time series data [ 47 ]. Convolutional neural networks (CNN) have also been used for time series data [ 43 , 48 ] and do not suffer from the stability issues of RNNs while enabling parallel processing which is not possible with RNN type models. CNN models have shown good performance on physiological time series data for emotion classification, [ 49 ] and mental fatigue [ 50 ] using EOG, which is not generally practical in field operations with high levels of activity.…”

Unsupervised Learning for Product Use Activity Recognition: An Exploratory Study of a “Chatty Device”

“…They did not look at activities that could impact a product, like vibrations within the environment. Russell et al [ 27 ] also primarily focused on the context of human activity recognition. They set out to create and validate a field-based data collection and assessment method to aid human activity recognition in the mountains with terrain and fatigue variations.…”