Predicting Performance Improvement of Human Activity Recognition Model by Additional Data Collection

Abstract

The development of a machine-learning-based human activity recognition (HAR) system using body-worn sensors is mainly composed of three phases: data collection, model training, and evaluation. During data collection, the HAR developer collects labeled data from participants wearing inertial sensors. In the model training phase, the developer trains the HAR model on the collected training data. In the evaluation phase, the developer evaluates the trained HAR model on the collected test data. When the HAR model cannot achieve the target recognition accuracy, the developer iterates the above procedures by taking certain measures, including collecting additional training data, until the re-trained model achieves the target accuracy. However, collecting labeled data for HAR requires additional time and incurs high monetary costs. In addition, it is difficult to determine the amount and type of data to collect for achieving the target accuracy while reducing costs. To address this issue, this paper proposes a new method that predicts the performance improvement of the current HAR model, i.e., it determines the level of performance improvement achievable by re-training the HAR model with additional data, before collecting the additional data. Thus, the method enables the HAR developer to establish a strategy for additional data collection by providing advice such as "If labeled data for the Walking and Running activities from two additional participants is collected, the HAR accuracy of the current HAR model for Walking will improve by 20%." To achieve this, a neural network called AIP-Net is proposed to estimate the improvement in performance by analyzing the feature space of the current HAR model using the proposed entropy-based attention mechanism. The performance of AIP-Net was evaluated on eight HAR datasets using leave-one-dataset-out cross-validation.

HARE: Unifying the Human Activity Recognition Engineering Workflow

“…Data collection and labeling in sensor-based HAR are significant challenges [19]. Existing research studies, such as those conducted in [20], have attempted to address these issues, highlighting the impact of additional data on performance and proposing methods to predict the performance improvement of HAR models after collecting additional data. However, these methods often encounter limitations when synchronizing data from multiple standalone IMU devices or collecting video recordings alongside IMU data for relabeling and quality assurance.…”

Dynamic and Distributed Intelligence over Smart Devices, Internet of Things Edges, and Cloud Computing for Human Activity Recognition Using Wearable Sensors

“…One application of the IoT is human activity recognition (HAR), where smart devices can monitor and recognize human activities for various purposes. HAR is essential in a number of industries, including the sport [3,4], healthcare [5][6][7], and smart environment industries [8][9][10][11]; information about human activities has been collected using smartphones and wearable sensor technologies [12][13][14][15][16].…”

“…Smartphone sensors have been widely used in machine learning; they have been used to recognize different categories of daily life activities. In recent research, several papers used smartphone sensors and focused on enhancing prediction accuracy and optimizing algorithms to speed up processing [14,15,32].…”