Human activity recognition based on smartphone and wearable sensors using multiscale DCNN ensemble

Sena, Jessica; Barreto, Jesimon; Caetano, Carlos; Cramer, Guilherme; Schwartz, William Robson

doi:10.1016/j.neucom.2020.04.151

Cited by 61 publications

(29 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Depending on the specifics of eccentric motions in, e.g., gym exercises (as we study them in this paper) these durations can get extended to 1-3 s [78,79]. Modeling such variations of motion lengths is important for deriving effective feature representations, be it explicitly through feature engineering or implicitly through end-to-end learning [80,81]. Our model explicitly captures varying lengths of movement signals by utilizing multi-length kernel windows (multi-window) for the convolution operation, i.e., feature extraction.…”

Section: Multiple Kernel Window Size For Capturing Varying Motion Lengthmentioning

confidence: 99%

Complex Deep Neural Networks from Large Scale Virtual IMU Data for Effective Human Activity Recognition Using Wearables

Kwon

Abowd

Plötz

2021

Sensors

View full text Add to dashboard Cite

Supervised training of human activity recognition (HAR) systems based on body-worn inertial measurement units (IMUs) is often constrained by the typically rather small amounts of labeled sample data. Systems like IMUTube have been introduced that employ cross-modality transfer approaches to convert videos of activities of interest into virtual IMU data. We demonstrate for the first time how such large-scale virtual IMU datasets can be used to train HAR systems that are substantially more complex than the state-of-the-art. Complexity is thereby represented by the number of model parameters that can be trained robustly. Our models contain components that are dedicated to capture the essentials of IMU data as they are of relevance for activity recognition, which increased the number of trainable parameters by a factor of 1100 compared to state-of-the-art model architectures. We evaluate the new model architecture on the challenging task of analyzing free-weight gym exercises, specifically on classifying 13 dumbbell execises. We have collected around 41 h of virtual IMU data using IMUTube from exercise videos available from YouTube. The proposed model is trained with the large amount of virtual IMU data and calibrated with a mere 36 min of real IMU data. The trained model was evaluated on a real IMU dataset and we demonstrate the substantial performance improvements of 20% absolute F1 score compared to the state-of-the-art convolutional models in HAR.

show abstract

Section: Multiple Kernel Window Size For Capturing Varying Motion Lengthmentioning

confidence: 99%

Complex Deep Neural Networks from Large Scale Virtual IMU Data for Effective Human Activity Recognition Using Wearables

Kwon

Abowd

Plötz

2021

Sensors

View full text Add to dashboard Cite

show abstract

“…Teng et al [15] proposed a network based on convolutional neurons with a local loss after each CNN module, they compared a baseline model containing three CNN layers and one Fully Connected layer, with the same model having the first time similarity matching loss, a second time crossentropy loss and the third time a combination between the two previous losses. Sena et al [16] divided the data into several inputs according to the type of sensor, then for each of them they built a deep CNN to extract temporal scales and features. Their method employs a DCNN, which is made up of two convolutional layers followed by a Maxpooling layer.…”

Section: Review Of Literature IImentioning

confidence: 99%

A Hybrid Deep Neural Network for Human Activity Recognition based on IoT Sensors

Benhaili¹,

Balouki²,

Moumoun³

2021

IJACSA

View full text Add to dashboard Cite

Internet of things (IOT) sensors, has received a lot of interest in recent years due to the rise of application demands in domains like ubiquitous and context-aware computing, activity surveillance, ambient assistive living and more specifically in Human activity recognition. The recent development in deep learning allows to extract high-level features automatically, and eliminates the reliance on traditional machine learning techniques, which depended heavily on hand crafted features. In this paper, we introduce a network that can identify a variety of everyday human actions that can be carried out in a smart home environment, by using raw signals generated from Internet of Thing's motion sensors. We design our architecture basing on a combination of convolutional neural network (CNN) and Gated recurrent unit (GRU) layers. The CNN is first deployed to extract local and scale-invariance features, then the GRU layers are used to extract sequential temporal dependencies. We tested our model called (CNGRU) on three public datasets. It achieves an accuracy better or comparable to existing state of the art models.

show abstract

“…Perangkat pintar yang telah digunakan seperti smartphone [9], smartwatch, dan smartglasses [21]. Dari beberapa sensor yang ada pada perangkat pintar, akselerometer [9], giroskop [4] dan magnometer [14] merupakan sensor yang telah digunakan untuk pengambilan data HAR.…”

Section: Metode Penelitianunclassified

“…Tabel 3 menunjukkan fitur-fitur yang sering digunakan dalam konteks HAR menggunakan sensor inersia. Meskipun kedua jenis fitur tersebut memiliki informasi yang berbeda, beberapa penelitian menggunakan hanya salah satu domain saja untuk mengurangi kompleksitas dan pertimbangan komputasi [14][53] [54]. Di sisi lain, penggunaan kedua domain tersebut secara bersamaan memiliki nilai lebih karena menangkap informasi dari berbagai sudut pandang data.…”

Section: De-noisingunclassified

“…al. [3] [44] Multi-layer Perceptron (MLP) [7], [9], [20] Long-Short Term Memory (LSTM) [10], [55], [51] Deep Neural Network (DNN) [42] Deep Convolution Neural Networks (DCNN) [14], [46] Extreme Learning Machine (ELM) [60] Machine Learning Support Vector Machine (SVM) [8], [13], [14], [15], [44], [43], [48], [20], [58], [21], [41], [53], [61], [62], [51] Decision Tree (DT) [4], [63], [8], [43], [58], [21], [51] Genetic Algorithm (GA) [43] Artificial Neural Network (ANN) [4], [15], [44] Hidden Markov Model [7], [13] K Nearest Neighbor (KNN) [3], [8], [14], [20], [62], [63], [51] Extremely Randomized Tree [20] Linear Regression <...>…”

Section: Ensemblementioning

confidence: 99%

See 1 more Smart Citation

Studi Literatur Human Activity Recognition (HAR) Menggunakan Sensor Inersia

Pradani

Mahananto²

2021

RESTI

View full text Add to dashboard Cite

Human activity recognition (HAR) is one of the topics that is being widely researched because of its diverse implementation in various fields such as health, construction, and UI / UX. As MEMS (Micro Electro Mechanical Systems) evolves, HAR data acquisition can be done more easily and efficiently using inertial sensors. Inertial sensor data processing for HAR requires a series of processes and a variety of techniques. This literature study aims to summarize the various approaches that have been used in existing research in building the HAR model. Published articles are collected from ScienceDirect, IEEE Xplore, and MDPI over the past five years (2017-2021). From the 38 studies identified, information extracted are the overview of the areas of HAR implementation, data acquisition, public datasets, pre-process methods, feature extraction approaches, feature selection methods, classification models, training scenarios, model performance, and research challenges in this topic. The analysis showed that there is still room to improve the performance of the HAR model. Therefore, future research on the topic of HAR using inertial sensors can focus on extracting and selecting more optimal features, considering the robustness level of the model, increasing the complexity of classified activities, and balancing accuracy with computation time.

show abstract

Human activity recognition based on smartphone and wearable sensors using multiscale DCNN ensemble

Cited by 61 publications

References 7 publications

Complex Deep Neural Networks from Large Scale Virtual IMU Data for Effective Human Activity Recognition Using Wearables

Complex Deep Neural Networks from Large Scale Virtual IMU Data for Effective Human Activity Recognition Using Wearables

A Hybrid Deep Neural Network for Human Activity Recognition based on IoT Sensors

Studi Literatur Human Activity Recognition (HAR) Menggunakan Sensor Inersia

Contact Info

Product

Resources

About