Leveraging CNN and Transfer Learning for Vision-based Human Activity Recognition

Deep, Samundra; Zheng, Xi

doi:10.1109/itnac46935.2019.9078016

Cited by 41 publications

(9 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The results of our HAR model (accuracy range 60–97%) are consistent with previous studies that classified activities of the lower limb in healthy people and people with medical conditions that affect their movement (e.g., Parkinson’s disease) which have reported model accuracy between 75% and 99% [ 14 , 25 , 26 , 27 , 35 , 36 , 46 , 47 , 48 ]. These previous studies use a single HAR model to classify between 5 and 12 activities resulting in an overall accuracy for the single model.…”

Section: Discussionsupporting

confidence: 88%

Human Activity Recognition for People with Knee Osteoarthritis—A Proof-of-Concept

Tan

Beheshti

Binnie

et al. 2021

Sensors

View full text Add to dashboard Cite

Clinicians lack objective means for monitoring if their knee osteoarthritis patients are improving outside of the clinic (e.g., at home). Previous human activity recognition (HAR) models using wearable sensor data have only used data from healthy people and such models are typically imprecise for people who have medical conditions affecting movement. HAR models designed for people with knee osteoarthritis have classified rehabilitation exercises but not the clinically relevant activities of transitioning from a chair, negotiating stairs and walking, which are commonly monitored for improvement during therapy for this condition. Therefore, it is unknown if a HAR model trained on data from people who have knee osteoarthritis can be accurate in classifying these three clinically relevant activities. Therefore, we collected inertial measurement unit (IMU) data from 18 participants with knee osteoarthritis and trained convolutional neural network models to identify chair, stairs and walking activities, and phases. The model accuracy was 85% at the first level of classification (activity), 89–97% at the second (direction of movement) and 60–67% at the third level (phase). This study is the first proof-of-concept that an accurate HAR system can be developed using IMU data from people with knee osteoarthritis to classify activities and phases of activities.

show abstract

Section: Discussionsupporting

confidence: 88%

Human Activity Recognition for People with Knee Osteoarthritis—A Proof-of-Concept

Tan

Beheshti

Binnie

et al. 2021

Sensors

View full text Add to dashboard Cite

show abstract

“…The DL techniques used in HAR can be divided into three parts such as deep neural networks (DNN), hybrid deep learning (HDL) models, and transfer learning (TL) based models . (Shown in Figure S.5 of Supporting document) The DNN includes the models like convolutional neural networks (CNN) (Deep and Zheng 2019;Liu et al 2020;Zeng et al 2014), recurrent neural networks (RNN) (Murad and Pyun 2017) and RNN variants which include long short-term memory (LSTM) and gated recurrent unit (GRU) (Zhu et al 2019;Du et al 2019;Fazli et al 2021). In hybrid HAR models, the combination of CNN and RNN models is trained on spatio-temporal data.…”

Section: Artificial Intelligence Models In Harmentioning

confidence: 99%

“…Examples of such devices include wearable sensors (Pham et al 2020), electronic device sensors like smartphone inertial sensor (Qi et al 2018;Zhu et al 2019), camera devices like Kinect (Wang et al 2019a;Phyo et al 2019), closed-circuit television (CCTV) (Du et al 2019), and some commercial off-theshelf (COTS) equipment's (Ding et al 2015;Li et al 2016). The use of diverse sources makes HAR important for multifaceted applications domains, such as healthcare (Pham et al 2020;Zhu et al 2019;Wang et al 2018), surveillance (Thida et al 2013;Deep and Zheng 2019;Vaniya and Bharathi 2016;Shuaibu et al 2017;Beddiar et al 2020) remote care to elderly people living alone (Phyo et al 2019;Deep and Zheng 2019;, smart home/office/city (Zhu et al 2019;Deep and Zheng 2019;Fan et al 2017), and various monitoring application like sports, and exercise (Ding et al 2015). The widespread use of HAR is beneficial for the safety and quality of life for humans (Ding et al 2015;Chen et al 2020).…”

Section: Introductionmentioning

confidence: 99%

Human activity recognition in artificial intelligence framework: a narrative review

et al. 2022

View full text Add to dashboard Cite

Human activity recognition (HAR) has multifaceted applications due to its worldly usage of acquisition devices such as smartphones, video cameras, and its ability to capture human activity data. While electronic devices and their applications are steadily growing, the advances in Artificial intelligence (AI) have revolutionized the ability to extract deep hidden information for accurate detection and its interpretation. This yields a better understanding of rapidly growing acquisition devices, AI, and applications, the three pillars of HAR under one roof. There are many review articles published on the general characteristics of HAR, a few have compared all the HAR devices at the same time, and few have explored the impact of evolving AI architecture. In our proposed review, a detailed narration on the three pillars of HAR is presented covering the period from 2011 to 2021. Further, the review presents the recommendations for an improved HAR design, its reliability, and stability. Five major findings were: (1) HAR constitutes three major pillars such as devices, AI and applications; (2) HAR has dominated the healthcare industry; (3) Hybrid AI models are in their infancy stage and needs considerable work for providing the stable and reliable design. Further, these trained models need solid prediction, high accuracy, generalization, and finally, meeting the objectives of the applications without bias; (4) little work was observed in abnormality detection during actions; and (5) almost no work has been done in forecasting actions. We conclude that: (a) HAR industry will evolve in terms of the three pillars of electronic devices, applications and the type of AI. (b) AI will provide a powerful impetus to the HAR industry in future. Supplementary Information The online version contains supplementary material available at 10.1007/s10462-021-10116-x.

show abstract

“…Referring to the steps from phase 1 illustrated in Figure 1, raw data gathered from the available sensor devices are immediately pre-processed (1,2). Data are then fragmented in small time-windows to capture fine gestures involved within the AR events (3).…”

Section: Teacher/learner Phasementioning

confidence: 99%

“…These techniques are being used as a starting point for training classification machine-learning models, such as in a convolutional neural network (CNN) that allows classification models to reduce the training investment. In particular, researchers have employed transfer learning to extend the features extraction when training machine-learning classifiers [3]. Two of the attributes commonly investigated for CNNs are architectures [4] and fine-tuning [5], where researchers use already trained CNN architectures and extend them by incorporating new convolutional layers to fine-tune the machine-learning models.…”

Section: Introductionmentioning

confidence: 99%

The Use of Transfer Learning for Activity Recognition in Instances of Heterogeneous Sensing

et al. 2021

View full text Add to dashboard Cite

Transfer learning is a growing field that can address the variability of activity recognition problems by reusing the knowledge from previous experiences to recognise activities from different conditions, resulting in the leveraging of resources such as training and labelling efforts. Although integrating ubiquitous sensing technology and transfer learning seem promising, there are some research opportunities that, if addressed, could accelerate the development of activity recognition. This paper presents TL-FmRADLs; a framework that converges the feature fusion strategy with a teacher/learner approach over the active learning technique to automatise the self-training process of the learner models. Evaluation TL-FmRADLs is conducted over InSync; an open access dataset introduced for the first time in this paper. Results show promising effects towards mitigating the insufficiency of labelled data available by enabling the learner model to outperform the teacher’s performance.

show abstract

Leveraging CNN and Transfer Learning for Vision-based Human Activity Recognition

Cited by 41 publications

References 13 publications

Human Activity Recognition for People with Knee Osteoarthritis—A Proof-of-Concept

Human Activity Recognition for People with Knee Osteoarthritis—A Proof-of-Concept

Human activity recognition in artificial intelligence framework: a narrative review

The Use of Transfer Learning for Activity Recognition in Instances of Heterogeneous Sensing

Contact Info

Product

Resources

About