Coarse-Fine Convolutional Deep-Learning Strategy for Human Activity Recognition

Avilés‐Cruz, Carlos; Ferreyra-Ramírez, Andrés; Zúñiga-López, Arturo; Villegas-Cortéz, Juan

doi:10.3390/s19071556

Cited by 56 publications

(38 citation statements)

References 39 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Vision and inertial sensing modalities have been used individually to achieve human action recognition, e.g., [13,14,15,16,17,18,19,20,21,22,23]. Furthermore, the use of deep learning models or deep neural networks have proven to be more effective than conventional approaches for human action recognition.…”

Section: Introductionmentioning

confidence: 99%

“…The main advantage of these sensors include their wearability—Thus, they are not limited to a specific field of view. Often, 3-axis acceleration signals from their accelerometers and 3-axis angular velocity signals from their gyroscopes are used for conducting human action recognition, e.g., [17,18,19,20,21,22,23]. These sensors have also limitations in terms of not capturing a complete representation of actions.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Fusion of Video and Inertial Sensing for Deep Learning–Based Human Action Recognition

Wei

Jafari

Kehtarnavaz

2019

Sensors

View full text Add to dashboard Cite

This paper presents the simultaneous utilization of video images and inertial signals that are captured at the same time via a video camera and a wearable inertial sensor within a fusion framework in order to achieve a more robust human action recognition compared to the situations when each sensing modality is used individually. The data captured by these sensors are turned into 3D video images and 2D inertial images that are then fed as inputs into a 3D convolutional neural network and a 2D convolutional neural network, respectively, for recognizing actions. Two types of fusion are considered—Decision-level fusion and feature-level fusion. Experiments are conducted using the publicly available dataset UTD-MHAD in which simultaneous video images and inertial signals are captured for a total of 27 actions. The results obtained indicate that both the decision-level and feature-level fusion approaches generate higher recognition accuracies compared to the approaches when each sensing modality is used individually. The highest accuracy of 95.6% is obtained for the decision-level fusion approach.

show abstract

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Fusion of Video and Inertial Sensing for Deep Learning–Based Human Action Recognition

Wei

Jafari

Kehtarnavaz

2019

Sensors

View full text Add to dashboard Cite

show abstract

“…For the experimental evaluation they used crowded scenes as a dataset, utilizing 100 video sequences and five well-defined abnormality categories. Avilés et al [ 14 ] presented a coarse-fine convolutional method for human activity recognition using smartphone sensors. Ordóñez et al [ 15 ] have also proposed techniques for wearable human activity recognition using multimodal deep convolutional and LSTM recurrent neural networks.…”

Section: Related Workmentioning

confidence: 99%

Real-Time Abnormal Event Detection for Enhanced Security in Autonomous Shuttles Mobility Infrastructures

Tsiktsiris

Dimitriou

Lalas

et al. 2020

Sensors

View full text Add to dashboard Cite

Autonomous vehicles (AVs) are already operating on the streets of many countries around the globe. Contemporary concerns about AVs do not relate to the implementation of fundamental technologies, as they are already in use, but are rather increasingly centered on the way that such technologies will affect emerging transportation systems, our social environment, and the people living inside it. Many concerns also focus on whether such systems should be fully automated or still be partially controlled by humans. This work aims to address the new reality that is formed in autonomous shuttles mobility infrastructures as a result of the absence of the bus driver and the increased threat from terrorism in European cities. Typically, drivers are trained to handle incidents of passengers’ abnormal behavior, incidents of petty crimes, and other abnormal events, according to standard procedures adopted by the transport operator. Surveillance using camera sensors as well as smart software in the bus will maximize the feeling and the actual level of security. In this paper, an online, end-to-end solution is introduced based on deep learning techniques for the timely, accurate, robust, and automatic detection of various petty crime types. The proposed system can identify abnormal passenger behavior such as vandalism and accidents but can also enhance passenger security via petty crimes detection such as aggression, bag-snatching, and vandalism. The solution achieves excellent results across different use cases and environmental conditions.

show abstract

“…For coarse-medium-fine feature extraction and fusion, a coarsefine convolutional deep-learning strategy is proposed for human activity recognition [1]. The ideas of learning and predicting coarse and fine classes have been explored in computer vision.…”

Section: Coarse-to-fine Classificationmentioning

confidence: 99%

Fine-grained activities recognition with coarse-grained labeled multi-modal data

Zhang

et al. 2020

Adjunct Proceedings of the 2020 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of The

View full text Add to dashboard Cite

Fine-grained human activities recognition focuses on recognizing event-or action-level activities, which enables a new set of Internetof-Things (IoT) applications such as behavior analysis. Prior work on fine-grained human activities recognition relies on supervised sensing, which makes the fine-grained labeling labor-intensive and difficult to scale up. On the other hand, it is much more practical to collect coarse-grained label at the level of activity of daily living (e.g., cooking, working), especially for real-world IoT systems. In this paper, we present a framework that learns fine-grained human activities recognition with coarse-grained labeled and a small amount of fine-grained labeled multi-modal data. Our system leverages the implicit physical knowledge on the hierarchy of the coarse-and fine-grained labels and conducts data-driven hierarchical learning that take into account the coarse-grained supervised prediction for fine-grained semi-supervised learning. We evaluated our framework and CFR-TSVM algorithm on the data gathered from real-world experiments. Results show that our CFR-TSVM achieved an 81% recognition accuracy over 10 fine-grained activities, which reduces the prediction error of the semi-supervised learning baseline TSVM by half.

show abstract

Coarse-Fine Convolutional Deep-Learning Strategy for Human Activity Recognition

Cited by 56 publications

References 39 publications

Fusion of Video and Inertial Sensing for Deep Learning–Based Human Action Recognition

Fusion of Video and Inertial Sensing for Deep Learning–Based Human Action Recognition

Real-Time Abnormal Event Detection for Enhanced Security in Autonomous Shuttles Mobility Infrastructures

Fine-grained activities recognition with coarse-grained labeled multi-modal data

Contact Info

Product

Resources

About