A deep neural network model for multi-view human activity recognition

Putra, Prasetia Utama; Shima, Keisuke; Shimatani, Koji

doi:10.1371/journal.pone.0262181

Cited by 17 publications

(7 citation statements)

References 57 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Convolutional neural networks (CNNs) and recurrent neural networks (RNNs) have recently emerged as promising candidates for human recognition thanks to advancements in deep learning [7]. These methods can learn hierarchical representations of images and videos that can be used to recognize activities.…”

Section: Related Workmentioning

confidence: 99%

“…Human interaction recognition recognizes and classifies human activities using visual data, such as images or videos, to recognize and organize them. Regarding recognizing human-human interactions (HHI) in computer vision, researchers have proposed various methods that use machine learning classifiers such as Random Forest, Support Vector Machine (SVM), Decision Trees, and HMM to recognize and categorize human interactions, such as shake_hands, hugs, and high-fives, into distinctly different classes [7,8]. These methods extract handcrafted elements, including spatialtemporal information, posture, and gesture, from images or videos and use classifiers to recognize and classify interactions.…”

Section: Image-based Hirmentioning

confidence: 99%

See 1 more Smart Citation

A Novel Human Interaction Framework Using Quadratic Discriminant Analysis with HMM

Bukht,

Mudawi,

Alotaibi

et al. 2023

CMC

View full text Add to dashboard Cite

Human-human interaction recognition is crucial in computer vision fields like surveillance, human-computer interaction, and social robotics. It enhances systems' ability to interpret and respond to human behavior precisely. This research focuses on recognizing human interaction behaviors using a static image, which is challenging due to the complexity of diverse actions. The overall purpose of this study is to develop a robust and accurate system for human interaction recognition. This research presents a novel image-based human interaction recognition method using a Hidden Markov Model (HMM). The technique employs hue, saturation, and intensity (HSI) color transformation to enhance colors in video frames, making them more vibrant and visually appealing, especially in low-contrast or washed-out scenes. Gaussian filters reduce noise and smooth imperfections followed by silhouette extraction using a statistical method. Feature extraction uses the features from Accelerated Segment Test (FAST), Oriented FAST, and Rotated BRIEF (ORB) techniques. The application of Quadratic Discriminant Analysis (QDA) for feature fusion and discrimination enables high-dimensional data to be effectively analyzed, thus further enhancing the classification process. It ensures that the final features loaded into the HMM classifier accurately represent the relevant human activities. The impressive accuracy rates of 93% and 94.6% achieved in the BIT-Interaction and UT-Interaction datasets respectively, highlight the success and reliability of the proposed technique. The proposed approach addresses challenges in various domains by focusing on frame improvement, silhouette and feature extraction, feature fusion, and HMM classification. This enhances data quality, accuracy, adaptability, reliability, and reduction of errors.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Image-based Hirmentioning

confidence: 99%

A Novel Human Interaction Framework Using Quadratic Discriminant Analysis with HMM

Bukht,

Mudawi,

Alotaibi

et al. 2023

CMC

View full text Add to dashboard Cite

show abstract

“…Deep learning is advantageous in the sense that it performs automatic feature extraction, which is particularly useful when dealing with challenging datasets. The paper [85], proposed a Deep Neural Network (DNN) consisting of a combination of Convolutional Neural Networks (CNN), attention layers, LSTM, and softmax layers to improve the accuracy of predictions and for short-term predicting. The attention layer selectively concentrates on the important parts of the input sequence, while the softmax is the activation function that normalizes the outputs.…”

Section: The Approaches Used In Hap/harmentioning

confidence: 99%

Predicting Human Activity – State of the Art

Awong¹,

Zielińska²

2023

PAR

View full text Add to dashboard Cite

Predicting human actions is a very actual research field. Artificial intelligence methods are commonly used here. They enable early recognition and classification of human activities. Such knowledge is extremely needed in the work on robots and other interactive systems that communicate and cooperate with people. This ensures early reactions of such devices and proper planning of their future actions. However, due to the complexity of human actions, predicting them is a difficult task. In this article, we review state-of-the-art methods and summarize recent advances in predicting human activity. We focus in particular on four approaches using machine learning methods, namely methods using: artificial neural networks, support vector machines, probabilistic models and decision trees. We discuss the advantages and disadvantages of these approaches, as well as current challenges related to predicting human activity. In addition, we describe the types of sensors and data sets commonly used in research on predicting and recognizing human actions. We analyze the quality of the methods used, based on the prediction accuracy reported in scientific articles. We describe the importance of the data type and the parameters of machine learning models. Finally, we summarize the latest research trends. The article is intended to help in choosing the right method of predicting human activity, along with an indication of the tools and resources necessary to effectively achieve this goal.

show abstract

“…In this work, we take inspiration from the multi-view human action recognition (MVHAR) field to address the problem of animal behavior recognition, particularly the WDS behavior in rats. There are two main approaches for MVHAR; the first one is to train end-to-end neural networks ( Putra et al, 2018 , 2022 ; Wang et al, 2018 ; Vyas et al, 2020 ), but the performance is tied to the use of large MVHAR datasets ( Weinland et al, 2006 ; Gkalelis et al, 2009 ; Wang et al, 2014 ; Shahroudy et al, 2016 ; Guo et al, 2022 ). Equivalent animal datasets are not available, and producing them would be costly.…”

Section: Introductionmentioning

confidence: 99%

Multi-view image-based behavior classification of wet-dog shake in Kainate rat model

Negrete

Arai

Natsume

et al. 2023

Front. Behav. Neurosci.

View full text Add to dashboard Cite

The wet-dog shake behavior (WDS) is a short-duration behavior relevant to the study of various animal disease models, including acute seizures, morphine abstinence, and nicotine withdrawal. However, no animal behavior detection system has included WDS. In this work, we present a multi-view animal behavior detection system based on image classification and use it to detect rats’ WDS behavior. Our system uses a novel time-multi-view fusion scheme that does not rely on artificial features (feature engineering) and is flexible to adapt to other animals and behaviors. It can use one or more views for higher accuracy. We tested our framework to classify WDS behavior in rats and compared the results using different amounts of cameras. Our results show that the use of additional views increases the performance of WDS behavioral classification. With three cameras, we achieved a precision of 0.91 and a recall of 0.86. Our multi-view animal behavior detection system represents the first system capable of detecting WDS and has potential applications in various animal disease models.

show abstract

A deep neural network model for multi-view human activity recognition

Cited by 17 publications

References 57 publications

A Novel Human Interaction Framework Using Quadratic Discriminant Analysis with HMM

A Novel Human Interaction Framework Using Quadratic Discriminant Analysis with HMM

Predicting Human Activity – State of the Art

Multi-view image-based behavior classification of wet-dog shake in Kainate rat model

Contact Info

Product

Resources

About