An End-to-End Human Abnormal Behavior Recognition Framework for Crowds With Mentally Disordered Individuals

Hao, Yixue; Tang, Zaiyang; Alzahrani, Bander A.; Alotaibi, Reem; Alharthi, Reem; Zhao, Miaomiao; Mahmood, Arif

doi:10.1109/jbhi.2021.3122463

Cited by 21 publications

(3 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…When people with mental illnesses behave abnormally in public settings, they risk hurting themselves and other people's bodies and minds. As a result, it's important to use visual surveillance devices to keep an eye on their activities [ 12 ]. AI has proved significant with several techniques like YOLOv3 combined with the K-Means algorithm, GIoUloss, focal loss, and Darknet32, which has yielded promising results in detecting such abnormal behaviours [ 13 ].…”

Section: Types Of Behavioral Analysismentioning

confidence: 99%

Machine learning for cognitive behavioral analysis: datasets, methods, paradigms, and research directions

Bhatt,

Sethi,

Tasgaonkar

et al. 2023

Brain Inf.

View full text Add to dashboard Cite

Human behaviour reflects cognitive abilities. Human cognition is fundamentally linked to the different experiences or characteristics of consciousness/emotions, such as joy, grief, anger, etc., which assists in effective communication with others. Detection and differentiation between thoughts, feelings, and behaviours are paramount in learning to control our emotions and respond more effectively in stressful circumstances. The ability to perceive, analyse, process, interpret, remember, and retrieve information while making judgments to respond correctly is referred to as Cognitive Behavior. After making a significant mark in emotion analysis, deception detection is one of the key areas to connect human behaviour, mainly in the forensic domain. Detection of lies, deception, malicious intent, abnormal behaviour, emotions, stress, etc., have significant roles in advanced stages of behavioral science. Artificial Intelligence and Machine learning (AI/ML) has helped a great deal in pattern recognition, data extraction and analysis, and interpretations. The goal of using AI and ML in behavioral sciences is to infer human behaviour, mainly for mental health or forensic investigations. The presented work provides an extensive review of the research on cognitive behaviour analysis. A parametric study is presented based on different physical characteristics, emotional behaviours, data collection sensing mechanisms, unimodal and multimodal datasets, modelling AI/ML methods, challenges, and future research directions.

show abstract

Section: Types Of Behavioral Analysismentioning

confidence: 99%

Machine learning for cognitive behavioral analysis: datasets, methods, paradigms, and research directions

Bhatt,

Sethi,

Tasgaonkar

et al. 2023

Brain Inf.

View full text Add to dashboard Cite

show abstract

“…Moreover, manual labeling of large datasets could be highly time consuming, often creating bottlenecks in the data preparation process. To address these limitations, unsupervised methods for analyzing behavioral data have been suggested in order to automatically group data without predefined labels, thereby enhancing efficiency and scalability [26][27][28]. For example, pose estimation methods have been considered in [29,30] to map behavior, based on limb positions, in recorded videos.…”

Section: Introductionmentioning

confidence: 99%

An LSTM-based adversarial variational autoencoder framework for self-supervised neural decoding of behavioral choices

Salsabilian,

Lee,

Margolis

et al. 2024

J. Neural Eng.

View full text Add to dashboard Cite

Objective. This paper presents data-driven solutions to address two challenges in the problem of linking neural data and behavior: 1) unsupervised analysis of behavioral data and automatic label generation from behavioral observations, and 2) extraction of subject-invariant features for the development of generalized neural decoding models.Approach. For behavioral analysis and label generation, an unsupervised method, which employs an autoencoder to transform behavior data into a cluster-friendly feature space is presented. The model iteratively refines the assigned clusters with soft clustering assignment loss, and gradually improves the learned feature representations. To address subject variability in decoding neural activity, adversarial learning in combination with a long short-term memory-based adversarial variational autoencoder (LSTM-AVAE) model is employed. By using an adversary network to constrain the latent representations, the model captures shared information among subjects' neural activity, making it proper for cross-subject transfer learning.Main results. The proposed approach is evaluated using cortical recordings of Thy1-GCaMP6s transgenic mice obtained via widefield calcium imaging during a motivational licking behavioral experiment. The results show that the proposed model achieves an accuracy of 89.7% in cross-subject neural decoding, outperforming other well-known autoencoder-based feature learning models. These findings suggest that incorporating an adversary network eliminates subject dependency in representations, leading to improved cross-subject transfer learning performance, while also demonstrating the effectiveness of LSTM-based models in capturing the temporal dependencies within neural data.Significance. Results demonstrate the feasibility of the proposed framework in unsupervised clustering and label generation of behavioral data, as well as achieving high accuracy in cross-subject neural decoding, indicating its potentials for relating neural activity to behavior.

show abstract

“…Human motion, as an important component of body language, also conveys important emotional information. Human motion recognition is widely used in biomedical monitoring [2], clinical evaluation [3], and sports competition [4]. Research has shown that human motion is an important way of expressing emotions [5].…”

Section: Introductionmentioning

confidence: 99%

Attention‐based sensor fusion for emotion recognition from human motion by combining convolutional neural network and weighted kernel support vector machine and using inertial measurement unit signals

Zhao

Guo

Sun

et al. 2023

IET Signal Processing

View full text Add to dashboard Cite

The remarkable development of human–computer interactions has created an urgent need for machines to be able to recognise human emotions. Human motions play a key role in emphasising and conveying emotions to meet the complexity of daily application scenarios, such as medical rehabilitation and social education. Therefore, this paper aims to explore hidden emotional states from human motions. Accordingly, we proposed a novel approach for emotion recognition using multiple inertial measurement unit (IMU) sensors worn on different body parts. First, the mapping relationship between emotion and human motion was established through fuzzy comprehensive evaluation, and data were collected for six emotional states: sleepy, bored, excited, tense, angry, and distressed. Second, the preprocessed data were used as input in a lightweight convolutional neural network to extract discriminative features. Third, an attention‐based sensor fusion module was developed to obtain the importance scores of each IMU sensor for generating a fused feature representation. In the recognition phase, we constructed a weighted kernel support vector machine (SVM) model with an auxiliary fuzzy function to improve the weight calculation method of kernel functions in a multiple kernel SVM. Finally, the results obtained are compared with those of similar state‐of‐the‐art studies, the proposed method showed a higher accuracy (99.02%) for the six emotional states mentioned above. These findings may promote the development of social robots with non‐verbal emotional communication capabilities.

show abstract

An End-to-End Human Abnormal Behavior Recognition Framework for Crowds With Mentally Disordered Individuals

Cited by 21 publications

References 31 publications

Machine learning for cognitive behavioral analysis: datasets, methods, paradigms, and research directions

Machine learning for cognitive behavioral analysis: datasets, methods, paradigms, and research directions

An LSTM-based adversarial variational autoencoder framework for self-supervised neural decoding of behavioral choices

Attention‐based sensor fusion for emotion recognition from human motion by combining convolutional neural network and weighted kernel support vector machine and using inertial measurement unit signals

Contact Info

Product

Resources

About