Skeleton-Based Abnormal Behavior Detection Using Secure Partitioned Convolutional Neural Network Model

Qiu, Jiefan; Yan, Xing‐Gang; Wang, Wei; Wei, Wei; Fang, Kai

doi:10.1109/jbhi.2021.3137334

Cited by 18 publications

(7 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Extensive research has been conducted on these systems, primarily focusing on the detection of abnormal behavior such as ‘fighting’ and ‘fainting’. This involves the utilization of technologies such as object detection and recognition, tracking, pose estimation, movement detection, and anomaly detection of objects [ 2 , 3 , 4 , 5 , 6 , 7 , 8 , 9 ]. Jha et al [ 8 ] proposed an N-YOLO model designed for the detection of abnormal behaviors, such as fighting.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Enhancing Surveillance Systems: Integration of Object, Behavior, and Space Information in Captions for Advanced Risk Assessment

Jeon,

Ko,

Cheoi

2024

Sensors

View full text Add to dashboard Cite

This paper presents a novel approach to risk assessment by incorporating image captioning as a fundamental component to enhance the effectiveness of surveillance systems. The proposed surveillance system utilizes image captioning to generate descriptive captions that portray the relationship between objects, actions, and space elements within the observed scene. Subsequently, it evaluates the risk level based on the content of these captions. After defining the risk levels to be detected in the surveillance system, we constructed a dataset consisting of [Image-Caption-Danger Score]. Our dataset offers caption data presented in a unique sentence format, departing from conventional caption styles. This unique format enables a comprehensive interpretation of surveillance scenes by considering various elements, such as objects, actions, and spatial context. We fine-tuned the BLIP-2 model using our dataset to generate captions, and captions were then interpreted with BERT to evaluate the risk level of each scene, categorizing them into stages ranging from 1 to 7. Multiple experiments provided empirical support for the effectiveness of the proposed system, demonstrating significant accuracy rates of 92.3%, 89.8%, and 94.3% for three distinct risk levels: safety, hazard, and danger, respectively.

show abstract

Section: Introductionmentioning

confidence: 99%

“…These various studies [ 2 , 3 , 4 , 5 , 6 , 7 , 8 , 9 ] have improved performance by incorporating diverse feature information related to abnormal behavior. Nevertheless, conventional methods exhibit certain limitations.…”

Section: Introductionmentioning

confidence: 99%

Enhancing Surveillance Systems: Integration of Object, Behavior, and Space Information in Captions for Advanced Risk Assessment

Jeon,

Ko,

Cheoi

2024

Sensors

View full text Add to dashboard Cite

show abstract

“…Specifically, researchers focus on human behavior analysis based on multi-target tracking (MOT) for healthcare systems [ 3 ]. MOT is a topic of interest in the field of computer vision, which has broad prospects in fields, including intelligent video monitoring [ 4 , 5 , 6 ], assisted driving [ 7 , 8 , 9 ], smart agriculture [ 10 , 11 ], and behavior analysis [ 12 , 13 , 14 ]. The main task is to track multiple objects in a video sequence, assign unique identifiers (IDs) to each object, maintain the stability of identity when occlusion and interaction occur, and finally obtain the object’s motion track.…”

Section: Introductionmentioning

confidence: 99%

Multi-Target Tracking Based on a Combined Attention Mechanism and Occlusion Sensing in a Behavior-Analysis System

Zhou

Chan

Qiu

et al. 2023

Sensors

View full text Add to dashboard Cite

Multi-object tracking (MOT) is a topic of great interest in the field of computer vision, which is essential in smart behavior-analysis systems for healthcare, such as human-flow monitoring, crime analysis, and behavior warnings. Most MOT methods achieve stability by combining object-detection and re-identification networks. However, MOT requires high efficiency and accuracy in complex environments with occlusions and interference. This often increases the algorithm’s complexity, affects the speed of tracking calculations, and reduces real-time performance. In this paper, we present an improved MOT method combining an attention mechanism and occlusion sensing as a solution. A convolutional block attention module (CBAM) calculates the weights of space and channel attention from the feature map. The attention weights are used to fuse the feature maps to extract adaptively robust object representations. An occlusion-sensing module detects an object’s occlusion, and the appearance characteristics of an occluded object are not updated. This can enhance the model’s ability to extract object features and improve appearance feature pollution caused by the short-term occlusion of an object. Experiments on public datasets demonstrate the competitive performance of the proposed method compared with the state-of-the-art MOT methods. The experimental results show that our method has powerful data association capability, e.g., 73.2% MOTA and 73.9% IDF1 on the MOT17 dataset.

show abstract

“…These methods are mostly considered as unsupervised learning and can be divided into two types of methods. The first method extracts features from the original video and then uses clustering [1][2][3] or classifiers [4,5] for anomaly detection. The second method reconstructs or predicts the input sequence and calculate the error between the real sequence and the generated sequence.…”

Section: Introductionmentioning

confidence: 99%

“…Subsequently, many studies have improved ST‐GCN. Similar to skeleton‐based action recognition methods [23], most studies [2–5, 16, 24–26] modelled the spatio‐temporal information of motion targets using the ST‐GCN. Although GCN has strong spatial feature learning capability, it tends to ignore global dependencies, which are crucial for behavioural understanding.…”

Section: Introductionmentioning

confidence: 99%

A Spatio‐Temporal Enhanced Graph‐Transformer AutoEncoder embedded pose for anomaly detection

Zhu,

Wei,

2023

IET Computer Vision

View full text Add to dashboard Cite

Due to the robustness of skeleton data to human scale, illumination changes, dynamic camera views, and complex backgrounds, great progress has been made in skeleton‐based video anomaly detection in recent years. The spatio‐temporal graph convolutional network has been proven to be effective in modelling the spatio‐temporal dependencies of non‐Euclidean data such as human skeleton graphs, and the autoencoder based on this basic unit is widely used to model sequence features. However, due to the limitations of the convolution kernel, the model cannot capture the correlation between non‐adjacent joints, and it is difficult to deal with long‐term sequences, resulting in an insufficient understanding of behaviour. To address this issue, this paper applies the Transformer to the human skeleton and proposes the Spatio‐Temporal Enhanced Graph‐Transformer AutoEncoder (STEGT‐AE) to improve the capability of modelling. In addition, the multi‐memory model with skip connections is employed to provide different levels of coding features, thereby enhancing the ability of the model to distinguish similar heterogeneous behaviours. Furthermore, the STEGT‐AE has a single encoder‐double decoder architecture, which can improve the detection performance by the combining reconstruction and prediction error. The experimental results show that performances of STEGT‐AE is significantly better than other advanced algorithms on four baseline datasets.

show abstract

Skeleton-Based Abnormal Behavior Detection Using Secure Partitioned Convolutional Neural Network Model

Cited by 18 publications

References 34 publications

Enhancing Surveillance Systems: Integration of Object, Behavior, and Space Information in Captions for Advanced Risk Assessment

Enhancing Surveillance Systems: Integration of Object, Behavior, and Space Information in Captions for Advanced Risk Assessment

Multi-Target Tracking Based on a Combined Attention Mechanism and Occlusion Sensing in a Behavior-Analysis System

A Spatio‐Temporal Enhanced Graph‐Transformer AutoEncoder embedded pose for anomaly detection

Contact Info

Product

Resources

About