A Hybrid Deep Learning and Visualization Framework for Pushing Behavior Detection in Pedestrian Dynamics

Abstract

Crowded event entrances could threaten the comfort and safety of pedestrians, especially when some pedestrians push others or use gaps in crowds to gain faster access to an event. Studying and understanding pushing dynamics leads to designing and building more comfortable and safe entrances. Researchers—to understand pushing dynamics—observe and analyze recorded videos to manually identify when and where pushing behavior occurs. Despite the accuracy of the manual method, it can still be time-consuming, tedious, and hard to identify pushing behavior in some scenarios. In this article, we propose a hybrid deep learning and visualization framework that aims to assist researchers in automatically identifying pushing behavior in videos. The proposed framework comprises two main components: (i) Deep optical flow and wheel visualization; to generate motion information maps. (ii) A combination of an EfficientNet-B0-based classifier and a false reduction algorithm for detecting pushing behavior at the video patch level. In addition to the framework, we present a new patch-based approach to enlarge the data and alleviate the class imbalance problem in small-scale pushing behavior datasets. Experimental results (using real-world ground truth of pushing behavior videos) demonstrate that the proposed framework achieves an 86% accuracy rate. Moreover, the EfficientNet-B0-based classifier outperforms baseline CNN-based classifiers in terms of accuracy.

Pushing and Non-pushing Forward Motion in Crowds: A Systematic Psychological Observation Method for Rating Individual Behavior in Pedestrian Dynamics

“…While on the subject, possible future applications are described below. The first and probably the most prominent future study could be automating behavior detection by utilizing artificial intelligence (AI) [30,31]. As it was mentioned in the limitations, the rating process is time consuming and laborious, but an automated AI system could dramatically decrease the rating time by assisting raters in appraising clear cases while flagging the ambiguous ones.…”

Thermographic Fault Diagnosis of Shaft of BLDC Motor

“…Neural networks are used for fault diagnosis, the verification of technical equipment, image recognition, and industrial fault diagnosis systems [ 15 , 16 , 17 , 18 ]. There are several types of Convolutional Neural Networks (CNNs): GoogLeNet [ 19 , 20 , 21 ], ResNet50 [ 22 , 23 , 24 ], and EfficientNet-b0 [ 25 , 26 , 27 ]. CNNs can assign importance to objects in the thermal image.…”

“…EfficientNet-b0, available in Matlab software, consists of 290 layers. More information on GoogLeNet, ResNet50, and EfficientNet-b0 is described in the literature [ 19 , 20 , 21 , 22 , 23 , 24 , 25 , 26 , 27 ].…”

Herbal Leaf Classification Using Deep Learning Model Efficientnetv2b0

“…The EfficientNetV2 model was created to have faster training times and more efficient parameters than previous models [13]. The CNN EfficientNetV2B0 architecture model has been implemented in research with a case study of early detection of queuing pressure at the entrance to an event, where this research uses moving image or video data in real time [14]. The experimental results in this research were able to identify visitor queue pushing behavior with an accuracy rate of 87%, and when compared to the EfficientNetV1B0 model, it got an accuracy of 83% [15].…”

“…The CNN EfficientNetV2B0 architecture model has been implemented in research with a case study of early detection of queuing pressure at the entrance to an event, where this research uses moving image or video data in real time [14]. The experimental results in this research were able to identify visitor queue pushing behavior with an accuracy rate of 87%, and when compared to the EfficientNetV1B0 model, it got an accuracy of 83% [15]. EfficientNetV2B0 itself has the advantage of forming a much smaller model and faster convergence speed with minimal computing costs.…”