Study on the Interaction Behaviors Identification of Construction Workers Based on ST-GCN and YOLO

Abstract

The construction industry is accident-prone, and unsafe behaviors of construction workers have been identified as a leading cause of accidents. One important countermeasure to prevent accidents is monitoring and managing those unsafe behaviors. The most popular way of detecting and identifying workers’ unsafe behaviors is the computer vision-based intelligent monitoring system. However, most of the existing research or products focused only on the workers’ behaviors (i.e., motions) recognition, limited studies considered the interaction between man-machine, man-material or man-environments. Those interactions are very important for judging whether the workers’ behaviors are safe or not, from the standpoint of safety management. This study aims to develop a new method of identifying construction workers’ unsafe behaviors, i.e., unsafe interaction between man-machine/material, based on ST-GCN (Spatial Temporal Graph Convolutional Networks) and YOLO (You Only Look Once), which could provide more direct and valuable information for safety management. In this study, two trained YOLO-based models were, respectively, used to detect safety signs in the workplace, and objects that interacted with workers. Then, an ST-GCN model was trained to detect and identify workers’ behaviors. Lastly, a decision algorithm was developed considering interactions between man-machine/material, based on YOLO and ST-GCN results. Results show good performance of the developed method, compared to only using ST-GCN, the accuracy was significantly improved from 51.79% to 85.71%, 61.61% to 99.11%, and 58.04% to 100.00%, respectively, in the identification of the following three kinds of behaviors, throwing (throwing hammer, throwing bottle), operating (turning on switch, putting bottle), and crossing (crossing railing and crossing obstacle). The findings of the study have some practical implications for safety management, especially workers’ behavior monitoring and management.

A Review of Computer Vision-Based Monitoring Approaches for Construction Workers’ Work-Related Behaviors

“…Table 1 presents other papers that integrated multiple algorithms to achieve improved visual recognition. [113] Detect workers and predict their movement trajectories SURF + improved GMM + HOG + SVM Li et al [114] Hardhat-wearing tracking detection YOLO v5 + Strong SORT Lee et al [115] PPE usage detection YOLACT employs MobineNetv3 + DeepSORT Li et al [116] Recognize construction workers' activities YOLO + ST-GCN (Spatial temporal graph convolutional networks)…”

“…Similarly, Cai et al [128] introduced an attention direction estimation method to identify groups of construction workers and subsequently classified their activities using LSTM. Li et al [116] identified three activities of construction workers through YOLO and ST-GCN, which are throwing, operating and crossing. Torabi et al [129] and Yang et al [96] employed YOWO53 and Transformer, respectively, demonstrating high recognition accuracy on today's advanced computing hardware.…”

Construction Activity Recognition Method Based on Object Detection, Attention Orientation Estimation, and Person Re-Identification

“…This method was then used for activity classification through LSTM. Li et al [60] identified three types of construction worker behaviors: throwing, operating, and crossing, using YOLO and ST-GCN. Torabi et al [61] proposed a YOWO53 model that recognizes construction activities.…”

Analysis of Geometric Characteristics of Cracks and Delamination in Aerated Concrete Products Using Convolutional Neural Networks

“…The use of intelligent monitoring systems based on computer vision in the field of security is one of the most popular methods today [24]. In the study [25], the authors developed methods for detecting safety violations among workers. As a result, a decision-making algorithm was defined that takes into account the interaction between the human, machine and material.…”