Computer Vision-Based Hazard Identification of Construction Site Using Visual Relationship Detection and Ontology

Abstract

Onsite systematic monitoring benefits hazard prevention immensely. Hazard identification is usually limited due to the semantic gap. Previous studies that integrate computer vision and ontology can address the semantic gap and detect the onsite hazards. However, extracting and encoding regulatory documents in a computer-processable format often requires manual work which is costly and time-consuming. A novel and universally applicable framework is proposed that integrates computer vision, ontology, and natural language processing to improve systematic safety management, capable of hazard prevention and elimination. Visual relationship detection based on computer vision is used to detect and predict multiple interactions between objects in images, whose relationships are then coded in a three-tuple format because it has abundant expressiveness and is computer-accessible. Subsequently, the concepts of construction safety ontology are presented to address the semantic gap. The results are subsequently recorded into the SWI Prolog, a commonly used tool to run Prolog (programming of logic), as facts and compared with triplet rules extracted from using natural language processing to indicate the potential risks in the ongoing work. The high-performance results of Recall@100 demonstrated that the chosen method can precisely predict the interactions between objects and help to improve onsite hazard identification.

Investigation and Development of Methods and Algorithms for Analyzing Video-Eeg Monitoring of Delayed Cerebral Ischemia After Subarachnoid Haemorrhage

“…The second important feature of the developed architecture is that the Actor Prolog language is a translator to Java (Morozov et al, 2015) that enables one to obtain a high-performance code and simplifies the connection of the software system with the OpenCV library. Note that only a few neural-symbolic systems in the world are based on professional programming languages/libraries like Python (Mojarad et al, 2020, Tarau, 2021, Li et al, 2022, Winters et al, 2022, Ciatto et al, 2022, Kotlin (Ciatto et al, 2021), PyTorch (Manhaeve et al, 2018, Yang et al, 2020, TensorFlow (Li et al, 2022, Ciatto et al, 2022, and Keras (Mojarad et al, 2020). Most neural-symbolic systems are applied only for solving toy problems and only a few neural-symbolic systems applied modern neural architectures like YOLO (Khan et al, 2019) and VGG16 (Padalkar et al, 2023).…”

Utilizing 360-Degree Images for Synthetic Data Generation in Construction Scenarios

“…Effectively monitoring construction sites in real-time demands robust computer vision (CV) models trained on diverse datasets capturing different viewpoints, surface properties, and lighting conditions (Li et al, 2022;Sami Ur Rehman et al, 2022). This diversity ensures accurate and adaptable surveillance for improved safety and efficiency.…”