Fusion of Photogrammetry and Video Analysis for Productivity Assessment of Earthwork Processes

Abstract

The high complexity of modern large‐scale construction projects leads their schedules to be sensitive to delays. At underground construction sites, the earthwork processes are vital, as most of the following tasks depend on them. This article presents a method for estimating the productivity of soil removal by combining two technologies based on computer vision: photogrammetry and video analysis. Photogrammetry is applied to create a time series of point clouds throughout excavation, which are used to measure the volume of the excavated soil for daily estimates of productivity. Video analysis is used to generate statistics regarding the construction activities for estimating productivity at finer time scales, when combined with the output from the photogrammetry pipeline. As there may be multiple causes for specific productivity levels, the automated generation of progress and activity statistics from both measurement methods supports interpretation of the productivity estimates. Comparison to annotated ground truth for the tracking and activity monitoring method highlights the reliability of the extracted information. The suitability of the approach is demonstrated by two case studies of real‐world urban excavation projects.

Edge‐Enhanced Matching for Gradient‐Based Computer Vision Displacement Measurement

“…Vision sensors, among these new techniques, have been broadly applied for civil engineering problems. Famous applications of vision‐sensing techniques include dynamic displacement monitoring (Cha et al., ; Park et al., ; Yoon et al., ), three‐axes (i.e., X‐axis, Y‐axis, and depth) displacement measurement (Park et al., ; Abdelbarr et al., ), surface displacement/strain measurement (Luo et al., ; Almeida et al., ), vision‐based structural analysis (Chen et al., ; Sharif et al., ; Park et al., ), cable tensile force evaluation (Kim et al., ), bridge‐lining inspection (Zhu et al., ), rocking motion and landslide monitoring (Debella‐Gilo and Kääb, ; Greenbaum et al., ), automatic construction progress assessment (Bügler et al., ), 3D object finding in point cloud (Sharif et al., ), surface crack/defection detection based on texture‐based video processing (Cord and Chambon, ; Chen et al., ) or deep learning (Cha et al., ; Cha et al, ; Zhang et al., ), vehicle classification based on spectrogram features (Yeum et al., ), and intelligent transportation (Chen et al., ; Fernandez‐Llorca et al., ). With advancement in image sensors and computer techniques such as computer vision, cloud computing, and wireless data transfer, vision sensors have become more cost‐effective and computation‐efficient, thus have high potential in field application for SHM problems.…”

Capturing and Understanding Workers’ Activities in Far‐Field Surveillance Videos with Deep Action Recognition and Bayesian Nonparametric Learning

“…The literature on complex construction activity recognition is extensive and focuses particularly on productivity analysis of construction equipment (Zou and Kim, ; Gong and Caldas, ; Rezazadeh Azar et al., ; Yang et al., ; Bügler et al., ). For example, Gong and Caldas () proposed a description‐based method to analyze the productivity of concrete placing with a tower crane and concrete buckets.…”

“…The method employed the prior knowledge of the concrete placing process, which was broken into a variety of actions; it described how these actions unfold in the planned spatial locations and temporal sequences. Similar description‐based methods were used to analyze the productivity of earthwork (Bügler et al., ) and dirt loading (Rezazadeh Azar et al., ). Description‐based methods assume that the knowledge of activities of interest (e.g., activity locations and sequences) can be captured and defined in advance.…”

“…(), and Bügler et al. () are description‐based (Aggarwal and Ryoo, ); they model an activity as an occurrence of its subevent that deterministically satisfies temporal, spatial, or logical relations. For example, Gong and Caldas () investigated concrete placing of a tower crane and concrete buckets.…”

Classical Planning Model‐Based Approach to Automating Construction Planning on Earthwork Projects

“…Global positioning system (Pradhananga and Teizer, ) and other localization sensors (Li et al, ) were applied on earthwork projects for real‐time equipment tracking and cycle time analysis. Researchers also investigated the use of images and videos for earthwork volume takeoff (Siebert and Teizer, ) and productivity analysis (Bugler et al, ; Rezazadeh Azar et al, ). Such information provides valuable inputs for adaptive earthmoving operations simulation (Montaser et al, ), earthwork allocation optimization (Easa, ; Hare et al, ; Ji et al, ), and highway design optimization (Vázquez‐Méndez et al, ; Jong et al., ).…”