A neural network–based approach for fill factor estimation and bucket detection on construction vehicles

Abstract

Bucket fill factor is of paramount importance in measuring the productivity of construction vehicles, which is the percentage of materials loaded in the bucket within one scooping. Additionally, the locational information of the bucket is also indispensable for scooping trajectory planning. Some research has been conducted to measure it via state-of-the-art computer vision approaches, but their robustness against various environment conditions is not considered. The aim of this study is to fill this gap and six distinctive environment settings are included. Images captured by a stereo camera are used to generate point clouds before being structured into 3D maps. This novel preprocessing pipeline for deep learning is originally proposed and its feasibility has been validated through this study. Moreover, multitask learning is employed to exploit the positive relationship among two tasks: fill factor prediction and bucket detection. Therefore, after preprocessing, 3D maps are forwarded to a faster region with convolutional neural network incorporated with an improved residual neural network. The value of fill factor is acquired via a classification and probabilistic-based approach, which is novel, achieving an inspiring result (overall volume estimation accuracy: 95.23% and detection precision: 92.62%) at the same time.

Fast estimation of loader's shovel load volume by 3D reconstruction of material piles

“…Guevara et al [9] used a binocular stereo camera to construct a 3D point cloud on the material surface of a bulldozer bucket, and used the Alpha-shape algorithm of Delaunay triangulation to estimate the effective shovel load of the bucket. Lu et al [10,11] developed a new perception system based on the stereo vision perception method, as well as advanced technologies such as point cloud registration, splicing, and surface interpolation, to realize the 3D point cloud reconstruction of materials in the loader bucket and accurate estimation of shovel loading. The aforementioned study produced an accurate assessment of the volume of materials in a single bucket during earthwork, offering reliable assurance for realtime estimation of earthwork volume and real-time evaluation of operational efficiency.…”

“…The if condition in Eq. (11) ensures that this constraint is satisfied because the corresponding points of the left and right images must appear on the same horizontal polar line, because the left and right images obtained by the binocular camera have been rectified in advance. Therefore, the likelihood probability model derived from Eq.…”

“…However, it is challenging to assess the operational efficiency in real time, necessitating a single shovel measurement. Existing research on single shovel measurements mainly focuses on constructing a 3D point cloud model of the bucket and the materials in the bucket, estimating the volume of materials in the bucket, and evaluating the bucket filling rate and shovel loading efficiency [8][9][10][11]. However, buckets of different earth-moving machines have different specifications and sizes.…”

Intelligent monitoring and evaluation for the prefabricated construction schedule

“…In recent years, CVB technologies have been greatly developed regarding the breakthrough related to the convolutional neural network (CNN)‐based deep learning method. This method has been applied in the construction field and has shown reliable performance in module detection (Zheng et al., 2020), bucket detection (Lu et al., 2021), ergonomic evaluation (Yan et al., 2017; Zhang et al., 2018), non‐hard‐hat‐use monitoring (Shen et al., 2021), structure health monitoring (Ngeljaratan et al., 2021; Sajedi & Liang, 2021), cable force estimation (Tian et al., 2021), drone scheduling (Yi & Sutrisna, 2021), and heavy machinery detection (Arabi et al., 2020), and so forth. However, few studies have been conducted to achieve schedule progress monitoring based on the CVB technology incorporated with CNN‐based deep learning.…”

A deep learning approach for construction vehicles fill factor estimation and bucket detection in extreme environments

“…However, they did not consider the effect of environmental variation and obtained an accuracy of 95%. Lu et al (2021) broke through the limitation of specific environments, and six normal environments were considered; the fill factor estimation task and bucket detection task were completed with 95.23% and 92.62% accuracy, respectively. However, fill factor estimation and bucket detection research are mainly performed under ideal environments, specific single environments, or several normal environments without considering the actual harsh environments at construction sites.…”