Background Appearance Modeling with Applications to Visual Object Detection in an Open‐Pit Mine

Abstract

This paper addresses the problem of detecting people and vehicles on a surface mine by presenting an architecture that combines the complementary strengths of deep convolutional networks (DCN) with cluster‐based analysis. We highlight that using a DCN in a naïve black box approach results in a significantly high rate of errors due to the lack of mining‐specific training data and the unique landscape in a mine site. In this work, we propose a background model that exploits the abundance of background‐only images to discover the natural clusters in visual appearance using features extracted from the DCN. Both a simple nearest cluster‐based background model and an extended model with cosine features are investigated for their ability to identify and suppress potential false positives made by the DCN. Furthermore, localization of objects of interest is enabled through region proposals, which have been tuned to increase recall within the constraints of a computational budget. Finally, a soft fusion framework is presented to combine the estimates of both the DCN and background model to improve the accuracy of the detection. Our system is tested on over 11 km of real mine site data in both day and night conditions where we were able to detect both light and heavy vehicles along with mining personnel. We show that the introduction of our background model improves the detection performance. In particular, soft fusion of the background model and the DCN output produces a relative improvement in the F1 score of 46% and 28% compared to a baseline pretrained DCN and a DCN retrained with mining images, respectively.

What makes a place? Building bespoke place dependent object detectors for robotics

“…This has some similarity to our approach here; we build on the ideas from [1] and [28] to understand what influences an appropriate resolution for local expert detectors. Bewley and Upcroft presented similar evidence for the need to adapt object detectors to their operating environment in [29], augmenting a generalised object detector by validating each detection against a background model trained on environment specific data. This approach can also be viewed as exploiting place as context, with some similarities to methods which use context within an image to boost detector performance, e.g.…”

Predicting the availability of continuous mining systems using LSTM neural network

“…In addition to the works, related to the mining process, there are also works in which models are developed for detecting the geomechanical anomalies, [39][40][41] analysis of available resources, [42][43][44][45] assessment the impact of mining works on the environment. 46,47 The machine learning method is also used for risk assessment of landslides at the open pit, 48 visual detection of objects at the open pit that can classify workers and mining machinery, 49 or prediction the health risks of drivers caused by vibrations during truck transport. 50…”

Systematic Review of Machine Learning Applications in Mining: Exploration, Exploitation, and Reclamation

“…A gas control [103][104][105] study was conducted to predict the generation of hazardous gases in mines. There were studies on occupational safety [106,107] that detected the equipment and predicted accident probabilities, thereby preventing accidents and facilitating the designing of suitable ventilations [108].…”