Enabling human–infrastructure interfaces for inspection using augmented reality

Abstract

Decaying infrastructure maintenance cost allocation depends heavily on accurate and safe inspection in the field. New tools to conduct inspections can assist in prioritizing investments in maintenance and repairs. The industrial revolution termed as “Industry 4.0” is based on the intelligence of machines working with humans in a collaborative workspace. Contrarily, infrastructure management has relied on the human for making day-to-day decisions. New emerging technologies can assist during infrastructure inspections, to quantify structural condition with more objective data. However, today’s owners agree in trusting the inspector’s decision in the field over data collected with sensors. If data collected in the field is accessible during the inspections, the inspector decisions can be improved with sensors. New research opportunities in the human–infrastructure interface would allow researchers to improve the human awareness of their surrounding environment during inspections. This article studies the role of Augmented Reality (AR) technology as a tool to increase human awareness of infrastructure in their inspection work. The domains of interest of this research include both infrastructure inspections (emphasis on the collection of data of structures to inform management decisions) and emergency management (focus on the data collection of the environment to inform human actions). This article describes the use of a head-mounted device to access real-time data and information during their field inspection. The authors leverage the use of low-cost smart sensors and QR code scanners integrated with Augmented Reality applications for augmented human interface with the physical environment. This article presents a novel interface architecture for developing Augmented Reality–enabled inspection to assist the inspector’s workflow in conducting infrastructure inspection works with two new applications and summarizes the results from various experiments. The main contributions of this work to computer-aided community are enabling inspectors to visualize data files from database and real-time data access using an Augmented Reality environment.

Machine learning and structural health monitoring overview with emerging technology and high-dimensional data source highlights

“…This new content delivery paradigm has enabled engineers to simultaneously immerse themselves in a fully connected physical and digital world. An AR-enabled infrastructure inspection interface was developed by Maharjan et al 349 The system was coupled with low-cost smart sensors and QR code scanners to assist the workflow of the inspectors. It is intuitive and much easier to apply ML and DL algorithms and display AR devices' results.…”

Extended Reality (XR) for Condition Assessment of Civil Engineering Structures: A Literature Review

“…Maharjan et al [ 125 ] developed and validated a novel human–machine interaction system through HoloLens, which assists the inspector in the field during data collection and decision-making processes. In essence, the study integrated two new applications with the HMD: a sensor–MR connection and a QR code–HoloLens connection.…”

First Impressions: A Visual Catalogue for Understanding Interactions with Novel Interfaces in Augmented Reality

“…The two most common AR experience in the marketplace are Mobile AR applications [36,37,38,39] and Head Mounted Displays (HMDs, such as [34,35,26]). Due to equipment costs, HMDS are typically used in more professional capacities [20,30,19,2].…”