An augmented reality maintenance assistant with real-time quality inspection on handheld mobile devices

Abstract

With the advances of industry 4.0, augmented reality (AR) devices are being deployed across the manufacturing sector to enhance worker perception and e ciency. AR is often used to deliver spatially relevant work instructions on mobile devices for maintenance procedures on the factory oor. In these situations, workers use their mobile devices to view instructions in the form of 3D animations and annotations that directly overlay the equipment being maintained. Workers then follow the AR instructions and must ultimately rely on their own judgement and knowledge of the procedure as they progress from step to step. An AR assistant that could validate each stage of the procedure in real time and provide the worker with feedback on any observed errors would ensure that each maintenance procedure is completed successfully. This work presents a mobile, quality inspection system for AR maintenance procedures that is capable of assessing the maintenance task in real time. The system is designed for deployment on handheld mobile devices and can thus manage the challenges inherent to performing quality inspection with a non-xed vision system. This work enumerates four essential qualities of mobile quality inspection tools and outlines some of the challenges encountered during the development of such a system. In the end, testing established that the system could provide adequate assistance for capturing inspection images, accurately process the captured images using machine vision, and generate detailed feedback from the quality inspection in a timely manner.

Augmented Reality in Industry 4.0 Assistance and Training Areas: A Systematic Literature Review and Bibliometric Analysis

“…This approach is echoed in studies by Drouot et al [114] and Zhang et al [119], where AR is presented as a tool to reduce errors and improve efficiency in assembly processes, as well as to reduce the mental workloads of operators. This is echoed by Frandsen et al [131], who demonstrate the capability of AR for maintenance at the enterprise level by integrating real-time quality assessment into work instructions. This approach allows for the self-assessment of quality by maintenance personnel, which is consistent with the goal of using AR to reduce errors and improve efficiency in assembly processes.…”