An Augmented Reality Maintenance Assistant with Real-time Quality Inspection on Handheld Mobile Devices

Abstract

With the advances of industry 4.0, augmented reality (AR) devices are being deployed across the manufacturing sector to enhance worker perception and efficiency. AR is often used to deliver spatially relevant work instructions on mobile devices for maintenance procedures on the factory floor. In these situations, workers use their mobile devices to view instructions in the form of 3D animations and annotations that directly overlay the equipment being maintained. Workers then follow the AR instructions and must ultimately rely on their own judgement and knowledge of the procedure as they progress from step to step. An AR assistant that could validate each stage of the procedure in real time and provide the worker with feedback on any observed errors would ensure that each maintenance procedure is completed successfully. This work presents a mobile, quality inspection system for AR maintenance procedures that is capable of assessing the maintenance task in real time. The system is designed for deployment on handheld mobile devices and can thus manage the challenges inherent to performing quality inspection with a non-fixed vision system. This work enumerates four essential qualities of mobile quality inspection tools and outlines some of the challenges encountered during the development of such a system. In the end, testing established that the system could provide adequate assistance for capturing inspection images, accurately process the captured images using machine vision, and generate detailed feedback from the quality inspection in a timely manner.