MANTRA: An Effective System Based on Augmented Reality and Infrared Thermography for Industrial Maintenance

Abstract

In recent years, the benefits of both Augmented Reality (AR) technology and infrared thermography (IRT) have been demonstrated in the industrial maintenance sector, allowing maintenance operations to be carried out in a safer, faster, and more efficient manner. However, there still exists no solution that optimally combines both technologies. In this work, we propose a new AR system—MANTRA—with specific application to industrial maintenance. The system can automatically align virtual information and temperature on any 3D object, in real time. This is achieved through the joint use of an RGB-D sensor and an IRT camera, leading to high accuracy and robustness. To achieve this objective, a pose estimation method that combines a deep-learning-based object detection method, YOLOV4, together with the template-based LINEMOD pose estimation method, as well as a model-based 6DOF pose tracking technique, was developed. The MANTRA system is validated both quantitatively and qualitatively through a real use-case, demonstrating the effectiveness of the system compared to traditional methods and those using only AR.

Thermographic Fault Diagnosis of Ventilation in BLDC Motors

“…Fault diagnosis based on thermal imaging is non-contact and non-invasive. It is a highly effective method of fault diagnosis [13][14][15][16][17][18][19][20][21]. However, the high cost of thermal imaging cameras is a limitation for fault diagnosis applications.…”

“…The following parameters were found for motors with corroded permanent magnets (reduction of 0.15 mm): reduced efficiency by 8% and reduced rated speed by 70 rpm [19]. A review of the industrial applications of infrared thermography is presented in [20,21].…”

Design of Augmented Reality Training Content for Railway Vehicle Maintenance Focusing on the Axle-Mounted Disc Brake System

“…As a result, it proved that the manual improved maintenance efficiency. Ortega et al [7] presented a system called Mantra, which was combined with AR technologies. They demonstrated the effectiveness of the system based on real use cases by comparing it with an existing method and applying the system to industrial maintenance.…”

“…Users can interact with a virtual object overlaid on a real image [15]. AR-based maintenance content has been used in a limited range of fields (e.g., industrial sites, the aviation industry, and the automobile industry) [5][6][7][8]. However, AR content specialized for railway vehicle maintenance is very scarce.…”

IoV-Based Collision Avoidance Architecture Using Machine Learning Prediction

“…In section II, the overview of the IoV-based collision avoidance architecture proposed in [1], [2] is gived, which can eliminate the assumption that every vehicle in the considered scenario should be with communication ability. In section III, we start to introduce our idea detailedly, which cooperates with YOLOv4 [19], [20]. Particularly, we will use videos recorded above a road to simulate the view that an infrastructure has observed.…”