Multimodal training and tele-assistance systems for the maintenance of industrial products

Abstract

The paper describes an application based on Virtual and Augmented Reality technologies specifically developed to support maintenance operations of industrial products. Two scenarios have been proposed. In the first an operator learns how to perform a maintenance operation in a multimodal Virtual Reality environment that mixes a traditional instruction manual with the simulation, based on visual and haptic technologies, of the maintenance training task. In the second scenario, a skilled user operating in a multimodal Virtual Reality environment can remotely train another operator who sees the instructions about how the operations should be correctly performed, which are superimposed onto the real product. The paper presents the development of the application as well as its testing with users. Furthermore limits and potentialities of the use of Virtual and Augmented Reality technologies for training operators for product maintenance are discussed.

Augmented reality technology in the manufacturing industry: A review of the last decade

“…Studies dealing with HMI have been developed by and Tegeltija et al (2016). The technical solutions that fall among the intermittent concepts embrace the use of haptic devices Ferrise et al, 2013;Webel et al 2013) and the development of user interfaces (Henderson and Feiner, 2011;Gola nski et al, 2014;Chen, Chi, Kang and Hsieh, 2016).…”

An augmented reality based system for remote collaborative maintenance instruction of complex products

“…The service enabled a support person to point to a real object in a remote site by overlaying a virtual object, which is named as "air stamp," onto video image. Ferrise et al [20] gave skilled operators the possibility to share their expertise with remote operators at the occurrence and in a more natural way. The operator performing the maintenance task informed the skilled operator of the status of the operation, and the skilled one can teach him how to correctly perform the task.…”

Virtual co-location to support remote assistance for inflight maintenance in ground training for space missions

“…Ferrise et al [9] tackle the domain of maintenance operations of industrial product. In addition to using VR technology to support an operator to learn performing maintenance operations by combining traditional instruction manuals with simulation, AR technology is employed to extend the scenario to tele-assistance.…”