An augmented reality based system for remote collaborative maintenance instruction of complex products

Abstract

Product maintenance is a knowledge intensive task. With the increasing complexity of modern large scale products, the maintenance process usually is carried out collaboratively by onsite technicians and remote experts. Augmented reality provides a novel interactive way for collaborative maintenance work. In an augmented instruction scene, the real maintenance product is combined with visual information and thus helps the technician to understand the process, which will be more efficient than traditional maintenance manual or oral words. In order to reduce the data quantity transferred between the local technician site and remote expert site, a framework of augmented reality based collaborative maintenance real-time instruction platform under network environment is proposed to achieve efficiently guidance from remote expert. The augmented information for maintenance process instruction is classified and their visualization methods are realized. During the collaborative maintenance process, the instruction information is shared based on different patterns. A prototyping system is developed for onsite technician and remote expert. The augmented collaborative maintenance process instruction is illustrated by replacing an air filter of an auto engine.

Assembly oriented control algorithm of collaborative disassembly and assembly operation in collaborative virtual maintenance process

“…According to the description of expression (13), it can be seen that with the iterative multiplication of the row elements of matrixes DOM and RDM, when the sub information matrix of the ith component PM i K i has been processed, the sub information matrix of the (i + 1)th component PM i+1 K i+1 will become the first row element of the residual matrix RDM. Therefore, in the disassembly operation process of CVM, multi operators and trainees can acquire the relevant disassembly information of the next (i + 1)th component when they accomplished the collaborative disassembly operation of the current ith component of the assembly.…”

“…The studies on key techniques of CVM and the developments of training systems are gradually concerned by many researchers [7][8][9][10][11][12][13]. However, the research fields and contents involved in CVM are quite extensive and complicated, and the related supporting technologies and the application developments need to be studied further.…”

6DoF Object Tracking based on 3D Scans for Augmented Reality Remote Live Support

“…These solutions are either solely showing augmentations in a 2D static manner [37] or they use outdated methods like marker tracking [30,38]. Recently, in [39] a Remote Live Support system was presented based on a client-server architecture that uses the Vuforia tracking framework but requires user collaboration in setting up the tracker by taking pictures from different views around the object.…”

Automatic generation of augmented reality guided assembly instructions using expert demonstration

“…This would not be suitable on large-scale processes, as a large number of human assemblers would require a large number of subject matter experts ready to provide feedback. Similar research was conducted by Wang [128] but with the addition of animated 3D models.…”