Line-based initialization method for mobile augmented reality in aircraft assembly

Abstract

A fast line-based initialization method for mobile augmented reality in aircraft assembly is proposed in this paper. The challenge of recognition and pose estimation of aircraft structural parts in monocular images on mobile devices is mainly addressed. Rather than local feature points, straight lines are extracted and utilized here since they are more suitable for aircraft structural parts, which usually have a solid color and lack textures to generate invariant descriptors. The geometric relationship between 3D world lines and their projections on the camera image are built to estimate the relative 6-DOF pose including rotation and translation, respectively. The inertial sensors of mobile devices are used to provide partial rotation information directly, and the rest is obtained through a voting process. To shrink the search space and, therefore, improve the speed and robustness of initialization, a color-based ROI mask and a vertical lines extraction method are proposed. To determine the translation, the color mask is first used to roughly approximate the real pose and then a possible-correspondence map of lines is built by parallelism alignment combined with the rotation already obtained. This paper describes the details of our method and also shows its effectiveness with experiments. Moreover, the proposed method is especially suitable for but not limited to the aircraft assembly field.

Augmented reality in support of Industry 4.0—Implementation challenges and success factors

“…Within different industries, the range of applications is broad. AR can be used for design and manufacturing applications [14], assembly operations, either in training [6,15] or as an online guidance system for operators [9,16,17]. In logistic, 'pick-byvision' is a prominent concept utilising AR to indicate picking locations and quantities [18][19][20][21][22][23].…”

Bare-Hand Depth Perception Used in Augmented Reality Assembly Supporting

“…Therefore, the registration methods only dependent on natural features in the scene were researched. Han and Zhao [1] thought mechanical parts usually had straight-line features. It could be used in aircraft structural parts recognition and pose estimation.…”

Augmented reality in support of intelligent manufacturing – A systematic literature review

“…Natural markers within buildings have been proposed as a solution for such a problem but are not widespread in research. While marker-less tracking has the advantage that no markers need to be placed within the area the system is used, it was still in an early stage of (Radkowski and Stritzke 2012); (Real and Marcelino 2011); (Han and Zhao 2017;Wu and Wang 2011). Future research can focus on the new possibilities in manufacturing applications due to recent technological advancements.…”