Handheld 3-D Scanning with Automatic Multi-View Registration Based on Visual-Inertial Navigation

Abstract

In this article an approach to a mobile 3-D handheld scanner with additional sensory information is proposed. It fully automatically builds a multi-view 3-D scan. Conventionally complex post processing or expensive position trackers are used to realize such a process. Therefore a combination of a visual and inertial motion tracking system is developed to deal with the position tracking. Both sensors are integrated into the 3-D scanner and their data are fused for robustness during swift scanner movements and for long term stability. This article presents an overview over the system architecture, the navigation process, surface registration aspects, and measurement results.

A multi-view stereo based 3D hand-held scanning system using visual-inertial navigation and structured light

“…We extended the system described in [6] for 3D handheld scanning approach based on multiview stereo. The system proposed in [11] used a high dynamic range (HDR) image sensor besides stereo camera for visual pose estimation while we do not employ an extra camera for parameter estimation in our setup. The stereo camera utilizes a pair of Basler Ace2500 14/gm monochrome cameras connected with A100P Zeus pocket projector and mySen-C INS device.…”

“…Several researchers employed robotic manipulators, passive arms, turntables and electromagnetic devices to accomplish 3D handheld scanning, but these devices not only restrict the user's mobility and need accurate external hand-eye calibration but these external positioning systems are also considered to be the largest and most expensive part of 3D sensing systems [9].Despite various advantages of digital camera, the geometric and perspective geometry issues entangles the geometric information obtained from cameras thus making it hard to get real time pose estimations solely from image sensors and user may overcome these issues using inertial measurement unit (IMU) or inertial navigation system (INS) which is a better solution to digital camera in term of measuring rate and temporal precision [8].The purpose of INS is to estimate the relative pose and position of the system between different viewpoints and accomplish the multiview registration using these parameters [10,11].For visual inertial navigation, both the visual and inertial pose may be fused either in time or stochastically and one sensor may support pose estimation within another sensor's estimation process [8]. In this research, we propose a handheld 3D sensing system based on stereo camera, IMU and projector for close range applications.…”

Multiview registration-based handheld 3D profiling system using visual navigation and structured light

“…The problems of self-occlusion, object size, and limited field of view restrict the 3D modeling system to render the 3D model in single measurement and thus multiview integration approach is required [7] . The usage of robotic manipulators, turntables, electromagnetic devices, and passive arms limits user's mobility and requires high accuracy in external hand-eye calibration and also constitutes the largest and most expensive part of the 3D handheld scanner [7][8][9] . It is hard to get the real-time pose and position solely from image data as the geometric information from camera is entangled in the radiometric and perspective geometry issues [7] .…”

“…It is hard to get the real-time pose and position solely from image data as the geometric information from camera is entangled in the radiometric and perspective geometry issues [7] . Inertial measurement unit (IMU) solves this problem through estimating the relative orientation and translation between different views and registering the different view point clouds [9,10] .…”

“…This research utilized the second approach for the 3D modeling device. The literature also reported the visual inertial navigation approaches [8,9,[21][22][23] based on the fusion of visual and inertial poses. The research [21] reported the fusion of visual and inertial poses based on an extended Kalman filter (EKF) which considered the IMU and camera as "black boxes" to be operated independently.…”