Panoramic Stereo Imaging of a Bionic Compound-Eye Based on Binocular Vision

Abstract

With the rapid development of the virtual reality industry, one of the bottlenecks is the scarcity of video resources. How to capture high-definition panoramic video with depth information and real-time stereo display has become a key technical problem to be solved. In this paper, the optical optimization design scheme of panoramic imaging based on binocular stereo vision is proposed. Combined with the real-time processing algorithm of multi detector mosaic panoramic stereo imaging image, a panoramic stereo real-time imaging system is developed. Firstly, the optical optimization design scheme of panoramic imaging based on binocular stereo vision is proposed, and the space coordinate calibration platform of ultra-high precision panoramic camera based on theodolite angle compensation function is constructed. The projection matrix of adjacent cameras is obtained by solving the imaging principle of binocular stereo vision. Then, a real-time registration algorithm of multi-detector mosaic image and Lucas-Kanade optical flow method based on image segmentation are proposed to realize stereo matching and depth information estimation of panoramic imaging, and the estimation results are analyzed effectively. Experimental results show that the stereo matching time of panoramic imaging is 30 ms, the registration accuracy is 0.1 pixel, the edge information of depth map is clearer, and it can meet the imaging requirements of different lighting conditions.

Research on Panorama Generation from a Multi-Camera System by Object-Distance Estimation

“…The rigorous panoramic imaging model is further proposed by incorporating the eccentric vectors among the optical centers of the cameras into the panoramic imaging model. The rigorous model is theoretically more accurate than the ideal model [25]. However, both the ideal and rigorous panoramic imaging models are developed without consideration of the varied object-distance information in a scene.…”

“…With the development of sensors and computing abilities, many panoramic camera systems have been developed by a combination of multiple cameras [10][11][12][13][14][15][16][17][18][19][20][21][22][23][24], which can be divided into two categories: monocular panorama systems and binocular panorama systems. Although a binocular panorama system is able to provide the 3D scene information needed for seamless panorama generation and 3D reconstruction, it requires higher complexity of system design and processing than that of the monocular panorama system [25]. Therefore, the development of monocular panorama systems using multiple cameras and the generation of accurate panoramic images remain crucial in the fields of remote sensing and computer vision applications.…”

“…Panoramas are generated based on the second method, which relies on a panoramic imaging model derived from the pre-calibrated geometric relationships between cameras. Both the ideal and rigorous panoramic spherical and cylindrical imaging models have been proposed [24][25][26][27][28] and successfully applied for vehicle-based remote sensing mapping and panoramic SLAM 3D reconstruction, particularly with the professional panoramic camera LadyBug3 [25]. However, it is important to mention that both the ideal and rigorous panoramic imaging models are developed under the assumption of constant object-distance information in the surroundings.…”

Robot Bionic Vision Technologies: A Review

“…The optical optimization design scheme and algorithm for panoramic imaging based on binocular stereo vision were proposed, and a panoramic stereo real-time imaging system was developed [37].…”