Real-Time Robust Stereo Visual SLAM System Based on Bionic Eyes

Liu, Yanqing; Zhu, Dongchen; Peng, Jingquan; Wang, Xianshun; Wang, Lei; Chen, Lili; Li, Jiamao; Zhang, Xiaolin

doi:10.1109/tmrb.2020.3011981

Cited by 12 publications

(6 citation statements)

References 36 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The eye imitation system of Tokyo Institute of Technology can realize multiple human eye movements at the same time, but it is only one-dimensional horizontal motion. Therefore, the human eye imitation system is not perfect [6][7][8][9][10][11][12][13][14][15][16][17]. In 2012, Yi-Chiao Lee came up with a two-degree-of-freedom camera with parallel motors.…”

Section: Status Of Bionic Eye Researchmentioning

confidence: 99%

Robot Bionic Eye Motion Posture Control System

Zhang¹,

Lee

2023

Electronics

View full text Add to dashboard Cite

This paper mainly studies the structure and system of robot bionic eye. Given that most robots usually run on battery power, the STM32L053C8T6 with high efficiency and super low power consumption was selected as the main control. By carrying IMU, the bionic eye attitude data can be acquired quickly and accurately, and the measurement data of accelerometer and gyroscope can be fused by the algorithm to obtain stable and accurate bionic eye attitude data. Thus, the precise control of the motor can be realized through the drive control system equipped with PCA9685, which can enhance the motion control precision of robot bionic eye. In the present study, three kinds of IMU sensors, MPU6050, MPU9250, and WT9011G4K, were selected to carry out experiments. Finally, MPU9250 with better power consumption and adaptability is selected. This is the attitude acquisition device of bionic eye. In addition, three different filters, CF, GD, and EKF, were used for data fusion and comparison. The experimental result showed that the dynamic mean errors of CF, GD, and EKF are 0.62°, 0.61°, and 0.43°, respectively, and the static mean errors are 0.1017°, 0.1001°, and 0.0462°, respectively. The result showed that, after the use of EKF, the robot bionic eye system designed in this paper can significantly reduce the attitude angle error and effectively improve the image quality. It ensures accuracy and reduces power consumption and cost, which has lower requirements on hardware and is easier to popularize.

show abstract

Section: Status Of Bionic Eye Researchmentioning

confidence: 99%

Robot Bionic Eye Motion Posture Control System

Zhang¹,

Lee

2023

Electronics

View full text Add to dashboard Cite

show abstract

“…This technique requires at least two or more imaging devices to simultaneously acquire image information of the target [23][24][25][26]. It is a bionic design inspired by the compound eye of insects or the eyes of humans [27,28]. It works by comparing the disparity between different image positions and then converting it into thickness information to complete the 3D reconstruction of the target.…”

Section: Multi-cameramentioning

confidence: 99%

A single camera unit-based three-dimensional surface imaging technique

Wang

Guo

Kim

et al. 2023

Int J Adv Manuf Technol

View full text Add to dashboard Cite

The main objective of this study is to develop a single-camera unit-based three-dimensional surface imaging technique that could be used to reduce the disparity error in three-dimensional (3D) image reconstruction and simplify the calibration process of the imaging system. The current advanced stereoscopic 3D imaging system uses a pair of imaging devices (e.g., complementary metal-oxide semiconductor (CMOS) or charge-coupled device (CCD)), imaging lenses, and other accessories (e.g., light sources, polarizing filters) and diffusers.) To reconstruct the 3D scene, the system needs to calibrate the camera and compute a disparity map. However, in most cases in the industry, a pair of imaging devices is not ideally identical, so it is a necessary step to finely adjust and compensate for camera orientation, lens focal length, and intrinsic parameters for each camera.More importantly, conventional stereoscopic systems may respond differently to incident light reflected from the target surface. It is possible for the pixel information in the left and right images to be slightly different. This results in an increase in disparity error, even though the stereovision system is calibrated and compensated for rotation and vertical offsets between two cameras. This thesis aims to solve the aforementioned challenges by proposing a new stereo vision scheme based on only one camera to obtain target 3D data by 3D image reconstruction of two images obtained from two different camera positions.iii DEDICATION This thesis is dedicated to my family, who always supports me and prays for my success.

show abstract

“…A robot that can execute these two functions will be more intelligent. Such a robot would need to exploit the smooth pursuit of eyes [ 16 ], coordinated eye–head movement [ 17 ], target detection and the combined movement of the eyes, head and robot body to carry out these two functions. Studies have achieved many positive results in these aspects.…”

Section: Introductionmentioning

confidence: 99%

“…Second, only two-dimensional (2D) image information is applied when gaze shifts to targets are implemented, while 3D information is ignored. Third, the studies of smooth pursuit [ 16 ], eye–head coordination [ 17 ], gaze shift and approach are independent and have not been integrated. Fourth, bionic eyes are different from human eyes; for example, some of them are two eyes that are fixed without movement or move with only 1 DOF, whereas some of them use special cameras or a single camera.…”

Section: Introductionmentioning

confidence: 99%

Gaze Point Tracking Based on a Robotic Body–Head–Eye Coordination Method

Feng¹,

Wang

Huang³

et al. 2023

Sensors

View full text Add to dashboard Cite

When the magnitude of a gaze is too large, human beings change the orientation of their head or body to assist their eyes in tracking targets because saccade alone is insufficient to keep a target at the center region of the retina. To make a robot gaze at targets rapidly and stably (as a human does), it is necessary to design a body–head–eye coordinated motion control strategy. A robot system equipped with eyes and a head is designed in this paper. Gaze point tracking problems are divided into two sub-problems: in situ gaze point tracking and approaching gaze point tracking. In the in situ gaze tracking state, the desired positions of the eye, head and body are calculated on the basis of minimizing resource consumption and maximizing stability. In the approaching gaze point tracking state, the robot is expected to approach the object at a zero angle. In the process of tracking, the three-dimensional (3D) coordinates of the object are obtained by the bionic eye and then converted to the head coordinate system and the mobile robot coordinate system. The desired positions of the head, eyes and body are obtained according to the object’s 3D coordinates. Then, using sophisticated motor control methods, the head, eyes and body are controlled to the desired position. This method avoids the complex process of adjusting control parameters and does not require the design of complex control algorithms. Based on this strategy, in situ gaze point tracking and approaching gaze point tracking experiments are performed by the robot. The experimental results show that body–head–eye coordination gaze point tracking based on the 3D coordinates of an object is feasible. This paper provides a new method that differs from the traditional two-dimensional image-based method for robotic body–head–eye gaze point tracking.

show abstract

Real-Time Robust Stereo Visual SLAM System Based on Bionic Eyes

Cited by 12 publications

References 36 publications

Robot Bionic Eye Motion Posture Control System

Robot Bionic Eye Motion Posture Control System

A single camera unit-based three-dimensional surface imaging technique

Gaze Point Tracking Based on a Robotic Body–Head–Eye Coordination Method

Contact Info

Product

Resources

About