Ctrl-VIO: Continuous-Time Visual-Inertial Odometry for Rolling Shutter Cameras

Lang, Xiaolei; Lv, Jiajun; Huang, Jianxin; Ma, Yukai; Liu, Yong; Zuo, Xingxing

doi:10.1109/lra.2022.3202349

Cited by 15 publications

(11 citation statements)

References 64 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Another high-dimensional representation is to parameterize the camera trajectory with B-splines. The methods of [14] and Ctrl-VIO [9] include a continuoustime representation with B-splines, raising the dimension of the RS modeling. Ctrl-VIO introduces more B-spline control points into the state vector for optimization, which greatly increases the computational cost.…”

Section: Related Workmentioning

confidence: 99%

“…According to equation (9), the fundamental matrix between the GS image and RS image is unknown. Thus, we need to acquire the RS pose of the nth row of the RS image.…”

Section: Msckf Feature Correctionmentioning

confidence: 99%

“…Low-dimensional representations such as the constant velocity model [7] introduce modeling errors that reduce the accuracy of pose estimations. High-dimensional representations such as constant acceleration models [8] or B-splines [9] introduce more variables that need to be optimized, resulting in a massively increased computational cost.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Point feature correction based rolling shutter modeling for EKF-based visual-inertial odometry

Zhang,

Zhang

2023

Meas. Sci. Technol.

View full text Add to dashboard Cite

The application of consumer-grade rolling-shutter (RS) camera in VIO system deserves attention. In previous works, VIO systems with RS camera usually adopt different dimensions of RS modeling to represent the camera motion. Although the complex camera motion expression improves the accuracy, adding more variables greatly increases the computation load. Therefore, considering the limitations of camera motion modeling, this paper introduces a point feature correction strategy to improve performance of the system in both speed and accuracy. The optimized camera poses and high-frequency IMU are employed to correct the point features on RS image to new pixel coordinates in global-shutter view (at the middle of RS image readout time) by epipolar transfer. At the same time, we handle the zero velocity and abnormal cases of RS modeling to improve the robustness of the system. Furthermore, the readout time of the RS camera is online self-calibrated in the stage of state augmentation and can quickly converge and stabilize to near the true value. And the readout time of the RS camera is locally observable, except in two degenerate motions.  Experimental results demonstrate that the proposed method outperforms the state-of-the-art methods on a public dataset in terms of accuracy and computational cost. We port our algorithm to android-based mobile phone and our system outputs real-time trajectories on mobile phone. Then, we use the motion capture system to obtain the ground truth phone pose to test the performance of our algorithm.

show abstract

Section: Related Workmentioning

confidence: 99%

“…According to equation (9), the fundamental matrix between the GS image and RS image is unknown. Thus, we need to acquire the RS pose of the nth row of the RS image.…”

Section: Msckf Feature Correctionmentioning

confidence: 99%

See 1 more Smart Citation

Point feature correction based rolling shutter modeling for EKF-based visual-inertial odometry

Zhang,

Zhang

2023

Meas. Sci. Technol.

View full text Add to dashboard Cite

show abstract

“…In order to fuse the heterogeneous sensors, accurate alignment of the asynchronous sensors is the prerequisite. Without special hardware support for synchronization, timeoffset can also be online calibrated in estimators, such as online timeoffset estimation in Visual-Inertial(VI) system [9,[11][12][13] and LIC system [4]. Further efforts are required to deal with different sensor frequencies and sampling time instants.…”

Section: Introductionmentioning

confidence: 99%

Continuous-Time Fixed-Lag Smoothing for LiDAR-Inertial-Camera SLAM

Lang

et al. 2023

IEEE/ASME Trans. Mechatron.

View full text Add to dashboard Cite

Localization and mapping with heterogeneous multisensor fusion have been prevalent in recent years. To adequately fuse multi-modal sensor measurements received at different time instants and different frequencies, we estimate the continuoustime trajectory by fixed-lag smoothing within a factor-graph optimization framework. With the continuous-time formulation, we can query poses at any time instants corresponding to the sensor measurements. To bound the computation complexity of the continuous-time fixed-lag smoother, we maintain temporal and keyframe sliding windows with constant size, and probabilistically marginalize out control points of the trajectory and other states, which allows preserving prior information for future sliding-window optimization. Based on continuoustime fixed-lag smoothing, we design tightly-coupled multi-modal SLAM algorithms with a variety of sensor combinations, like the LiDAR-inertial and LiDAR-inertial-camera SLAM systems, in which online timeoffset calibration is also naturally supported. More importantly, benefiting from the marginalization and our derived analytical Jacobians for optimization, the proposed continuous-time SLAM systems can achieve real-time performance regardless of the high complexity of continuoustime formulation. The proposed multi-modal SLAM systems have been widely evaluated on three public datasets and selfcollect datasets. The results demonstrate that the proposed continuous-time SLAM systems can achieve high-accuracy pose estimations and outperform existing state-of-the-art methods. To benefit the research community, we will open source our code at https://github.com/APRIL-ZJU/clic.

show abstract

“…In recent years, with the development of computer vision technology, simultaneous localization and mapping (SLAM) based in cameras has been developed rapidly, where visual inertial odometry (VIO) uses measurements from vision sensors and inertial measurement units (IMU) to determine the relative motion of a carrier [ 1 , 2 ]. The camera can capture rich environmental information, while the IMU can provide accurate position estimation in a short period of time, and both have the characteristics of small size and low power consumption.…”

Section: Introductionmentioning

confidence: 99%

A Novel Deep Learning Approach to 5G CSI/Geomagnetism/VIO Fused Indoor Localization

Yang

Cheng

Jia

et al. 2023

Sensors

View full text Add to dashboard Cite

For positioning tasks of mobile robots in indoor environments, the emerging positioning technique based on visual inertial odometry (VIO) is heavily influenced by light and suffers from cumulative errors, which cannot meet the requirements of long-term navigation and positioning. In contrast, positioning techniques that rely on indoor signal sources such as 5G and geomagnetism can provide drift-free global positioning results, but their overall positioning accuracy is low. In order to obtain higher precision and more reliable positioning, this paper proposes a fused 5G/geomagnetism/VIO indoor localization method. Firstly, the error back propagation neural network (BPNN) model is used to fuse 5G and geomagnetic signals to obtain more reliable global positioning results; secondly, the conversion relationship from VIO local positioning results to the global coordinate system is established through the least squares principle; and finally, a fused 5G/geomagnetism/VIO localization system based on the error state extended Kalman filter (ES-EKF) is constructed. The experimental results show that the 5G/geomagnetism fusion localization method overcomes the problem of low accuracy of single sensor localization and can provide more accurate global localization results. Additionally, after fusing the local and global positioning results, the average positioning error of the mobile robot in the two scenarios is 0.61 m and 0.72 m. Compared with the VINS-mono algorithm, our approach improves the average positioning accuracy in indoor environments by 69.0% and 67.2%, respectively.

show abstract

Ctrl-VIO: Continuous-Time Visual-Inertial Odometry for Rolling Shutter Cameras

Cited by 15 publications

References 64 publications

Point feature correction based rolling shutter modeling for EKF-based visual-inertial odometry

Point feature correction based rolling shutter modeling for EKF-based visual-inertial odometry

Continuous-Time Fixed-Lag Smoothing for LiDAR-Inertial-Camera SLAM

A Novel Deep Learning Approach to 5G CSI/Geomagnetism/VIO Fused Indoor Localization

Contact Info

Product

Resources

About