Gyroscope-based video stabilisation with auto-calibration

Abstract

Abstract-We propose a technique for joint calibration of a wide-angle rolling shutter camera (e.g. a GoPro) and an externally mounted gyroscope. The calibrated parameters are time scaling and offset, relative pose between gyroscope and camera, and gyroscope bias. The parameters are found using non-linear least squares minimisation using the symmetric transfer error as cost function.The primary contribution is methods for robust initialisation of the relative pose and time offset, which are essential for convergence. We also introduce a robust error norm to handle outliers. This results in a technique that works with general video content and does not require any specific setup or calibration patterns.We apply our method to stabilisation of videos recorded by a rolling shutter camera, with a rigidly attached gyroscope. After recording, the gyroscope and camera are jointly calibrated using the recorded video itself. The recorded video can then be stabilised using the calibrated parameters.We evaluate the technique on video sequences with varying difficulty and motion frequency content. The experiments demonstrate that our method can be used to produce high quality stabilised videos even under difficult conditions, and that the proposed initialisation is shown to end up within the basin of attraction. We also show that a residual based on the symmetric transfer error is more accurate than residuals based on the recently proposed epipolar plane normal coplanarity constraint, and that the use of robust errors is a critical component to obtain an accurate calibration.

Beyond Correlation Filters: Learning Continuous Convolution Operators for Visual Tracking

“…Unlike object tracking, feature point tracking is the task of accurately estimating the motion of distinctive key-points. It is a core component in many vision systems [1,27,39,48]. Most feature point tracking methods are derived from the classic Kanade-Lucas-Tomasi (KLT) tracker [34,44].…”

Trajectory representation and landmark projection for continuous-time structure from motion

“…Using tracking instead of feature matching means that landmarks that are detected more than once will be tracked multiple times by the system. The camera-IMU extrinsics, gyroscope bias, and time offset, were given an initial estimate using the Crisp (Ovrén and Forssén 2015) toolbox. Since Crisp does not support accelerometer measurements, we then refined the initial estimate using Kontiki, described in section 5.1, by optimizing over a short part of the full sequence with the accelerometer bias as a parameter to optimize.…”

Deep Iterative Frame Interpolation for Full-frame Video Stabilization

“…3D methods model the camera trajectory in 3D space for stabilization. To stabilize camera motion, various techniques, including Structure from Motion (SfM) [Liu et al 2009], depth information [Liu et al 2012], 3D plane constraints [Zhou et al 2013], projective 3D reconstruction [Buehler et al 2001], light field [Smith et al 2009], and 3D rotation estimation via gyroscope [Bell et al 2014;Karpenko et al 2011;Ovrén and Forssén 2015], have been used. 2.5D approaches use partial information from 3D models, and can handle reconstruction failure cases.…”