Feature‐based recursive observer design for homography estimation and its application to image stabilization

Abstract

This paper presents a new algorithm for online estimation of a sequence of homographies applicable to image sequences obtained from robotic vehicles equipped with vision sensors. The approach taken exploits the underlying Special Linear group structure of the set of homographies along with gyroscope measurements and direct point-feature correspondences between images to develop temporal filter for the homography estimate. Theoretical analysis and experimental results are provided to demonstrate the robustness of the proposed algorithm. The experimental results show excellent performance and robustness even in the case of very fast camera motions (relative to frame rate) and severe occlusions.

Attitude, Linear Velocity and Depth Estimation of a Camera Observing a Planar Target Using Continuous Homography and Inertial Data

“…According to [8,Corollary 3.2], good conditioning of the solutions P (t) to the CREs and exponential stability of the proposed observer rely on the uniform observability of the pair (A (t), C (t)) obtained by setting x = 0 in the expressions of the matrices A and C given by (13). One verifies that…”

“…Since the quality of the continuous homography estimation depends heavily on the capability of rejecting outliers of point matchings, we have implemented an M-estimator-like observer for the estimation of the homography between every two consecutive images, which is then used to compute the continuous homography via a logarithm conversion. This M-estimator-like homography observer is a modified version of the homography observer proposed in [13] but is not presented here due to space limitation. However, the reader can appreciate its performance and robustness via the following video link: https://youtu.be/x75RpjoJ9HM Although the combined implementation of image processing and estimation algorithm runs at about 50Hz, the continuous homography estimate is only obtained at every 50ms (20Hz) due to the lower acquisition frequency of the VI-sensor camera.…”

A Homography-Based Dynamic Control Approach Applied to Station Keeping of Autonomous Underwater Vehicles Without Linear Velocity Measurements

“…The vehicle is equipped with three vertical thrusters for heave, pitch and roll actuations, two horizontal thrusters for yaw and surge . The homography estimation algorithm [12] has been implemented in C++, combined with OpenCV for image processing, on the ground station laptop with an Intel Core i7-7700HQ octa-core CPU running at 2.8 GHz. The transmission of the data from the camera and the IMU to the ground station is carried out through the high speed Ethernet cable.…”

Nonlinear observer design on SL(3) for homography estimation by exploiting point and line correspondences with application to image stabilization

“…These are just a very nonexhaustive list of examples. On the other hand, other less explored research directions involve the Special Linear group SL(3) for homography estimation problem (Malis, Hamel, Mahony & Morin 2009, Mahony, Hamel, Morin & Malis 2012, Hamel, Mahony, Trumpf, Morin & Hua 2011, Hua, Hamel, Mahony & Allibert 2017, Hua, Trumpf, Hamel, Mahony & Morin 2019 and some newly introduced Lie groups for the Simultaneous Localization and Mapping (SLAM) problem such as the SLAM n (3) group , the VSLAM n (3) group (van Goor, Mahony, Hamel & Trumpf 2019), and the SE n+1 (2) group (Barrau & Bonnabel 2015).…”

“…The present paper addresses the problem of nonlinear observer design on SL(3) with application to homography estimation and image stabilization. To our knowledge, most existing works on the topic have been proposed by the authors of this paper (Malis et al 2009, Hua, Hamel, Mahony & Allibert 2017, Hua et al 2019. Before providing further discussions, let us first recall on the homography and its importance in practice.…”