Robust multiple homography estimation: An ill-solved problem

Abstract

The estimation of multiple homographies between two piecewise planar views of a rigid scene is often assumed to be a solved problem. We show that contrary to popular opinion various crucial aspects of the task have not been adequately emphasised. We are motivated by a growing body of literature in robust multi-structure estimation that purports to solve the multi-homography estimation problem but in fact does not. We demonstrate that the estimation of multiple homographies is an ill-solved problem by deriving new constraints that a set of mutually compatible homographies must satisfy, and by showing that homographies estimated with prevailing methods fail to satisfy the requisite constraints on real-world data. We also explain why incompatible homographies imply inconsistent epipolar geometries. The arguments and experiments presented in this paper signal the need for a new generation of robust multi-structure estimation methods that have the capacity to enforce constraints on projective entities such as homography matrices.

Full Explicit Consistency Constraints in Uncalibrated Multiple Homography Estimation

“…A fact that will be of significance in what follows is that if the generalised eigenvalue problem (8) has a double eigenvalue, then this eigenvalue is a double root of p A,B . For the sake of completeness, we recall the argument presented in [36] which validates this fact and correct a misprint that has slipped into the original proof.…”

“…These latter authors also derived constraints for larger sets of homographies and views. Finally, in recent work [36] Szpak et al introduced what they dubbed the multiplicity and singularity constraints that apply to two or more, and three or more, homographies between two views, respectively.…”

Temporally Robust Global Motion Compensation by Keypoint-Based Congealing

“…IS aims to minimize the distortions and ghosting artifact in the overlap region. Recent works focus on different challenges, e.g., multi-plane scenes [20][21][22][23][24][25], the parallax issue [26][27][28], and motion blur [29]. In these works, input images have much less overlap than GMC.…”

Fitting Multiple Heterogeneous Models by Multi-Class Cascaded T-Linkage

“…In this case MCT works almost like regular T-linkage, except a first clustering is made with F matrices that biases the subsequent sampling of homographies. As observed by[20], initial recovery of an F matrix can profitably guide the sam-…”