Rolling Shutter Pose and Ego-Motion Estimation Using Shape-from-Template

Lao, Yizhen; Ait-Aider, Omar; Bartoli, Adrien

doi:10.1007/978-3-030-01216-8_29

Cited by 13 publications

(9 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This paper represents an extension of our previous work (Lao et al 2018) where we use SfT to solve RSAP. We here extend this principle to RSSfM.…”

Section: Contribution and Paper Organizationmentioning

confidence: 99%

“…We have previously shown that the RS effect can be explained by the GS projection of a virtually deformed shape which led to the analogy between the RSAP problem and SfT (Lao et al 2018). We also proposed a novel RSAP method which first recovers the virtual template deformation using SfT and then computes the pose and ego-motion parameters using a new 3D-3D registration method.…”

Section: Summary Of Contributionsmentioning

confidence: 99%

“…10, which first recovers the virtual deformed structure for each input RS image using NRSfM and then computes the actual rigid structure, camera pose and kinematics using a new 3D-3D registration method. • Together with our recent conference publication (Lao et al 2018), we bring a general unified framework to solve RS 3D vision problems which consists in two main steps, namely relaxation and upgrade.…”

Section: Summary Of Contributionsmentioning

confidence: 99%

“…SfT has already be made very fast in (Collins and Bartoli 2015;Magnenat et al 2015;Famouri et al 2018). Since the our previous work (Lao et al 2018), we have reimplemented our method using a realtime version of SfT. On average, it took around 0.91s for IsoRSAP (0.01s for isometric reconstruction and 0.9s for 3D-3D registration) and 11.6s for ConRSAP (10.6s for conformal reconstruction and 1.2s for 3D-3D registration).…”

Section: Running Timementioning

confidence: 99%

See 3 more Smart Citations

Solving Rolling Shutter 3D Vision Problems using Analogies with Non-rigidity

2020

Self Cite

View full text Add to dashboard Cite

We propose an original approach to absolute pose (AP) and Structure-from-Motion (SfM) which handles Rolling Shutter (RS) effects. Unlike most existing methods which either augment global shutter (GS) projection with velocity parameters or impose continuous time and motion through pose interpolation, we use local differential constraints. These are established by drawing analogies with non-rigid 3D vision techniques, namely Shape-from-Template (SfT) and Non-Rigid SfM (NRSfM).The proposed idea is to interpret the images of a rigid surface acquired by a moving RS camera as those of a virtually deformed surface taken by a GS camera. These virtually deformed surfaces are first recovered by relaxing the RS constraint using SfT or NRSfM. Then we upgrade the virtually deformed surface to the actual rigid structure and compute the camera pose and ego-motion by reintroducing the RS constraint. This uses a new 3D-3D registration procedure that minimizes a cost function based on the Euclidean 3D point distance. This is more stable and physically meaningful than the reprojection error or the algebraic distance used in previous work. Experimental results obtained with synthetic and real data show that the proposed methods outper-

show abstract

“…This paper represents an extension of our previous work (Lao et al 2018) where we use SfT to solve RSAP. We here extend this principle to RSSfM.…”

Section: Contribution and Paper Organizationmentioning

confidence: 99%

Section: Summary Of Contributionsmentioning

confidence: 99%

Section: Summary Of Contributionsmentioning

confidence: 99%

Section: Running Timementioning

confidence: 99%

See 2 more Smart Citations

Solving Rolling Shutter 3D Vision Problems using Analogies with Non-rigidity

2020

Self Cite

View full text Add to dashboard Cite

show abstract

“…Ignoring the effects of RS in computer vision applications results in performance degradation or even failure [2], [3]. Over the last decade, several works have revisited 3D computer vision by taking RS into account such as RS effects removal [4], [5], [6], [7], absolute pose estimation [2], [8], [9], [10], epipolar geometry [3], [11] and Structure from motion (SfM) [12], [13], [14], [15], [16], [17].…”

Section: Introductionmentioning

confidence: 99%

Rolling Shutter Homography and its Applications

Lao

Ait-Aider²

2021

IEEE Trans. Pattern Anal. Mach. Intell.

Self Cite

View full text Add to dashboard Cite

In this article we study the adaptation of the concept of homography to Rolling Shutter (RS) images. This extension has never been clearly adressed despite the many roles played by the homography matrix in multi-view geometry. We first show that a direct point-to-point relationship on a RS pair can be expressed as a set of 3 to 8 atomic 3x3 matrices depending on the kinematic model used for the instantaneous-motion during image acquisition. We call this group of matrices the RS Homography. We then propose linear solvers for the computation of these matrices using point correspondences. Finally, we derive linear and closed form solutions for two famous problems in computer vision in the case of RS images: image stitching and plane-based relative pose computation. Extensive experiments with both synthetic and real data from public benchmarks show that the proposed methods outperform state-of-art techniques.

show abstract