Horizon Lines in the Wild

Abstract

Single image horizon line estimation is one of the most fundamental geometric problems in computer vision. Knowledge of the horizon line enables a wide variety of applications, including: image metrology, geometry-aware object detection, and automatic perspective correction. Despite this demonstrated importance, progress on this task has stagnated. We believe the lack of a suitably large and diverse evaluation dataset is the primary cause. Existing datasets [2,3] are often small and were created to focus on evaluating methods that use a particular geometric cue (e.g., orthogonal vanishing points). Methods that perform well on such datasets often perform poorly in real-world conditions.We introduce Horizon Lines in the Wild (HLW), a new dataset for single image horizon line estimation. HLW is several orders of magnitude larger than any existing dataset for horizon line detection (containing 100 553 images), has a much wider variety of scenes and camera perspectives, and wasn't constructed to highlight the value of any particular geometric cue. The dataset (including models and sample code) is available for download at our project website [1].Using HLW, we investigate methods for directly estimating the horizon line using convolutional neural networks (CNNs), including both classification and regression formulations. We focus on the GoogleNet architecture and explore the impact of design and implementation choices on the accuracy of the resulting model. Additionally, we propose two post-processing strategies for aggregating horizon line estimates across subwindows.Our approach is fast, works in natural and man-made environments, does not fail catastrophically when vanishing point detection is difficult, and outperforms all existing methods on the challenging real-world imagery contained in HLW. Further, when combined with the recent method by Zhai et al. [4], which uses a CNN to provide global context for vanishing point esti- mation, we obtain state-of-the-art results on two existing benchmark datasets [2,3]. Our main contributions are: 1) a novel approach for using structure from motion to automatically label images with a horizon line, 2) a large evaluation dataset of images with labeled horizon lines, 3) a CNN-based approach for directly estimating the horizon line in a single image, and 4) an extensive evaluation of a variety of CNN design choices.[1] Horizon Lines in The Wild project website.

UprightNet: Geometry-Aware Camera Orientation Estimation From Single Images

“…• A regression baseline: a CNN that predicts roll and pitch angles, trained with an 1 loss. • DeepHorizon [48], a CNN classification approach to predicting horizon offsets and slopes. • Hold-Geoffroy et al [19]: a CNN classification method for predicting horizon lines and fields of view.…”

Neural-Guided RANSAC: Learning Where to Sample Model Hypotheses

“…As mentioned before, some images in the HLW dataset [52] have their horizon outside the image. Some of these images contain virtually no visual cue where the horizon exactly lies.…”

“…i.e. we use the square root of the task loss after a magnitude of 0.25, which is the magnitude up to which the AUC is calculated when evaluating on HLW [52]. Qualitative Results.…”

Blind Geometric Distortion Correction on Images Through Deep Learning

“…Recently, Zhai et al [43] proposed to use deep convolutional neural networks to estimate the horizon line by aggregating the global image context with the clue of the vanishing point. Workman et al [40] goes further and directly estimates the horizon line in the single image. Unlike these specialized methods, our approach is generalizable for multi-type distortion correction using a single network.…”