Multi-View Instance Matching with Learned Geometric Soft-Constraints

Abstract

We present a new approach for matching urban object instances across multiple ground-level images for the ultimate goal of city-scale mapping of objects with high positioning accuracy. What makes this task challenging is the strong change in view-point, different lighting conditions, high similarity of neighboring objects, and variability in scale. We propose to turn object instance matching into a learning task, where image-appearance and geometric relationships between views fruitfully interact. Our approach constructs a Siamese convolutional neural network that learns to match two views of the same object given many candidate image cut-outs. In addition to image features, we propose utilizing location information about the camera and the object to support image evidence via soft geometric constraints. Our method is compared to existing patch matching methods to prove its edge over state-of-the-art. This takes us one step closer to the ultimate goal of city-wide object mapping from street-level imagery to benefit city administration.

Context Aware Object Geotagging

“…However, identical features may be mismatched in case the recurring objects sit nearby. To solve this issue, Nassar et al [20,21] employed the soft geometry constraint on geo-location of camera pose to identify a same object that appears in two views. They concatenate camera pose information together with image features and decode them using a CNN.…”