An Evaluation of Recent Local Image Descriptors for Real-World Applications of Image Matching

Abstract

This paper discusses and compares the best and most recent local descriptors, evaluating them on increasingly complex image matching tasks, encompassing planar and non-planar scenarios under severe viewpoint changes. This evaluation, aimed at assessing descriptor suitability for real-world applications, leverages the concept of Approximated Overlap error as a means to naturally extend to non-planar scenes the standard metric used for planar scenes. According to the evaluation results, most descriptors exhibit a gradual performance degradation in the transition from planar to non-planar scenes. The best descriptors are those capable of capturing well not only the local image context, but also the global scene structure. Data-driven approaches are shown to have reached the matching robustness and accuracy of the best hand-crafted descriptors.

Which Is Which? Evaluation of Local Descriptors for Image Matching in Real-World Scenarios

“…With respect to a recent evaluation using a very similar setup protocol [3], some differences in the descriptor relative rank can be noted (e.g. RootSIFT in WISW evaluation is better than RFD, as opposed to what reported in [3]).…”

“…However, none of these approaches is without drawbacks, since ground-truth may not be available for some image region (a,b), it can be erroneously estimated (a,c,d), or it can be biased towards the considered application (a,d). In WISW, the benchmark for non-planar scenes first introduced in [3], and based on a piecewise approximation of the overlap error, is used. As shown in Sec.…”

“…The dataset for the non-planar case contains images from 35 different scenes used in other works (19 having 3 images, the remaining 16 with 2 images only), for a total of 19 × (3 − 1) + 16 × (2 − 1) = 73 image pairs. The dataset extends the one used in [3], which included only 42 image pairs (see the first two rows of Fig. 4 for some examples).…”

“…4 for some examples). Notice that in addition to the images to probe, this dataset contains reference matches that are employed both for computing the approximated overlap error and for refining correspondences according to their local flow [3]. Evaluation protocol.…”

“…Evaluation protocol. Non-planar scene evaluation follows the protocol described in [3], employing the approximated overlap error defined in [6] for computing the ground-truth. Unlike other non-planar benchmarks, the approximated overlap relies on the whole local descriptor patch and not on the keypoint position only, thus being a natural extension of the overlap error to the non-planar case.…”