3rd Place: A Global and Local Dual Retrieval Solution to Facebook AI Image Similarity Challenge

Abstract

As a basic task of computer vision, image similarity retrieval is facing the challenge of large-scale data and image copy attacks. This paper presents our 3rd place solution to the matching track of Image Similarity Challenge (ISC) 2021 organized by Facebook AI. We propose a multi-branch retrieval method of combining global descriptors and local descriptors to cover all attack cases. Specifically, we attempt many strategies to optimize global descriptors, including abundant data augmentations, selfsupervised learning with a single Transformer model, overlay detection preprocessing. Moreover, we introduce the robust SIFT feature and GPU Faiss for local retrieval which makes up for the shortcomings of the global retrieval. Finally, KNN-matching algorithm is used to judge the match and merge scores. We show some ablation experiments of our method, which reveals the complementary advantages of global and local features.

A Benchmark and Asymmetrical-Similarity Learning for Practical Image Copy Detection

“…Another difference from DISC21 is that the NDEC prohibits training on the query and reference sets. We note that some state-of-the-art methods (Yokoo 2021;Sun et al 2021) actually use 25, 000 queries from DISC21 for training and achieves extra benefits. However, in realistic ICD, using queries for training is not quite feasible.…”

“…NDEC significantly challenges the state-of-the-art methods. We further benchmark NDEC with state-of-the-art methods (Wang et al 2021a;Sun et al 2021;Yokoo 2021;Papadakis and Addicam 2021;Wang et al 2021b) (top performing methods in DISC2021 (Douze et al 2021)) in Fig. 7.…”