Single- and Cross-Modality Near Duplicate Image Pairs Detection via Spatial Transformer Comparing CNN

Abstract

Recently, both single modality and cross modality near-duplicate image detection tasks have received wide attention in the community of pattern recognition and computer vision. Existing deep neural networks-based methods have achieved remarkable performance in this task. However, most of the methods mainly focus on the learning of each image from the image pair, thus leading to less use of the information between the near duplicate image pairs to some extent. In this paper, to make more use of the correlations between image pairs, we propose a spatial transformer comparing convolutional neural network (CNN) model to compare near-duplicate image pairs. Specifically, we firstly propose a comparing CNN framework, which is equipped with a cross-stream to fully learn the correlation information between image pairs, while considering the features of each image. Furthermore, to deal with the local deformations led by cropping, translation, scaling, and non-rigid transformations, we additionally introduce a spatial transformer comparing CNN model by incorporating a spatial transformer module to the comparing CNN architecture. To demonstrate the effectiveness of the proposed method on both the single-modality and cross-modality (Optical-InfraRed) near-duplicate image pair detection tasks, we conduct extensive experiments on three popular benchmark datasets, namely CaliforniaND (ND means near duplicate), Mir-Flickr Near Duplicate, and TNO Multi-band Image Data Collection. The experimental results show that the proposed method can achieve superior performance compared with many state-of-the-art methods on both tasks.

Transformer-Based Deep Learning Network for Tooth Segmentation on Panoramic Radiographs

“…ViT is based on a complete self-attention Transformer structure without using CNN. The image is divided into fixed-size patches, then the patches are fed into the linear projections along with their positions [27] . We apply the transformer-based image segmentation algorithm SWin-Unet for the tooth image segmentation task.…”

flowSim: Near duplicate detection for flow cytometry data

“…Most NDD papers describe NDD models or general principles of the NDD task, but not tools. There are no general reviews describing the application of NDD tools (only of general algorithms) and most algorithms [23,[39][40][41][42][43] refers to near-copy detection of fraudulent images, near-duplicate genetic sequences detection and other applica- The small datasets were suitable to generate a manual reference of the homogeneous patterns, while the large datasets were suitable to test flowSim when there are thousands of patterns to filter. Our first tests evaluated flowSim based on a comparison with the manual groupings, suggesting that flowSim was fit for the NDD task and is tunable to a user's desired level of similarity given a dataset's composition.…”

“…Most NDD papers describe NDD models or general principles of the NDD task, but not tools. There are no general reviews describing the application of NDD tools (only of general algorithms) and most algorithms [23, 39–43] refers to near‐copy detection of fraudulent images, near‐duplicate genetic sequences detection and other applications based on features hard to adapt in our context (i.e., they are features optimized to identify similar sequences of characters or digital copies). Based on carefully constructed datasets to aid in the unbiased comparative evaluation of the NDD task on FCM data our results indicate flowSim is fit for this purpose.…”

Table Tennis Track Detection Based on Temporal Feature Multiplexing Network

“…Secondly, it had the capability of multimodal fusion. The input parameters of the Transformer model were one-dimensional features, which made it possible to input other one-dimensional features such as time and text into the Transformer model at the same time and fused them with the token after the feature map transformation [ 63 , 64 , 65 ]. Thirdly, it had a stronger learning ability, and the Transformer model used multiple self-attentive mechanisms for the whole feature map to learn, with each self-attention mechanism independently computing the subspace features before merging [ 66 , 67 ].…”