Composed query image retrieval based on triangle area triple loss function and combining CNN with transformer

Abstract

The existing typical combined query image retrieval methods adopt Euclidean distance as sample distance measurement method, and the model trained by triple loss function blindly pursues absolute distance between samples, resulting in unsatisfactory image retrieval performance. Meanwhile, these methods singularly adopt Convolutional Neural Network (CNN) to extract reference image features. However, receptive field of convolution operation has the characteristics of locality, which is easy to cause the loss of edge feature information of reference images. In view of shortcomings of these methods, the following improvements are proposed in this paper: (1) We propose Triangle Area Triple Loss Function (TATLF), which adopts Triangle Area (TA) as measurement of sample distance. TA comprehensively considers the absolute distance and included angle between samples, so that the trained model has better retrieval performance; (2) We combine CNN with Transformer to simultaneously extract local and edge features of reference images, which can effectively reduce the loss of reference images information. Specifically, CNN is adopted to extract local feature information of reference images. Transformer is used to pay attention to the edge feature information of reference images. Extensive experiments on two public datasets, Fashion200k and MIT-States, confirm the excellent performance of our proposed method.

DSMP-CNN: Dual Spin Max Pooling Convolutional Neural Network for Solar Cell Crack Detection

“…As an example, users who have experience in the data at hand may not necessarily have an understanding of how to build algorithms for CNNs, or vice versa, depending on their familiarity with the data 6 . Therefore, there is a surge of interest in automating CNN architectures, which will make the tuning of CNN architectures transparent to users without any domain knowledge [7][8][9][10] . A CNN architecture design algorithm can, on the other hand, promote wide adoption of CNN architectures, thus promoting the development of the eld of AI through the development of CNNs.Based on the type of domain knowledge that is required when implementing the algorithms for CNN architecture design, existing CNN architecture design algorithms can be broken down into two different categories.…”