Similarity Joins and Beyond: An Extended Set of Binary Operators with Order

Abstract

Abstract. Similarity joins are troublesome database operators that often produce results much larger than the user really needs or expects. In order to return the similar elements, similarity joins also require sorting during the retrieval process, although order is a concept not supported in the relational model. This paper proposes a solution to solve those two issues extending the similarity join concept to a broader set of binary operators, which aims at retrieving the most similar pairs and embedding the sorting operation only as an internal processing step, so as to comply with the relational theory. Additionally, our extension allows to explore another useful condition not previously considered in the similarity retrieval: the negation of predicates. Experiments performed on real and synthetic data show that our operators are fast enough to be used in real applications and scale well both for multidimensional and non-dimensional metric data.

Efficient Self-similarity Range Wide-joins Fostering Near-duplicate Image Detection in Emergency Scenarios

“…the value of attribute S in tuple t v feature vector of the k most similar attributes to each t 1 (Carvalho et al, 2015). A third type of similarity join is often described in the database literature (Silva et al, 2013): the kdistance join.…”

“…It retrieves the k pairs t 1 ,t 2 having the most similar values t 1 [S 1 ] and t 2 [S 2 ]. This operator is an instance of the similarity wide-join (Carvalho et al, 2015). Wide-joins retrieve the most similar pairs in general, sorting the tuple internally, allowing its processing to comply with the relational theory and executed efficiently.…”

“…The first is the similarity widejoin with a nested-loop approach (NLWJ), that is the self version of the baseline found in the literature (Carvalho et al, 2015). The second method is the Adaptive Cluster with k-means (ACMe -Section 2) (Li et al, 2015).…”

“…As a possible solution, recent approaches (Xiao et al, 2011;Carvalho et al, 2015) consider using well-known searching operators from both the database and information retrieval areas, namely similarity joins and wide-joins, to detect near-duplicate elements. Similarity joins (Silva et al, 2013) obtain pairs of elements similar among themselves, assuming each pair corresponds to the near-duplicate candidates obtained by the building phase.…”

“…However, those retrieval operators were applied only to detect nearduplicate elements in data represented by strings, as in (Xiao et al, 2011), but they were not explored in other domains such as images. In their turn, wide-joins (Carvalho et al, 2015) are designed to retrieve the overall most similar pairs, leading to an inherent combination of building and improvement phases in their processing. However, wide-joins process two distinct sets, while near-duplicate detection must combine a set with itself.…”