Comparing sets of patterns with the Jaccard index

Abstract

The ability to extract knowledge from data has been the driving force of Data Mining since its inception, and of statistical modelling long before even that. Actionable knowledge often takes the form of patterns, where a set of antecedents can be used to infer a consequent. In this paper we offer a solution to the problem of comparing different sets of patterns. Our solution allows comparisons between sets of patterns that were derived from different techniques (such as different classification algorithms), or made from different samples of data (such as temporal data or data perturbed for privacy reasons). We propose using the Jaccard index to measure the similarity between sets of patterns by converting each pattern into a single element within the set. Our measure focuses on providing conceptual simplicity, computational simplicity, interpretability, and wide applicability. The results of this measure are compared to prediction accuracy in the context of a real-world data mining scenario.

Accurate detection of Covid-19 patients based on Feature Correlated Naïve Bayes (FCNB) classification strategy

“…Thus, there are many distance functions, which are used to define a distance between items or elements. These distance functions such as; Cosine similarity (Shirkhorshidi et al 2015), Jaccard distance (Fletcher and Slam 2018), Manhattan distance (Pandit and Gupta 2011), Euclidean distance (Dokmanic et al 2015), etc. The cluster is constructed in a way that any two data items associated with the same cluster have the minimum value of distance and any two data items associated with different clusters have the maximum value of distance (Zhu et al 2019).…”

Interpreting a black box predictor to gain insights into early folding mechanisms

“…However, the patterns in the frequencies of the features and their cutoffs are relatively similar to the denovo model. We use the Jaccard similarity index, weighted by features frequency [15] , to estimate this similarty and obtain a value of 0.74, which could be considered high. Similar to denovo model, the scrambled model relies on high values of backbone feature interacting with helix but focuses more on the values of sheet and sidechain features for detecting early folding residues.…”

Applying Machine Learning of Erythrocytes Dynamic Antigens Store in Medicine

“…From this point, the Jaccard similarity can be calculated as shown in Table 3. The Jaccard similarity (coefficient) (Fletcher and Islam, 2018) is a term coined by Paul Jaccard to measure similarities between sets. It is defined as the size of the intersection divided by the size of the union of two sets.…”