Pruning and summarizing the discovered associations

Abstract

Association rules are a fundamental class of patterns that exist in data. The key strength of association rule mining is its completeness. It finds all associations in the data that satisfy the user specified minimum support and minimum confidence constraints. This strength, however, comes with a major drawback. It often produces a huge number of associations. This ~ is particularly true for data sets whose attributes are highly correlated. The huge number of associations makes it very difficult, if not impossible, for a human user to analyze in order to identify those interesting/useful ones. In this paper, we propose a novel technique to overcome this problem. The technique first prunes the discovered associations to remove those insignificant associations, and then finds a special subset of the unpruned associations to form a summary of the discovered associations.We call this subset of associations the direction setting (DS) rules as they set the directions that are followed by the rest of the associations. Using this summary, the user can focus on the essential aspects (or relationships) of the domain and selectively view the relevant details. The approach is effective because experiment results show that the set of DS rules is typically very small. They can be analyzed manually by a human user. The proposed technique has also been applied successfully to a number of real-life applications.

An Experiment with Association Rules and Classification: Post-Bagging and Conviction

“…Another way of measuring the independence of antecedent and consequent of a rule is by testing that hypothesis with a χ 2 test [19]. If the value of the statistic (equation 4) is close to zero the hypothesis of independence is accepted.…”

Distribution Rules with Numeric Attributes of Interest

“…interest), leverage or conviction [5]. The χ 2 statistical test has also been extensively used for testing the statistical independence between the antecedent and consequent of association rules [15].…”

Efficiently Identifying Exploratory Rules’ Significance

“…Liu et al [10] did research on association rules with a fixed attribute as consequent. They used a chi-square test to decide whether the antecedent of a rule is independent from its consequent or not, accepting only rules whose antecedent and consequent are positively correlated, thus, discarding rules which happen to appear interesting by chance.…”

“…Definitions of insignificant propositional exploratory rules are provided by Liu et al [10] and Bay and Pazzani [4].…”

“…Bay and Pazzani [4], Liu et. al [10] and Webb [15], developed techniques for identifying insignificant rules with qualitative attributes only (or descretized quantitative attributes). Aumann and Lindell [2] and Huang and Webb [8] both did research on exploratory rule significance with undescretized quantitative attributes as consequent.…”