Widened KRIMP: Better Performance through Diverse Parallelism

Abstract

We demonstrate that the previously introduced Widening framework is applicable to state-of-the-art Machine Learning algorithms. Using Krimp, an itemset mining algorithm, we show that parallelizing the search finds better solutions in nearly the same time as the original, sequential/greedy algorithm. We also introduce Reverse Standard Candidate Order (RSCO) as a candidate ordering heuristic for Krimp. 1 Introduction Research into parallelism in Machine Learning has primarily focused on reducing the execution time of existing algorithms, e.g., parallelized k-Means [23,17,14,26] and Dbscan [11,4,7]. There have been some exceptions, such as metalearning and ensemble methods [9], which have employed heterogeneous algorithms in parallel, and [3], which describes the application to simple examples. Recent work [2,15] describes Widening, a framework for employing parallel resources to increase accuracy. With Widening, measures of diversity are used to guarantee the parallel search paths' exploration of disparate regions within a solution space, thereby stepping around the common greedy algorithmic tendency to find local optima. Thus far, work has concentrated on a proof-of-concept and demonstrative application to algorithms for solving the Set Cover Problem and the creation of Decision Trees. This document describes the same approach, but with a state-of-the-art algorithm, Krimp [24]. Krimp finds "interesting" itemsets from a transactional database via the Minimum Description Length (MDL) principle [21]. The authors summarize the method as "the best set of patterns [being] the set of patterns that describes the data best," where the best set of itemsets is the set that provides the highest compression using MDL. The algorithm not only provides a solution to the problem of pattern explosion, thereby greatly reducing the set of itemsets used to generate association rules, but provides exceptional performance in other applications such as classification [24]. This paper demonstrates that it is possible to apply Widening to find even more interesting sets of itemsets than those found by the standard Krimp algorithm.

Communication-Free Widened Learning of Bayesian Network Classifiers Using Hashed Fiedler Vectors

“…The widening framework terms this Top-k-widening, i.e., M i+1 = s T op−k (r(M i )) : |M i+1 | = k. WIDENING begins to widen the search paths beyond a simple greedy mechanism when diversity is brought into play. The notion of diversity can be implemented in either the refining step as in [24,25] or in the selection step as in [11,12]. Given a diverse refinement operator, r ∆ (•), as in [24,25], where a diversity function, ∆, is imposed on the output, DIVERSE TOP-K WIDENING is described by…”

“…Not Faster." Although the demonstrated examples, such as WIDENED KRIMP [24], WIDENED HIERARCHICAL CLUS-TERING [11], WIDENED BAYESIAN NETWORKS [25] and BUCKET SELECTION [12] have been able to find superior solutions, i.e., "better," they have been unable to demonstrate this ability in a run-time that is comparable to the standard versions of the greedy algorithms. "Not faster" is not intended to mean "slower.…”

Widened Learning of Bayesian Network Classifiers

“…p -dispersion-sum has the property of pushing the resultant subset to the margins of the original set, whereas the subset derived using p -dispersion-min-sum is more representative of the dataset as whole [24]. Because of this property, and based on the results in [29], we favor p-dispersion-min-sum as the diverse subset selection method. …”

“…This enables the system as a whole to avoid local optima and potentially find better models than the greedy learning algorithm would otherwise find. Previous work [13,29] has demonstrated its viability on real world algorithms. This work builds on that with an application to the superexponentially-sized [28] hypothesis space of learning Bayesian Networks.…”

Diversity-Driven Widening of Hierarchical Agglomerative Clustering

“…Another approach that focuses on leveraging parallel computing resources to improve models generated by a data mining algorithm, rather than speeding up the computation, has been proposed in [1]. The technique has already been shown to work well for the set covering problem and KRIMP [17].…”