Bucket Selection: A Model-Independent Diverse Selection Strategy for Widening

Abstract

When using a greedy algorithm for finding a model, as is the case in many data mining algorithms, there is a risk of getting caught in local extrema, i.e., suboptimal solutions. Widening is a technique for enhancing greedy algorithms by using parallel resources to broaden the search in the model space. The most important component of widening is the selector, a function that chooses the next models to refine. This selector ideally enforces diversity within the selected set of models in order to ensure that parallel workers explore sufficiently different parts of the model space and do not end up mimicking a simple beam search. Previous publications have shown that this works well for problems with a suitable distance measure for the models, but if no such measure is available, applying widening is challenging. In addition these approaches require extensive, sequential computations for diverse subset selection, making the entire process much slower than the original greedy algorithm. In this paper we propose the bucket selector, a model-independent randomized selection strategy. We find that (a) the bucket selector is a lot faster and not significantly worse when a diversity measure exists and (b) it performs better than existing selection strategies in cases without a diversity measure.

Widening: using parallel resources to improve model quality

“…We then discuss alternatives for the practical realization of Widening for real world algorithms and also address the issues of communication overhead and diversity. Afterwards we summarize earlier encouraging experimental results (Akbar et al 2012;Ivanova and Berthold 2013;Fillbrunn et al 2017;Fillbrunn 2019), to illustrate the potential of Widening as a framework and also recap the Bucket Selector, a promising approach to inject diversity into Widening while at the same time reducing the amount of communication between parallel workers significantly. We conclude by discussing related work in more detail.…”

“…Approaches like roulette wheel selection (Bäck 1996) can also be applied in Widening, but have two disadvantages: They require multiple communication phases and they are not elitist, meaning that the best model is not always selected. A better selection method is the Bucket Selector first introduced by Fillbrunn et al (2017), where each parallel worker is responsible for models with a certain hash code. This hash code determines the partition a model is in.…”

Widened Learning of Index Tracking Portfolios

“…Diversity is an important issue in bio-and chem-informatics and has been studied regarding protein and molecular similarity in [11]. In data mining the effect of diversity on the parallel exploration of the solution space was studied in [12]- [16].…”

Communication-Free Widened Learning of Bayesian Network Classifiers Using Hashed Fiedler Vectors

“…The widening framework terms this Top-k-widening, i.e., M i+1 = s T op−k (r(M i )) : |M i+1 | = k. WIDENING begins to widen the search paths beyond a simple greedy mechanism when diversity is brought into play. The notion of diversity can be implemented in either the refining step as in [24,25] or in the selection step as in [11,12]. Given a diverse refinement operator, r ∆ (•), as in [24,25], where a diversity function, ∆, is imposed on the output, DIVERSE TOP-K WIDENING is described by…”

“…Not Faster." Although the demonstrated examples, such as WIDENED KRIMP [24], WIDENED HIERARCHICAL CLUS-TERING [11], WIDENED BAYESIAN NETWORKS [25] and BUCKET SELECTION [12] have been able to find superior solutions, i.e., "better," they have been unable to demonstrate this ability in a run-time that is comparable to the standard versions of the greedy algorithms. "Not faster" is not intended to mean "slower.…”