A Method to Accelerate Human in the Loop Clustering

Abstract

Data analysis tasks often require grouping of information to identify trends and associations. However, as the number of elements rises to the hundreds and thousands the cost of having a person perform the groupings unassisted quickly becomes prohibitive. Previous approaches have combined traditional clustering techniques with manual interaction steps, yielding human-in-the-loop clustering algorithms that incorporate user feedback by reweighting features or adjusting a similarity function. But in the real world, many grouping tasks lack both a feature set and a well-defined (dis)similarity metric, having only a subject matter expert with an implicit understanding of the correct relationships between elements based on the domain and the task at hand.We present a refine-and-lock clustering interaction model and demonstrate its effectiveness for cognitiveassisted human clustering over other interaction models such as split/merge and must-link/can't-link. Our approach offers effective automatic clustering assistance even in the absence of clear features or a definitive similarity metric; ensures that every cluster has final user approval; and exhibits at least a 3.94x improvement over other interactive clustering approaches in time to completion.

Interactive Clustering

“…The rest of the papers provide more unique solutions that often-despite similar names and high-level descriptions-can have different semantics, and it is unclear in some papers how the feedback from such operations is maintained over time. One such example is allowing the user to move data instances from one cluster to another (e.g., Basu et al [15], Coden et al [33]). Dubey et al [40] calls this "assignment feedback" and claims that manually moving data points between clusters supports the exploration of high-dimensional data.…”

“…In Arın et al [8], the user merges clusters until satisfied. In Coden et al [33], the tasks, such as grouping food items into categories for a restaurant menu, are presented as subjective and lacking both the feature set and well-defined dis-/similarity metrics. In most cases [11,20,31,35], the user iteratively re-defines the clustering tasks (e.g., focusing on a subset of data points and attributes) and refines the clustering parameters (e.g., algorithm, number of clusters, threshold), until a satisfactory solution is reached.…”

“…In Sourina and Liu [93], the tool provides various geometrical shapes (e.g., boxes and ellipses) to update the clustering parameters. Others include dragging and dropping data points from one cluster to another [21,33,47,66] and selecting data points that should/should not be in close proximity by interacting with the data points in a 2D projection [22,23,28,29,49].…”

“…For example, in Choo et al [30], El-Assady et al [42], Lee et al [67], users incorporate their domain knowledge by adjusting the weights of keywords used as features. In Coden et al [33], a user can move points from one cluster to another and with each move, the method adjusts weights used in the weighted semantic similarity measure. In Okabe and Yamada [78], the interactive clustering tool repeatedly uses "distance metric learning" to update a dissimilarity matrix to satisfy user's constraints through the interactive process.…”

“…Subjective measures may also include user agreement for a particular task. For example, Coden et al [33] uses the Adjusted Rank Index between groupings for several subjective tasks from five users (e.g., categorizing items on a restaurant menu). Another example is presented in Wu et al [110], which the goal is to reduce uncertainties during the matching process by asking users whether two chosen synonym or homonym fields match (yes or no answers).…”