Helping Users Sort Faster with Adaptive Machine Learning Recommendations

Abstract

Abstract. Sorting and clustering large numbers of documents can be an overwhelming task: manual solutions tend to be slow, while machine learning systems often present results that don't align well with users' intents. We created and evaluated a system for helping users sort large numbers of documents into clusters. iCluster has the capability to recommend new items for existing clusters and appropriate clusters for items. The recommendations are based on a learning model that adapts over time -as the user adds more items to a cluster, the system's model improves and the recommendations become more relevant. Thirty-two subjects used iCluster to sort hundreds of data items both with and without recommendations; we found that recommendations allow users to sort items more rapidly. A pool of 161 raters then assessed the quality of the resulting clusters, finding that clusters generated with recommendations were of statistically indistinguishable quality. Both the manual and assisted methods were substantially better than a fully automatic method.

Semantic Interaction for Visual Analytics: Inferring Analytical Reasoning for Model Steering

“…Such an implementation may lend itself more to hierarchical or categorical models where similarity is fundamentally based on membership to a topic or cluster. Drucker et al present an example of using semantic interaction for such sorting and categorization tasks [18].…”

“…Our framework will employ mixture models to appropriately emphasize specific models for corresponding regions of the layout where those models create a better match [49]. Another approach is to first divide the spatializations into discrete regions (similar to the approach in [18]), then apply the appropriate model to each collection of documents within the region. This approach is a good match for the processes of users we have observed, which indicate that users first bin information based on high-level similarity, then later organize information within each bin [5].…”

Beyond Control Panels: Direct Manipulation for Visual Analytics

“…So, you might need to combine multiple types of models in complex ways. For example, you could combine iCluster, which enables direct manipulation of a cluster membership model, 14 with ForceSpire to enable dynamic layouts of clusters in space, in much the same way analysts currently do manually. The space's continuity and flexibility could represent probabilistic membership.…”

Designing for effective end-user interaction with machine learning

“…Document clustering generally refers to the grouping of items from a relatively static corpus (e.g., a large but fixed corpus of papers that need to be organized by topic). Interactive document clustering involves the end-user in this process (e.g., Drucker et al, 2011;Hearst et al, 1995;Baker et al, 2009 Another active area of research in information retrieval involves document annotation (i.e., attaching metadata on documents such as text, images, and webpages). Research in this area has increased recently due to the prevalence and utility of tagging on the Web, which facilitates search and browsing.…”