BEAMES: Interactive Multimodel Steering, Selection, and Inspection for Regression Tasks

Das, Subhajit; Cashman, Dylan; Chang, Remco; Endert, Alex

doi:10.1109/mcg.2019.2922592

Cited by 41 publications

(35 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Human‐in‐the‐Loop Ensemble Learning . There are relevant works that involve the human in interpreting, debugging, refining, and comparing ensembles of models [DCCE19, LXL ∗ 18, NP20, SJS ∗ 18, XXM ∗ 19, ZWLC19]. These papers use bagging [Bre01] and boosting [CG16,FSA99,KMF ∗ 17] techniques for ranking and identifying the best combination of models in different application scenarios.…”

Section: Related Workmentioning

confidence: 99%

VisEvol: Visual Analytics to Support Hyperparameter Search through Evolutionary Optimization

Chatzimparmpas

Martins

Kucher

et al. 2021

Computer Graphics Forum

View full text Add to dashboard Cite

During the training phase of machine learning (ML) models, it is usually necessary to configure several hyperparameters. This process is computationally intensive and requires an extensive search to infer the best hyperparameter set for the given problem. The challenge is exacerbated by the fact that most ML models are complex internally, and training involves trial‐and‐error processes that could remarkably affect the predictive result. Moreover, each hyperparameter of an ML algorithm is potentially intertwined with the others, and changing it might result in unforeseeable impacts on the remaining hyperparameters. Evolutionary optimization is a promising method to try and address those issues. According to this method, performant models are stored, while the remainder are improved through crossover and mutation processes inspired by genetic algorithms. We present VisEvol, a visual analytics tool that supports interactive exploration of hyperparameters and intervention in this evolutionary procedure. In summary, our proposed tool helps the user to generate new models through evolution and eventually explore powerful hyperparameter combinations in diverse regions of the extensive hyperparameter space. The outcome is a voting ensemble (with equal rights) that boosts the final predictive performance. The utility and applicability of VisEvol are demonstrated with two use cases and interviews with ML experts who evaluated the effectiveness of the tool.

show abstract

Section: Related Workmentioning

confidence: 99%

VisEvol: Visual Analytics to Support Hyperparameter Search through Evolutionary Optimization

Chatzimparmpas

Martins

Kucher

et al. 2021

Computer Graphics Forum

View full text Add to dashboard Cite

show abstract

“…Another strategy for model steering is to select the best model from a model ensemble, which is usually found in clustering [102,118,121] and regression models [99,103,113,119]. Clustrophile 2 [102] is a visual analytics system for visual clustering analysis, which guides user selection of appropriate input features and clustering parameters through recommendations based on userselected results.…”

Section: Model Selection From An Ensemblementioning

confidence: 99%

“…Clustrophile 2 [102] is a visual analytics system for visual clustering analysis, which guides user selection of appropriate input features and clustering parameters through recommendations based on userselected results. BEAMES [103] was designed for multimodel steering and selection in regression tasks. It creates a collection of regression models by varying algorithms and their corresponding hyperparameters, with further optimization by interactive weighting of data instances and interactive feature selection and weighting.…”

Section: Model Selection From An Ensemblementioning

confidence: 99%

A survey of visual analytics techniques for machine learning

et al. 2020

View full text Add to dashboard Cite

Visual analytics for machine learning has recently evolved as one of the most exciting areas in the field of visualization. To better identify which research topics are promising and to learn how to apply relevant techniques in visual analytics, we systematically review 259 papers published in the last ten years together with representative works before 2010. We build a taxonomy, which includes three first-level categories: techniques before model building, techniques during modeling building, and techniques after model building. Each category is further characterized by representative analysis tasks, and each task is exemplified by a set of recent influential works. We also discuss and highlight research challenges and promising potential future research opportunities useful for visual analytics researchers.

show abstract

“…Wekinator enables a user to implicitly specify and steer models for music performance [FTC09]. BEAMES [DCCE18] allows a user to steer multiple models simultaneously by expressing priorities on individual data instances or data features. Heimerl et.…”

Section: Modeling In Visual Analyticsmentioning

confidence: 99%

“…However, in the modern era of big data, machine learning, and AI, visual analytics systems have begun to take on a new role: to help the user in refining machine learning models. Systems such as TreePOD [MLMP18], BEAMES [DCCE18], and Seq2SeqVis [SGB * 18] propose new visualization and interaction techniques not for a user to better understand their data, but to understand the characteristics of the machine learning models trained on their data and the effects of modifying their parameters and hyperparameters. The goal of these visual analytics systems is to produce a predictive model which will then be used on unseen data.…”

Section: Introductionmentioning

confidence: 99%

A User‐based Visual Analytics Workflow for Exploratory Model Analysis

Cashman

Humayoun

Heimerl

et al. 2019

Computer Graphics Forum

Self Cite

View full text Add to dashboard Cite

Many visual analytics systems allow users to interact with machine learning models towards the goals of data exploration and insight generation on a given dataset. However, in some situations, insights may be less important than the production of an accurate predictive model for future use. In that case, users are more interested in generating of diverse and robust predictive models, verifying their performance on holdout data, and selecting the most suitable model for their usage scenario. In this paper, we consider the concept of Exploratory Model Analysis (EMA), which is defined as the process of discovering and selecting relevant models that can be used to make predictions on a data source. We delineate the differences between EMA and the well‐known term exploratory data analysis in terms of the desired outcome of the analytic process: insights into the data or a set of deployable models. The contributions of this work are a visual analytics system workflow for EMA, a user study, and two use cases validating the effectiveness of the workflow. We found that our system workflow enabled users to generate complex models, to assess them for various qualities, and to select the most relevant model for their task.

show abstract

BEAMES: Interactive Multimodel Steering, Selection, and Inspection for Regression Tasks

Cited by 41 publications

References 19 publications

VisEvol: Visual Analytics to Support Hyperparameter Search through Evolutionary Optimization

VisEvol: Visual Analytics to Support Hyperparameter Search through Evolutionary Optimization

A survey of visual analytics techniques for machine learning

A User‐based Visual Analytics Workflow for Exploratory Model Analysis

Contact Info

Product

Resources

About