Fits and Starts: Enterprise Use of AutoML and the Role of Humans in the Loop

Crisan, Anamaria; Fiore-Gartland, Brittany

doi:10.48550/arxiv.2101.04296

Cited by 4 publications

(5 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We continued to the uploaded visualization examples and asked the participant to walk us through the authoring process. To help the participants delineate the authoring process and reflect on their choices, we selected questions from our interview guide, focusing on various topics based on prior literature and user studies on processes involving visualization authoring, user roles in work processes, and criteria for evaluating tools [3,4,16,23]. These questions included: We piloted the interviews with two participants to test the interview guide and pair-interview dynamics.…”

Section: Methodsmentioning

confidence: 99%

Understanding Visualization Authoring Techniques for Genomics Data in the Context of Personas and Tasks

van den Brandt,

L'Yi,

Nguyen

et al. 2024

Preprint

View full text Add to dashboard Cite

Genomics experts rely on visualization to extract and share insights from complex and large-scale datasets. Beyond off-the-shelf tools for data exploration, there is an increasing need for platforms that aid experts in authoring customized visualizations for both exploration and communication of insights. A variety of interactive techniques have been proposed for authoring data visualizations, such as template editing, shelf configuration, natural language input, and code editors. However, it remains unclear how genomics experts create visualizations and which techniques best support their visualization tasks and needs. To address this gap, we conducted two user studies with genomics researchers: (1) semi-structured interviews (n=20) to identify the tasks, user contexts, and current visualization authoring techniques and (2) an exploratory study (n=13) using visual probes to elicit users’ intents and desired techniques when creating visualizations. Our contributions include (1) a characterization of how visualization authoring is currently utilized in genomics visualization, identifying limitations and benefits in light of common criteria for authoring tools, and (2) generalizable design implications for genomics visualization authoring tools based on our findings on task- and user-specific usefulness of authoring techniques.All supplemental materials are available at https://osf.io/bdj4v/

show abstract

Section: Methodsmentioning

confidence: 99%

Understanding Visualization Authoring Techniques for Genomics Data in the Context of Personas and Tasks

van den Brandt,

L'Yi,

Nguyen

et al. 2024

Preprint

View full text Add to dashboard Cite

show abstract

“…including wrapping, can answer such questions even in much more detail. In this sense, Naive AutoML presents itself as more amenable to the growing demand for meaningful interaction between the tool and the human [31,4,5,30] compared to the currently adopted black-box approaches.…”

Section: A Prototype For a Stage Schemementioning

confidence: 99%

“…At this point, we shall stress that the main goal of this paper is not to present yet another tool for AutoML but to propose a different view on how automated machine learning could be done. Recent work has revealed that several aspects of the black-box character of current systems impede experts from trusting their output, and that there is a need for more interaction between the tool and the expert [31,4,5,30]. Since Naive AutoML effectively imitates a data scientist, it makes an important step into this direction by adopting an optimization process that is much more comprehensive for an expert who interacts with it.…”

Section: Introductionmentioning

confidence: 99%

Naive Automated Machine Learning -- A Late Baseline for AutoML

Mohr,

Wever

2021

Preprint

View full text Add to dashboard Cite

Automated Machine Learning (AutoML) is the problem of automatically finding the pipeline with the best generalization performance on some given dataset. AutoML has received enormous attention in the last decade and has been addressed with sophisticated black-box optimization techniques such as Bayesian Optimization, Grammar-Based Genetic Algorithms, and tree search algorithms. In contrast to those approaches, we present Naive AutoML, a very simple solution to AutoML that exploits important meta-knowledge about machine learning problems and makes simplifying, yet, effective assumptions to quickly come to high-quality solutions. While Naive AutoML can be considered a baseline for the highly sophisticated black-box solvers, we empirically show that those solvers are not able to outperform Naive AutoML; sometimes the contrary is true. On the other hand, Naive AutoML comes with strong advantages such as interpretability and flexibility and poses a strong challenge to current tools.

show abstract

“…While these results might suggest Quasi-Naive AutoML as a meaningful baseline over which one should be able to substantially improve, we see the actual role of Quasi-Naive AutoML as the door opener for sequential optimization of pipelines. The currently applied black-box optimizers come with a series of problems discussed in recent literature such as lack of interpretability and flexibility [5,4]. The naive approaches follow a sequential optimization approach, optimizing one component after the other.…”

Section: Introductionmentioning

confidence: 99%

Naive Automated Machine Learning

Mohr¹,

Wever²

2021

Preprint

View full text Add to dashboard Cite

An essential task of Automated Machine Learning (AutoML) is the problem of automatically finding the pipeline with the best generalization performance on a given dataset. This problem has been addressed with sophisticated black-box optimization techniques such as Bayesian Optimization, Grammar-Based Genetic Algorithms, and tree search algorithms. Most of the current approaches are motivated by the assumption that optimizing the components of a pipeline in isolation may yield sub-optimal results. We present Naive AutoML, an approach that does precisely this: It optimizes the different algorithms of a pre-defined pipeline scheme in isolation. The finally returned pipeline is obtained by just taking the best algorithm of each slot. The isolated optimization leads to substantially reduced search spaces, and, surprisingly, this approach yields comparable and sometimes even better performance than current state-of-the-art optimizers.

show abstract

Fits and Starts: Enterprise Use of AutoML and the Role of Humans in the Loop

Cited by 4 publications

References 24 publications

Understanding Visualization Authoring Techniques for Genomics Data in the Context of Personas and Tasks

Understanding Visualization Authoring Techniques for Genomics Data in the Context of Personas and Tasks

Naive Automated Machine Learning -- A Late Baseline for AutoML

Naive Automated Machine Learning

Contact Info

Product

Resources

About