Trust in AutoML

Drozdal, Jaimie; Weisz, Justin D.; Wang, Dakuo; Dass, Gaurav; Yao, Bingsheng; Zhao, Changruo; Müller, Michael; Lin, Ju; Su, Hui

doi:10.1145/3377325.3377501

Cited by 82 publications

(21 citation statements)

References 39 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Dove et al [209] surveyed to understand how design innovation is practiced in the ML domain in terms of user experience. [63], [70], [165], [174], [211] [124], [125], [128], [175], [217] [69], [112], [113], [118], [130], [131], [133], [135], [137], [140], [144], [152], [153], [155] [49], [57], [67], [98], [110], [114], [180], [192], [193], [203], [214], [215] [48], [59], [62], [75], [79], [111], [120], [156], [159], [185], [195], [197], [207], [218] [65], [83],…”

Section: Survey Resultsmentioning

confidence: 99%

“…Das et al [78] explored whether humans can improve performance using explainable AI capabilities with computer techniques that surpass humans. Drozdal et al [79] attempted to discover what is vital to ML engineers' trust in the Auto ML system. Ohn-Bar et al [80] studied an indoor navigation interface for blind people.…”

Section: User Studiesmentioning

confidence: 99%

“…Some of the prior works that target the developer as the human focus on novice ML engineers to help them develop ML systems faster [62,197]. Notably, the majority of works that target the developer side focused on ML engineers [59,65,71,75,79,120,156,160,185,195,196,198,199,201,207].…”

Section: The 'Human' In Hcmlmentioning

confidence: 99%

See 2 more Smart Citations

A Review of Recent Deep Learning Approaches in Human-Centered Machine Learning

Kaluarachchi

Reis

Nanayakkara

2021

Sensors

View full text Add to dashboard Cite

After Deep Learning (DL) regained popularity recently, the Artificial Intelligence (AI) or Machine Learning (ML) field is undergoing rapid growth concerning research and real-world application development. Deep Learning has generated complexities in algorithms, and researchers and users have raised concerns regarding the usability and adoptability of Deep Learning systems. These concerns, coupled with the increasing human-AI interactions, have created the emerging field that is Human-Centered Machine Learning (HCML). We present this review paper as an overview and analysis of existing work in HCML related to DL. Firstly, we collaborated with field domain experts to develop a working definition for HCML. Secondly, through a systematic literature review, we analyze and classify 162 publications that fall within HCML. Our classification is based on aspects including contribution type, application area, and focused human categories. Finally, we analyze the topology of the HCML landscape by identifying research gaps, highlighting conflicting interpretations, addressing current challenges, and presenting future HCML research opportunities.

show abstract

Section: Survey Resultsmentioning

confidence: 99%

Section: User Studiesmentioning

confidence: 99%

See 1 more Smart Citation

A Review of Recent Deep Learning Approaches in Human-Centered Machine Learning

Kaluarachchi

Reis

Nanayakkara

2021

Sensors

View full text Add to dashboard Cite

show abstract

“…In the future, we want to implement additional components to cover and simplify the Data Acquisition and Governance, Model Deployment, and Model Monitoring stages and improve the support of multi-user communications and collaborations in every stage and across different stages. Finally, as the relationship between user trust and AutoML systems is crucial to ensure that these tools can be trustworthy enough to be adopted responsibly by the public [70], we also plan to investigate the explainability and trust to allow the users to understand the results and automation better.…”

Section: Discussionmentioning

confidence: 99%

A Knowledge Graph-Based Data Integration Framework Applied to Battery Data Management

et al. 2021

View full text Add to dashboard Cite

Today, the automotive and transportation sector is undergoing a transformation process to meet the requirements of sustainable and efficient operations. This transformation mainly reveals itself by electric vehicles, hybrid electric vehicles, and electric vehicle sharing. One significant, and the most expensive, component in electric vehicles is the batteries, and the management of batteries is crucial. It is essential to perform constant monitoring of behavior changes for operational purposes and quickly adjust components and operations to these changes. Thus, to address these challenges, we propose a knowledge graph-based data integration framework for simplifying access and analysis of data accumulated through the operations of vehicles and related transportation systems. The proposed framework aims to enable the effortless analysis and navigation of integrated knowledge and the creation of additional data sets from this knowledge to use during the application of data analysis and machine learning. The knowledge graph serves as a significant component to simplify the extraction, enrichment, exploration, and generation of data in this framework. We have developed it according to the human-centered design, and various roles of the data science and machine learning life cycle can use it. Its main objective is to streamline the exploration and interaction with the integrated data to maximize human productivity. Finally, we present a battery use case to show the feasibility and benefits of the proposed framework. The use case illustrates the usage of the framework to extract knowledge from raw data, navigate and enrich it with additional knowledge, and generate data sets.

show abstract

“…[81] How did the system produce the model? [30] Users interpret the system's output and question the underlying mechanisms.…”

Section: Scrutabilitymentioning

confidence: 99%

How to Support Users in Understanding Intelligent Systems? Structuring the Discussion

Eiband

Buschek

Hußmann

2021

26th International Conference on Intelligent User Interfaces

View full text Add to dashboard Cite

The opaque nature of many intelligent systems violates established usability principles and thus presents a challenge for humancomputer interaction. Research in the field therefore highlights the need for transparency, scrutability, intelligibility, interpretability and explainability, among others. While all of these terms carry a vision of supporting users in understanding intelligent systems, the underlying notions and assumptions about users and their interaction with the system often remain unclear.We review the literature in HCI through the lens of implied user questions to synthesise a conceptual framework integrating user mindsets, user involvement, and knowledge outcomes to reveal, differentiate and classify current notions in prior work. This framework aims to resolve conceptual ambiguity in the field and enables researchers to clarify their assumptions and become aware of those made in prior work. We thus hope to advance and structure the dialogue in the HCI research community on supporting users in understanding intelligent systems. CCS CONCEPTS• Human-centered computing → HCI theory, concepts and models.

show abstract

Trust in AutoML

Cited by 82 publications

References 39 publications

A Review of Recent Deep Learning Approaches in Human-Centered Machine Learning

A Review of Recent Deep Learning Approaches in Human-Centered Machine Learning

A Knowledge Graph-Based Data Integration Framework Applied to Battery Data Management

How to Support Users in Understanding Intelligent Systems? Structuring the Discussion

Contact Info

Product

Resources

About