What you see is what you can change: Human-centered machine learning by interactive visualization

Sacha, Dominik; Sedlmair, Michael; Zhang, Leishi; Lee, John A.; Peltonen, Jaakko; Weiskopf, Daniel; North, Stephen C.; Keim, Daniel A.

doi:10.1016/j.neucom.2017.01.105

Cited by 150 publications

(104 citation statements)

References 58 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…More recently, a number of visual analytics methods have been developed to support the analysis of complex deep neural networks [5,20,24,26,29,34,39]. Liu et al [20] used a hybrid visualization that embedded debugging information into the node-link diagram to help diagnose convolutional neural networks (CNNs).…”

Section: Visualization For Model Analysismentioning

confidence: 99%

See 1 more Smart Citation

RuleMatrix: Visualizing and Understanding Classifiers with Rules

Yao

Bertini

2019

IEEE Trans. Visual. Comput. Graphics

209

135

View full text Add to dashboard Cite

With the growing adoption of machine learning techniques, there is a surge of research interest towards making machine learning systems more transparent and interpretable. Various visualizations have been developed to help model developers understand, diagnose, and refine machine learning models. However, a large number of potential but neglected users are the domain experts with little knowledge of machine learning but are expected to work with machine learning systems. In this paper, we present an interactive visualization technique to help users with little expertise in machine learning to understand, explore and validate predictive models. By viewing the model as a black box, we extract a standardized rule-based knowledge representation from its input-output behavior. Then, we design RuleMatrix, a matrix-based visualization of rules to help users navigate and verify the rules and the black-box model. We evaluate the effectiveness of RuleMatrix via two use cases and a usability study.

show abstract

Section: Visualization For Model Analysismentioning

confidence: 99%

“…Strobelt et al [39] utilized parallel coordinates to help researchers validate hypotheses about the hidden state dynamics of RNNs. Sacha et al [34] introduced a human-centered visual analytics framework to incorporate human knowledge in the machine learning process.…”

Section: Visualization For Model Analysismentioning

confidence: 99%

RuleMatrix: Visualizing and Understanding Classifiers with Rules

Yao

Bertini

2019

IEEE Trans. Visual. Comput. Graphics

209

135

View full text Add to dashboard Cite

show abstract

“… Interactive machine learning Often, domain experts lack proficiency in machine learning, whereas data analysts are uncertain of their findings related to unfamiliar domains [164]. To involve human feedback into systems, [164] proposed a method to visualize humans' interactions with machine learning methods, in which updates during iterative learning are human-centered, and domain knowledge is used to assist in execution and evaluation.…”

Section: The Role Of Human Beings In Knowledge Discoverymentioning

confidence: 99%

“…To involve human feedback into systems, [164] proposed a method to visualize humans' interactions with machine learning methods, in which updates during iterative learning are human-centered, and domain knowledge is used to assist in execution and evaluation. A more recent focus on interaction between humans and machines is the human-centered visualization technique [165,166].…”

Section: The Role Of Human Beings In Knowledge Discoverymentioning

confidence: 99%

Addressing Complexities of Machine Learning in Big Data: Principles, Trends and Challenges from Systematical Perspectives

Wang¹,

Zhao²,

Huang³

et al. 2017

Preprint

View full text Add to dashboard Cite

The concept of 'big data' has been widely discussed, and its value has been illuminated throughout a variety of domains. To quickly mine potential values and alleviate the ever-increasing volume of information, machine learning is playing an increasingly important role and faces more challenges than ever. Because few studies exist regarding how to modify machine learning techniques to accommodate big data environments, we provide a comprehensive overview of the history of the evolution of big data, the foundations of machine learning, and the bottlenecks and trends of machine learning in the big data era. More specifically, based on learning principals, we discuss regularization to enhance generalization. The challenges of quality in big data are reduced to the curse of dimensionality, class imbalances, concept drift and label noise, and the underlying reasons and mainstream methodologies to address these challenges are introduced. Learning model development has been driven by domain specifics, dataset complexities, and the presence or absence of human involvement. In this paper, we propose a robust learning paradigm by aggregating the aforementioned factors. Over the next few decades, we believe that these perspectives will lead to novel ideas and encourage more studies aimed at incorporating knowledge and establishing data-driven learning systems that involve both data quality considerations and human interactions.

show abstract

“…The question of how to effectively evaluate such systems is challenging. Indeed, human-in-the-loop approaches to machine learning bring forth not only numerous intelligibility and usability issues, but also open questions with respect to the evaluation of the various facets of the iML system, both as separate components and as a holistic entity [40]. Holzinger [27] argued that conducting methodically correct experiments and evaluations is difficult, time-consuming, and hard to replicate due to the subjective nature of the "human agents" involved.…”

Section: Introductionmentioning

confidence: 99%

Evaluation of Interactive Machine Learning Systems

Boukhelifa

Bezerianos

Lutton

2018

Human–Computer Interaction Series

View full text Add to dashboard Cite

The evaluation of interactive machine learning systems remains a difficult task. These systems learn from and adapt to the human, but at the same time, the human receives feedback and adapts to the system. Getting a clear understanding of these subtle mechanisms of co-operation and co-adaptation is challenging. In this chapter, we report on our experience in designing and evaluating various interactive machine learning applications from different domains. We argue for coupling two types of validation: algorithm-centered analysis, to study the computational behaviour of the system; and human-centered evaluation, to observe the utility and effectiveness of the application for end-users. We use a visual analytics application for guided search, built using an interactive evolutionary approach, as an exemplar of our work. We argue that human-centered design and evaluation complement algorithmic analysis, and can play an important role in addressing the "black-box" effect of machine learning. Finally, we discuss research opportunities that require human-computer interaction methodologies, in order to support both the visible and hidden roles that humans play in interactive machine learning.

show abstract

What you see is what you can change: Human-centered machine learning by interactive visualization

Cited by 150 publications

References 58 publications

RuleMatrix: Visualizing and Understanding Classifiers with Rules

RuleMatrix: Visualizing and Understanding Classifiers with Rules

Addressing Complexities of Machine Learning in Big Data: Principles, Trends and Challenges from Systematical Perspectives

Evaluation of Interactive Machine Learning Systems

Contact Info

Product

Resources

About