VIS4ML: An Ontology for Visual Analytics Assisted Machine Learning

Abstract

While many VA workflows make use of machine-learned models to support analytical tasks, VA workflows have become increasingly important in understanding and improving Machine Learning (ML) processes. In this paper, we propose an ontology (VIS4ML) for a subarea of VA, namely "VA-assisted ML". The purpose of VIS4ML is to describe and understand existing VA workflows used in ML as well as to detect gaps in ML processes and the potential of introducing advanced VA techniques to such processes. Ontologies have been widely used to map out the scope of a topic in biology, medicine, and many other disciplines. We adopt the scholarly methodologies for constructing VIS4ML, including the specification, conceptualization, formalization, implementation, and validation of ontologies. In particular, we reinterpret the traditional VA pipeline to encompass model-development workflows. We introduce necessary definitions, rules, syntaxes, and visual notations for formulating VIS4ML and make use of semantic web technologies for implementing it in the Web Ontology Language (OWL). VIS4ML captures the high-level knowledge about previous workflows where VA is used to assist in ML. It is consistent with the established VA concepts and will continue to evolve along with the future developments in VA and ML. While this ontology is an effort for building the theoretical foundation of VA, it can be used by practitioners in real-world applications to optimize model-development workflows by systematically examining the potential benefits that can be brought about by either machine or human capabilities. Meanwhile, VIS4ML is intended to be extensible and will continue to be updated to reflect future advancements in using VA for building high-quality data-analytical models or for building such models rapidly.

explAIner: A Visual Analytics Framework for Interactive and Explainable Machine Learning

“…In contrast to fully automated approaches such as AutoML [7] or Neural Architecture Search [20], IML strives to incorporate the human into the model building, training, and correction process to optimize a model [22]. VA can be applied to the IML workflow to boost the model development process through tailored visual interfaces [60]. VA for IML tightly integrates the user to promote further sensemaking during the model development workflow [21].…”

“…More recent work focuses not only on visual design but also on interactive, mixed-initiative workflows, as provided by Visual Analytics (VA) systems [21]. Also, an exploratory workflow [60] enables a more targeted analysis and design of ML models. Visual analytics further helps in bridging the gap between user knowledge and the insights XAI methods can provide.…”

Interactive Learning for Identifying Relevant Tweets to Support Real-time Situational Awareness

“…However, these tools do not employ machine learning for relevance classification and do not integrate user feedback to improve their underlying models or algorithms. Visual analytics has also been increasingly used to improve various machine learning processes, such as feature selection [13], attribute weighting [48], and labeling [8,9,18], and even understanding the models themselves [22,39,42]. Sacha et al [40] proposed a framework to discuss the various forms of human interaction with machine learning models in visual analytics systems and theorized that VA tools could increase knowledge and usability of machine learning components.…”

A User‐based Visual Analytics Workflow for Exploratory Model Analysis

“…The ontology for visual analytics assisted machine learning proposed by Sacha et al [SKKC19] offers the clearest background on which to describe our workflow's application to EMA. In that work, the authors present a fairly complete knowledge encoding of common concepts in visual analytics systems that use machine learning, and offer suggestions of how popular systems in the literature map onto that encoding.…”