Field experiment methodology for pair analytics

Kaastra, Linda T.; Fisher, Brian

doi:10.1145/2669557.2669572

Cited by 21 publications

(20 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Methodology and Study Design -Due to the variety of available interaction loops, we decided to conduct a pair analytics study [33], enabling each participant to transfer their individual workflow to the system. We )).…”

Section: User-studymentioning

confidence: 99%

See 1 more Smart Citation

explAIner: A Visual Analytics Framework for Interactive and Explainable Machine Learning

Spinner

Schlegel

Schäfer

et al. 2019

IEEE Trans. Visual. Comput. Graphics

185

146

View full text Add to dashboard Cite

MLModel ML Input ML Output Model State XAI Method XAI Input XAI Output explanations (visualizations, verbalizations, surrogate models, etc.)Abstract-We propose a framework for interactive and explainable machine learning that enables users to (1) understand machine learning models;(2) diagnose model limitations using different explainable AI methods; as well as (3) refine and optimize the models. Our framework combines an iterative XAI pipeline with eight global monitoring and steering mechanisms, including quality monitoring, provenance tracking, model comparison, and trust building. To operationalize the framework, we present explAIner, a visual analytics system for interactive and explainable machine learning that instantiates all phases of the suggested pipeline within the commonly used TensorBoard environment. We performed a user-study with nine participants across different expertise levels to examine their perception of our workflow and to collect suggestions to fill the gap between our system and framework. The evaluation confirms that our tightly integrated system leads to an informed machine learning process while disclosing opportunities for further extensions.

show abstract

Section: User-studymentioning

confidence: 99%

“…Finally, we evaluate the implemented system in a qualitative userstudy, with nine participants, ranging from model novices to model developers. During the pair analytics sessions [33], we analyze the usefulness of our tool while deriving ideas for future versions.…”

Section: Introductionmentioning

confidence: 99%

explAIner: A Visual Analytics Framework for Interactive and Explainable Machine Learning

Spinner

Schlegel

Schäfer

et al. 2019

IEEE Trans. Visual. Comput. Graphics

185

146

View full text Add to dashboard Cite

show abstract

“…Through the study sessions, we recorded and observed the dynamics between the SMEs and VAE, and provide some comments here, following Kaastra et al [KF14]. We observed instances of each of the three communicative mechanisms of joint action: grounding, advancing, and repairing.…”

Section: Expert Case Studiesmentioning

confidence: 92%

“…All three experts analyze multi‐party conversations in their research; in particular they focus on the effects of different modes of communication for reaching consensus. We used the pair analytics method of Kaastra et al [KF14], in which one member of our research team acted in the visual analytics expert (VAE) role working with the SME. In addition, another researcher was present to observe the interaction between the VAE and SME and aspects of insight generation.…”

Section: Expert Case Studiesmentioning

confidence: 99%

NEREx: Named‐Entity Relationship Exploration in Multi‐Party Conversations

El‐Assady

Sevastjanova

Gipp

et al. 2017

Computer Graphics Forum

View full text Add to dashboard Cite

We present NEREx, an interactive visual analytics approach for the exploratory analysis of verbatim conversational transcripts. By revealing different perspectives on multi‐party conversations, NEREx gives an entry point for the analysis through high‐level overviews and provides mechanisms to form and verify hypotheses through linked detail‐views. Using a tailored named‐entity extraction, we abstract important entities into ten categories and extract their relations with a distance‐restricted entity‐relationship model. This model complies with the often ungrammatical structure of verbatim transcripts, relating two entities if they are present in the same sentence within a small distance window. Our tool enables the exploratory analysis of multi‐party conversations using several linked views that reveal thematic and temporal structures in the text. In addition to distant‐reading, we integrated close‐reading views for a text‐level investigation process. Beyond the exploratory and temporal analysis of conversations, NEREx helps users generate and validate hypotheses and perform comparative analyses of multiple conversations. We demonstrate the applicability of our approach on real‐world data from the 2016 U.S. Presidential Debates through a qualitative study with three domain experts from political science.

show abstract

“…Methodology Due to the number of available views and settings in addition to the limited time of qualified experts to learn all aspects of a new system, we chose to conduct a pair analytics study [KF14]. We conducted five two-hour sessions in which a member of our team (henceforth referred to as visual analytics expert, short VAE) worked with the one domain expert (henceforth referred to as subject matter expert, short SME).…”

Section: Expert User Studiesmentioning

confidence: 99%

ThreadReconstructor: Modeling Reply‐Chains to Untangle Conversational Text through Visual Analytics

El‐Assady

Sevastjanova

Keim

et al. 2018

Computer Graphics Forum

View full text Add to dashboard Cite

a) Reply-Relation View (b) Thematic-Forest View showing each connected component as a separate tree, sorted by the number of posts.Figure 1: Thematic-Forest (1b) of untangled reply-chains from a full-conversation (1a) according to a content-focused query (left arcs) compared to a random-forest model trained on 13 features (right arcs). Model agreement and match to ground truth are shown using color. AbstractWe present ThreadReconstructor, a visual analytics approach for detecting and analyzing the implicit conversational structure of discussions, e.g., in political debates and forums. Our work is motivated by the need to reveal and understand single threads in massive online conversations and verbatim text transcripts. We combine supervised and unsupervised machine learning models to generate a basic structure that is enriched by user-defined queries and rule-based heuristics. Depending on the data and tasks, users can modify and create various reconstruction models that are presented and compared in the visualization interface. Our tool enables the exploration of the generated threaded structures and the analysis of the untangled reply-chains, comparing different models and their agreement. To understand the inner-workings of the models, we visualize their decision spaces, including all considered candidate relations. In addition to a quantitative evaluation, we report qualitative feedback from an expert user study with four forum moderators and one machine learning expert, showing the effectiveness of our approach.

show abstract

Field experiment methodology for pair analytics

Cited by 21 publications

References 20 publications

explAIner: A Visual Analytics Framework for Interactive and Explainable Machine Learning

explAIner: A Visual Analytics Framework for Interactive and Explainable Machine Learning

NEREx: Named‐Entity Relationship Exploration in Multi‐Party Conversations

ThreadReconstructor: Modeling Reply‐Chains to Untangle Conversational Text through Visual Analytics

Contact Info

Product

Resources

About