Gamifying the Human-in-the-Loop: Toward Increased Motivation for Training AI in Customer Service

Wiethof, Christina; Roocks, Tim; Bittner, Eva A. C.

doi:10.1007/978-3-031-05643-7_7

Cited by 4 publications

(6 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Our HiL Nexus principle also contributes to the body of knowledge on HiL design and configurations (Grønsund & Aanestad, 2020;Wiethof & Bittner, 2021). Our design knowledge provides justificatory insights into our HiL configuration.…”

Section: Discussionmentioning

confidence: 86%

“…Thus, support ticket quality is crucial for support agents, especially for supervised machine learning approaches that require properly labeled data (Jiang et al, 2020). HI systems often turn to Human-in-the-Loop (HiL) mechanisms asking employees to label their data during the initial training phase and operations (Wiethof & Bittner, 2021). The latter is particularly important to allow incremental learning to avoid datadrift, which describes a discrepancy between past training data and future test data (Mallick, Hsieh, Arzani, & Joshi, 2022;Tsymbal, 2004).…”

Section: Support Agents and Immediate Gratificationmentioning

confidence: 99%

“…The worth of customer touchpoints has long been researched in service research, where customer journeys are constituted by service encounters, which are enabled by touchpoints (Hogan, Almquist, & Glynn, 2005;Kronqvist & Leinonen, 2019). From a HI perspective, recent attention has moved towards human-in-the-loop configuration, where experts are involved in augmenting algorithms (Grønsund & Aanestad, 2020;Wiethof & Bittner, 2021). We argue that touchpoints have to augment frontline service encounters (Keyser et al, 2019), the aim is to design an intelligent system that augments support agent problem-solving capabilities (DR2), which keeps on learning after its initial development (DR1) and focuses on the touchpoints of support agents to provide an incentive to engage in the HiL activities (DR3).…”

Section: Hil-as-nexus (Solution Space)mentioning

confidence: 99%

See 2 more Smart Citations

Exploring the Shifting Dynamics of Information Source Selection: Situational Characteristics and the Rise of Artificial Intelligence

Meservy,

Fadel

et al. 2024

Proceedings of the Annual Hawaii International Conference on System Sciences

View full text Add to dashboard Cite

The rapid adoption of innovative technologies confronts IT-Service-Management (ITSM) with incoming support requests of increasing complexity. As a consequence, job demands and turnover rates of ITSM support agents increase. Recent technological advances have introduced assistance systems that rely on hybrid intelligence to provide support agents with contextually suitable historical solutions to help them solve customer requests. Hybrid intelligence systems rely on human input to provide high-quality data to train their underlying AI models. Yet, most agents have little incentives to label their data, lowering data quality and leading to diminishing returns of AI systems due to concept drifts. Following a design science research approach, we provide a novel Human-in-the-Loop design and hybrid intelligence system for ITSM support ticket recommendations, which incentivize agents to provide high-quality labels. Specifically, we leverage agents' need for instant gratification by simultaneously providing better results if they improve labeling automatically labeled support tickets.

show abstract

Section: Discussionmentioning

confidence: 86%

Section: Support Agents and Immediate Gratificationmentioning

confidence: 99%

Section: Hil-as-nexus (Solution Space)mentioning

confidence: 99%

See 1 more Smart Citation

Exploring the Shifting Dynamics of Information Source Selection: Situational Characteristics and the Rise of Artificial Intelligence

Meservy,

Fadel

et al. 2024

Proceedings of the Annual Hawaii International Conference on System Sciences

View full text Add to dashboard Cite

show abstract

“…Human trainers can also help in tagging and labeling data more accurately, which is a vital part of supervised learning. Their expertise ensures high-quality training data, leading to more effective and accurate chatbot responses [140].…”

Section: Integrating Human Oversightmentioning

confidence: 99%

Error Correction and Adaptation in Conversational AI: A Review of Techniques and Applications in Chatbots

Izadi,

Forouzanfar

2024

View full text Add to dashboard Cite

This study explores the progress of chatbot technology, focusing on the aspect of error correction to enhance these smart conversational tools. Chatbots, powered by artificial intelligence (AI), are increasingly prevalent across industries such as customer service, healthcare, e-commerce, and education. Despite their use and increasing complexity, chatbots are prone to errors like misunderstandings, inappropriate responses, and factual inaccuracies. These issues can have an impact on user satisfaction and trust. This research provides an overview of chatbots, conducts an analysis of errors they encounter, and examines different approaches to rectifying these errors. These approaches include using data-driven feedback loops, involving humans in the learning process, and adjusting through learning methods like reinforcement learning, supervised learning, unsupervised learning, semi-supervised learning, and meta-learning. Through real life examples and case studies in different fields, we explore how these strategies are implemented. Looking ahead, we explore the different challenges faced by AI-powered chatbots, including ethical considerations and biases during implementation. Furthermore, we explore the transformative potential of new technological advancements, such as explainable AI models, autonomous content generation algorithms (e.g., generative adversarial networks), and quantum computing to enhance chatbot training. Our research provides information for developers and researchers looking to improve chatbot capabilities, which can be applied in service and support industries to effectively address user requirements.

show abstract

“…[8]. This contrasts hybrid intelligence with, for example, human-in-the-loop learning, in which a human interacts with the AI system during the learning phase to improve the system through human involvement [12]. In HI, the human's and AI's capabilities are augmented, leveraging the strengths of the individual actors, and compensating for their weaknesses.…”

Section: Introductionmentioning

confidence: 99%

Developing Team Design Patterns for Hybrid Intelligence Systems

Zoelen

Mioch

Tajaddini

et al. 2023

Frontiers in Artificial Intelligence and Applications

View full text Add to dashboard Cite

With artificial intelligence (AI) systems entering our working and leisure environments with increasing adaptation and learning capabilities, new opportunities arise for developing hybrid (human-AI) intelligence (HI) systems, comprising new ways of collaboration. However, there is not yet a structured way of specifying design solutions of collaboration for hybrid intelligence (HI) systems and there is a lack of best practices shared across application domains. We address this gap by investigating the generalization of specific design solutions into design patterns that can be shared and applied in different contexts. We present a human-centered bottom-up approach for the specification of design solutions and their abstraction into team design patterns. We apply the proposed approach for 4 concrete HI use cases and show the successful extraction of team design patterns that are generalizable, providing re-usable design components across various domains. This work advances previous research on team design patterns and designing applications of HI systems.

show abstract

Gamifying the Human-in-the-Loop: Toward Increased Motivation for Training AI in Customer Service

Cited by 4 publications

References 31 publications

Exploring the Shifting Dynamics of Information Source Selection: Situational Characteristics and the Rise of Artificial Intelligence

Exploring the Shifting Dynamics of Information Source Selection: Situational Characteristics and the Rise of Artificial Intelligence

Error Correction and Adaptation in Conversational AI: A Review of Techniques and Applications in Chatbots

Developing Team Design Patterns for Hybrid Intelligence Systems

Contact Info

Product

Resources

About