An Approach for Ex-Post-Facto Analysis of Knowledge Graph-Driven Chatbots – The DBpedia Chatbot

Jalota, Rricha; Trivedi, Priyansh; Maheshwari, Gaurav; Ngomo, Axel-Cyrille Ngonga; Usbeck, Ricardo

doi:10.1007/978-3-030-39540-7_2

Cited by 5 publications

(5 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Given the relative importance of conversational design, and how the intended chatbot conversations actually play out in the meetings between chatbots and users, it is critical to leverage data from chatbot conversations for insight into user experience. Recent studies have used chatbot dialogue data for automated analysis approaches, such as text mining [10], sentiment analysis [11], and log analysis [12]. Qualitative approaches to analysis of such data are also presented to some extent.…”

Section: Analysis Of Chatbot Dialoguementioning

confidence: 99%

“…As such, the dialogues may provide insight into user experience, issues in service provision, and emerging user needs [9]. However, the research leveraging such chatbot dialogues as a source of insight typically extends only to the use of automated analysis approaches [10][11][12], which entails the risk of overlooking aspects of the conversation that would be observable to a human analyst. Only a few studies present qualitative analysis of chatbot dialogue data [13][14][15], and there is limited guidance on how to conduct such analysis.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Investigating the user experience of customer service chatbot interaction: a framework for qualitative analysis of chatbot dialogues

Følstad

Taylor

2021

Qual User Exp

View full text Add to dashboard Cite

The uptake of chatbots for customer service depends on the user experience. For such chatbots, user experience in particular concerns whether the user is provided relevant answers to their queries and the chatbot interaction brings them closer to resolving their problem. Dialogue data from interactions between users and chatbots represents a potentially valuable source of insight into user experience. However, there is a need for knowledge of how to make use of these data. Motivated by this, we present a framework for qualitative analysis of chatbot dialogues in the customer service domain. The framework has been developed across several studies involving two chatbots for customer service, in collaboration with the chatbot hosts. We present the framework and illustrate its application with insights from three case examples. Through the case findings, we show how the framework may provide insight into key drivers of user experience, including response relevance and dialogue helpfulness (Case 1), insight to drive chatbot improvement in practice (Case 2), and insight of theoretical and practical relevance for understanding chatbot user types and interaction patterns (Case 3). On the basis of the findings, we discuss the strengths and limitations of the framework, its theoretical and practical implications, and directions for future work.

show abstract

Section: Analysis Of Chatbot Dialoguementioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Investigating the user experience of customer service chatbot interaction: a framework for qualitative analysis of chatbot dialogues

Følstad

Taylor

2021

Qual User Exp

View full text Add to dashboard Cite

show abstract

“…Most chatbots applied personalized knowledge graphs in the studies concerning the type of knowledge graphs. As Jalota et al. (2019) mentioned in their article, entity detection in public knowledge graphs is challenging, and there might be more value in considering more personalized knowledge graphs.…”

Section: Types Of Knowledge Graphs and Chatbotsmentioning

confidence: 99%

“…Conversational chatbots, in particular, can learn and search for information from one or more databases. They derive answers to users' questions, put coherence into context and continuously develop (Jalota et al, 2019). A huge advantage of chatbots is their comprehensive customer support.…”

Section: Introductionmentioning

confidence: 99%

The role of knowledge graphs in chatbots

Rajabi,

George,

Kumar

2024

View full text Add to dashboard Cite

Purpose This study aims to investigate the applications of knowledge graphs in developing artificial intelligence (AI) assistants and chatbots by reviewing scholarly publications from different lenses and dimensions. The authors also analyze the various AI approaches used for knowledge graph-driven chatbots and discuss how implementing these techniques makes a difference in technology. Design/methodology/approach Over recent years, chatbots have emerged as a transformational force in interacting with the digital world in various domains, including customer service and personal assistants. Recently, chatbots have become a revolutionary tool for interacting with the digital world in various contexts, such as personal assistants and customer support. Incorporating knowledge graphs considerably improved the capabilities of chatbots by allowing them access to massive knowledge bases and enhancing their ability to understand queries. Furthermore, knowledge graphs enable chatbots to understand semantic links between elements and improve response quality. This study highlights the role of knowledge graphs in chatbots following a systematic review approach. They have been integrated into major health-care, education and business domains. Beyond improving information retrieval, knowledge graphs enhance the user experience and increase the range of fields in which chatbots can be used. Improving and enriching chatbot answers was also identified as one of the main advantages of knowledge graphs. This enriched response can increase user confidence and improve the accuracy of chatbot interactions, making them more trustworthy information sources. Findings Knowledge graph-based chatbots leverage extensive data retrieval to provide accurate and enriched responses, increasing user confidence and experience without requiring extensive training. The three major domains where knowledge graph-based chatbots have been used are health care, education and business. Practical implications Knowledge graph-based chatbots can better comprehend user queries and respond with relevant information efficiently without extensive training. Furthermore, knowledge graphs enable chatbots to understand semantic links between elements, allowing them to answer complicated and multi-faceted questions. This semantic comprehension improves response quality, making chatbots more successful in providing accurate and valuable information in various domains. Furthermore, knowledge graphs enable chatbots to provide consumers with individualized experiences by storing and recalling individual preferences, history or previous encounters. This study analyzes the role of knowledge graphs in chatbots following a systematic review approach. This study reviewed state-of-the-art articles to understand where and how chatbots have used knowledge graphs. The authors found health care, business and education as three main areas in which knowledge-graph-based chatbots have been mostly used. Chatbots have been developed in text, voice and visuals using various machine learning models, particularly natural language pocessing, to develop recommender systems to recommend suitable items, content or services based on user preferences and item associations. Originality/value This paper provides a comprehensive review of the current state of the field in using knowledge graphs in chatbots, focusing on machine learning models, domains and communication channels. The study highlights the prevalence of text and voice channels over visual ones and identifies research gaps and future directions. The paper’s insights can inform the design and development of chatbots using knowledge graphs and benefit both researchers and practitioners in AI, natural language processing and human–computer interaction. The paper is of interest to professionals in domains such as health care, education and business.

show abstract

“…Our evaluation framework is based on several independent lexical features that capture initiative and collaboration in dialogues. Simple automated measures based on discourse features, such as lexical and syntactic diversity, were previously adopted to reduce repetitive generic responses and to estimate question complexity [14,23]. We use an unsupervised approach, similar to the ones applied in language style matching [13] and in measuring quality of generated narratives [22], to a dialogue setting.…”

mentioning

confidence: 99%

An Analysis of Mixed Initiative and Collaboration in Information-Seeking Dialogues

Vakulenko

Kanoulas

Rijke

2020

Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval

View full text Add to dashboard Cite

The ability to engage in mixed-initiative interaction is one of the core requirements for a conversational search system. How to achieve this is poorly understood. We propose a set of unsupervised metrics, termed ConversationShape, that highlights the role each of the conversation participants plays by comparing the distribution of vocabulary and utterance types. Using ConversationShape as a lens, we take a closer look at several conversational search datasets and compare them with other dialogue datasets to better understand the types of dialogue interaction they represent, either driven by the information seeker or the assistant. We discover that deviations from the ConversationShape of a human-human dialogue of the same type is predictive of the quality of a human-machine dialogue. CCS CONCEPTS• Information systems → Information retrieval; • Computing methodologies → Discourse, dialogue and pragmatics.

show abstract

An Approach for Ex-Post-Facto Analysis of Knowledge Graph-Driven Chatbots – The DBpedia Chatbot

Cited by 5 publications

References 18 publications

Investigating the user experience of customer service chatbot interaction: a framework for qualitative analysis of chatbot dialogues

Investigating the user experience of customer service chatbot interaction: a framework for qualitative analysis of chatbot dialogues

The role of knowledge graphs in chatbots

An Analysis of Mixed Initiative and Collaboration in Information-Seeking Dialogues

Contact Info

Product

Resources

About