Usability of Chatbots: A Systematic Mapping Study

Ren, Ranci; Castro, John W.; Acuña, Silvia T.; Lara, Juan de

doi:10.18293/seke2019-029

Cited by 20 publications

(15 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…FAQs or user guides). We also plan to study the combination of sentiment analysis and behavioural design patterns [30] to create more likeable and effective chatbots [31]. Finally, security and access-control is another important aspect of any chatbot design as we may want to allow users to query (or not) certain aspects of our data depending on their profile.…”

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

Xatkit: A Multimodal Low-Code Chatbot Development Framework

Daniel¹,

Cabot

Deruelle³

et al. 2020

IEEE Access

113

View full text Add to dashboard Cite

Chatbot (and voicebot) applications are increasingly adopted in various domains such as e-commerce or customer services as a direct communication channel between companies and end-users. Multiple frameworks have been developed to ease their definition and deployment. While these frameworks are efficient to design simple chatbot applications, they still require advanced technical knowledge to define complex interactions and are difficult to evolve along with the company needs (e.g. it is typically impossible to change the NL engine provider). In addition, the deployment of a chatbot application usually requires a deep understanding of the targeted platforms, especially back-end connections, increasing the development and maintenance costs. In this paper, we introduce the Xatkit framework. Xatkit tackles these issues by providing a set of Domain Specific Languages to define chatbots (and voicebots and bots in general) in a platform-independent way. Xatkit also comes with a runtime engine that automatically deploys the chatbot application and manages the defined conversation logic over the platforms of choice. Xatkit's modular architecture facilitates the separate evolution of any of its components. Xatkit is open source and fully available online. INDEX TERMS Modeling, DSL, chatbot design, chatbot deployment.

show abstract

Section: Discussionmentioning

confidence: 99%

“…There are dozens of chatbot platforms. We first start by filtering out popular companies like HelloMyBot, 28 Inbenta, 29 BotCore, 30 1MilionBot 31 or Imperson 32 that offer bot consulting services but not a public tool to build the bot yourself.…”

Section: B Chatbot Development Platformsmentioning

confidence: 99%

Xatkit: A Multimodal Low-Code Chatbot Development Framework

Daniel¹,

Cabot

Deruelle³

et al. 2020

IEEE Access

113

View full text Add to dashboard Cite

show abstract

“…Studies show that chatbots that do not meet users' relatedness needs often result in user dissatisfaction and frustration [38] [74]. Other studies show that autonomy and competence also are important factors in determining the satisfaction with the chatbots [50]. Therefore, we hypothesize that:…”

Section: Sdt and Satisfactionmentioning

confidence: 98%

Design Foundations for AI Assisted Decision Making: A Self Determination Theory Approach

Vreede¹,

Raghavan²,

Vreede³

2021

Proceedings of the Annual Hawaii International Conference on System Sciences

View full text Add to dashboard Cite

Progress of technology and processing power has enabled the advent of sophisticated technology including Artificial Intelligence (AI) agents. AI agents have penetrated society in many forms including conversation agents or chatbots. As these chatbots have a social component to them, is it critical to evaluate the social aspects of their design and its impact on user outcomes. This study employs Social Determination Theory to examine the effect of the three motivational needs on user interaction outcome variables of a decision-making chatbot. Specifically, this study looks at the influence of relatedness, competency, and autonomy on user satisfaction, engagement, decision efficiency, and decision accuracy. A carefully designed experiment revealed that all three needs are important for user satisfaction and engagement while competency and autonomy is associated with decision accuracy. These findings highlight the importance of considering psychological constructs during AI design. Our findings also offer useful implications for AI designers and organizations that plan on using AI assisted chatbots to improve decision-making efforts.

show abstract

“…Finally, Ren et al [33] provided a systematic mapping study about the usability attributes for chatbots. Several specific quality assessment metrics and methods are discussed in this work.…”

Section: Related Workmentioning

confidence: 99%

Quality Assessment Methods for Textual Conversational Interfaces: A Multivocal Literature Review

Coppola

Ardito

2021

Information

View full text Add to dashboard Cite

The evaluation and assessment of conversational interfaces is a complex task since such software products are challenging to validate through traditional testing approaches. We conducted a systematic Multivocal Literature Review (MLR), on five different literature sources, to provide a view on quality attributes, evaluation frameworks, and evaluation datasets proposed to provide aid to the researchers and practitioners of the field. We came up with a final pool of 118 contributions, including grey (35) and white literature (83). We categorized 123 different quality attributes and metrics under ten different categories and four macro-categories: Relational, Conversational, User-Centered and Quantitative attributes. While Relational and Conversational attributes are most commonly explored by the scientific literature, we testified a predominance of User-Centered Attributes in industrial literature. We also identified five different academic frameworks/tools to automatically compute sets of metrics, and 28 datasets (subdivided into seven different categories based on the type of data contained) that can produce conversations for the evaluation of conversational interfaces. Our analysis of literature highlights that a high number of qualitative and quantitative attributes are available in the literature to evaluate the performance of conversational interfaces. Our categorization can serve as a valid entry point for researchers and practitioners to select the proper functional and non-functional aspects to be evaluated for their products.

show abstract

Usability of Chatbots: A Systematic Mapping Study

Cited by 20 publications

References 23 publications

Xatkit: A Multimodal Low-Code Chatbot Development Framework

Xatkit: A Multimodal Low-Code Chatbot Development Framework

Design Foundations for AI Assisted Decision Making: A Self Determination Theory Approach

Quality Assessment Methods for Textual Conversational Interfaces: A Multivocal Literature Review

Contact Info

Product

Resources

About