Proposing Usability-UX technologies for the design and evaluation of text-based chatbots

Abstract

Chatbots are interactive systems that communicate using natural language with human users, via a textual interface or voice activation. These tools are useful for many spheres of business such as Customer Service, Sales, Education and Learning, Health and Entertainment. Recently, chatbots have become popular, with significant growth in the software industry, especially text-based chatbots. This is encouraging developers to create their own tools, as well as attracting efforts from researchers into this area. Despite this highlight, technologies to guarantee the quality of chatbots and user satisfaction are not keeping up with the growing demand for these tools. Considering this, there is a need to propose technologies capable of supporting developers and development teams in the process of building and evaluating chatbots. Therefore, this research proposes to develop artifacts applicable to the design and evaluation process of chatbots, based on quality attributes identified in systematic literature reviews related to Usability and User Experience (UX), due to the importance and impact that these aspects have on user satisfaction and the perceived quality of the system. The first artifact is the U2Chatbot inspection checklist, developed to assist development teams in the process of identifying defects in text-based chatbots. The second artifact is a set of interface design patterns, DP-U2Chatbot, containing useful examples to support developers in the process of building chatbots. The technologies were subjected to the necessary evaluations. The results of the empirical study regarding the U2Chatbot inspection checklist indicated that participants considered the technology useful for discovering defects in chatbots, however, ease of use could be improved. The participants' experience discreetly influenced the effectiveness and efficiency of the technique, leading us to believe that professionals with a certain level of inspection experience can benefit more from the checklist. Regarding the evaluation of DP-U2Chatbot design patterns, the results generally indicated that the technology is easy to understand and useful in supporting the design of chatbots, helping to build better tools.