Evaluating and Informing the Design of Chatbots

Virtual Assistance in Any Context

“…Chaves and Gerosa (2019) describe intelligence as the ability of a chatbot to participate in a dialogue with an awareness of the discussed topic, while Jain et al (2018) believe that the intelligence of a chatbot can be also deduced from its ability to proactively ask suitable questions and to involve the participant in a meaningful and human-like dialogue. At a holistic level, in line with the proposed final taxonomy, the design elements for chatbots related to specific intelligence features can be described using 15 characteristics, which in turn can be categorized into the following 5 dimensions: The intelligence framework D 1 depicts the underlying cognitive system design delimiting the technical principles under which a chatbot communicates, processes information, and/or selects an action or response (Saravanan et al 2017;Knote et al 2018;Diederich et al 2019).…”

“…Chatbots as a form of conversational agents have been developed for different applications. This is due to the evolving of artificial intelligence (AI) and natural language processing (NLP), which is changing the way artificial assistants communicate and interact with people (Nguyen and Sidorova 2018;Jain et al 2018). By improving text-tospeech and speech-to-text communication, the use of chatbots has become more convenient and common (Bittner et al 2019).…”

“…For instance, new smart assistants, such as Cortana, Alexa, Google Assistant and Siri, have been designed with the intention of supporting users in everyday life as voice-activated intelligent personal assistants. The proliferation of these assistants has contributed to the popularity of chatbots worldwide (Di Prospero et al 2017;Gnewuch et al 2017;Jain et al 2018;Diederich et al 2019). This in turn has also led to an increasing use of domain-specific chatbots (Di Prospero et al 2017).…”

“…Although some elemental chatbot classification frameworks can be found in scientific literature, the research is dispersed into different thematic axes and research areas. Furthermore, the scientific and practical knowledge about chatbots has also grown in a segregated manner given a shortage of integrative perspectives to support chatbot development and design processes Jain et al 2018;Piccolo et al 2018). For instance, most scientific studies today concentrate on particular aspects of chatbots, such as the personality of cognitive chatbots, technical capabilities or their specific application Accepted after three revisions by the editors of the Special Issue.…”

Evaluation and Improvement of Chatbot Text Classification Data Quality Using Plausible Negative Examples

“…Chatbots, or "text messaging-based conversational agents", have received particular attention in 2010s (Jain et al, 2018). Many modern text-based chatbots use relatively simple NLP tools (Abdul-Kader and Woods, 2015), or avoid ML/NLP altogether (Pereira and Díaz, 2018), relying on conver-sation flow design and non-NLP inputs like buttons and quick-replies.…”