Designing Effective Interview Chatbots: Automatic Chatbot Profiling and Design Suggestion Generation for Chatbot Debugging

Abstract

Figure 1: iChatProfile dashboard. (a) an example chatbot profile. (b) examples of auto-generated design suggestions for improving a chatbot.

Democratizing Chatbot Debugging: A Computational Framework for Evaluating and Explaining Inappropriate Chatbot Responses

“…For example, the chatbot response "I wish I could... but I have no feet :-)" may be appropriate in the context of asking the chatbot to dance, but it's entirely inappropriate if the user indicates dancing as the hobby (Figure 2(a)). Such inappropriate responses (i.e., chatbot misbehavior) can lead to poor user experience or even abandoned conversations [7]. Therefore, it's critical for chatbot designers to ensure response appropriateness during the design processes.…”

“…In light of this, designers often conduct pilot studies to evaluate chatbot response appropriateness and iterate their designs accordingly (i.e., chatbot debugging) [7]. However, designers without technical backgrounds may face two challenges when it comes to detecting and understanding potentially inappropriate responses revealed by these studies.…”

An Ethical Conversational Agent to Respectfully Conduct In-Game Surveys

“…Conversational agents -interactive systems (embodied or not) that engage in conversation with the user [8]-, offer a new way to collect information, allowing to substitute a traditional survey with an agent that prompts the questions to the user. Indeed, conversational agents have shown to be effective for this task, as they increase both user's commitment with the survey and the quality of the information elicited [10] .…”

“…Against this background, we propose to introduce a conversational agent that conducts the survey in-game, as part of the game experience, with the aim of avoiding the detrimental effects of post-game questionnaires, and to ease participation by allowing to stay closer to the context of an ongoing exposure [17]. Nevertheless, this has also the risk of disturbing the game flow [24] if the chatbot does not properly identify when to prompt the user, or even result in the abandonment of the interview due to the player's cognitive overload [10]. Therefore, we argue that the conversational agent should be respectful with the user's engagement, and thus, we propose to embed the chatbot with a moral value of respect, which should guide the agent to perform the questionnaire without disturbing the user experience.…”

Extending Chatbots to Probe Users: Enhancing Complex Decision-Making Through Probing Conversations

“…In addition, they are increasingly being used as an alternative to conventional surveys, since participants were found to not only write answers to open questions that are not only longer but also of higher quality [48]. They have also been used to conduct interviews due to their potential to empathically handle users' open-ended questions and non-linear dialogues [17,47]. One main focus in Human-Computer Interaction (HCI) in recent years has been on designing them as interventions for (mental) health and wellbeing [3,[23][24][25]27,29,31,32,34,[40][41][42].…”