How do Humans Interact with Chatbots?: An Analysis of Transcripts

Abstract

Chatbots enable machines to emulate human conversation, and recent developments have resulted in many online systems for the public to use. Although a few studies have investigated how humans interact with such programs, we are not aware of any that have analyzed transcripts in depth. In this study, students interacted with two Web-based chatbots, Rose and Mitsuku, for five minutes and evaluated how well they thought the software emulated human conversation. We reviewed the transcripts and found that students used fairly simple language and made many text errors. There were no significant differences between the two systems in our experimental measures, but we found that Rose tended to change the topic more often and Mitsuku seemed more argumentative.

Optimizing Students’ Mental Health and Academic Performance: AI-Enhanced Life Crafting

“…Since then, conversational systems have come a long way via intelligent assistants like Siri (Apple), Alexa (Amazon), and Cortana (Microsoft), social chatbots aimed at general conversation, and task-focused chatbots (Park et al, 2018;Shum et al, 2018;Jacques et al, 2019). Chatbots are spreading fast among websites and online services in functional areas such as customer service, marketing, entertainment, healthcare, and more.…”

“…In order to improve the clarity of the discourse on chatbots, Braun and Matthes (2019) propose a framework via which chatbots can be categorized in terms of four characteristics beyond the functional application domain (see Table 1). Despite developments in speech recognition based on (a combination of) keywords, the development of conversational skills (e.g., actively keep a conversation going that feels natural) of AI has not improved in a similar pace (e.g., Park et al, 2018;Jacques et al, 2019).…”

What makes you continuously use chatbot services? Evidence from chinese online travel agencies

“…Over the years, many researchers have identified chatbots’ top priority as being accurate at emulating human conversation. For example, a specific test designed for the quality of chatbots evaluates machines’ ability to show intelligent behavior to understand human conversation (Park et al 2018 ). Nguyen ( 2019 ) also tried to examine whether a chatbot could improve the consumer experience when the chatbot possesses the capability of understanding.…”

“… Chatbots conduct intelligent behavior to understand human conversations. Park et al ( 2018 ) ELISA chatbot was evaluated in terms of information quality and service quality. Chatbots with understandability could provide better solutions and performance.…”

Automated Translation Preference with a Bilingual Chatbot

“…Often, this software is used to provide a more intuitive interface for retrieval of information or learning [5,6,7], but it can also be used just for entertainment or provide companionship for lonely people [13,20]. Many younger people have already interacted with chatbots and report positive experiences [21], and they may be more honest conversing with a chatbot than with other people because of its anonymous nature.…”