Entertaining and opinionated but too controlling

Abstract

Conversational systems typically focus on functional tasks such as scheduling appointments or creating todo lists. Instead we design and evaluate SlugBot (SB), one of 8 semifinalists in the 2018 Alexa Prize, whose goal is to support casual open-domain social interaction. This novel application requires both broad topic coverage and engaging interactive skills. We developed a new technical approach to meet this demanding situation by crowd-sourcing novel content and introducing playful conversational strategies based on storytelling and games. We collected over 10,000 conversations during August 2018 as part of the Alexa Prize competition. We also conducted an in-lab follow-up qualitative evaluation. Overall users found SB moderately engaging; conversations averaged 3.6 minutes and involved 26 user turns. However, users reacted very differently to different conversation subtypes. Storytelling and games were evaluated positively; these were seen as entertaining with predictable interactive structure. They also led users to impute personality and intelligence to SB. In contrast, search and general Chit-Chat induced coverage problems; here users found it hard to infer what topics SB could understand, with these conversations seen as being too system-driven. Theoretical and design implications suggest a move away from conversational systems that simply provide factual information. Future systems should be designed to have their own opinions with personal stories to share, and SB provides an example of how we might achieve this.

Assessing user reactions to intelligent personal assistants’ humorous responses

“…“What color are you [IPA]?”), action requests (“Sing Baby Shark”), rhetorical self‐expression (“It's my birthday”) and rhetorical philosophical utterances (“What is true beauty?”). The frequency of utterances that underline user expectations for human‐like behavior/response signal users' curiosity about their IPAs (Lopatovska, 2019), and expectations for the IPA's backstory (Bowden et al, 2019). This finding can be explained by the fact that such utterances would frequently not produce funny responses even in human‐to‐human interaction.…”

“…The literature suggests that humorous responses might also be shaped by the personas developed for each IPA. Having a backstory has been shown to improve user perception and conversational flow (Bowden et al, 2019). Amazon has been more open to discussing this than other companies.…”

Let's Get Personal: Personal Questions Improve SocialBot Performance in the Alexa Prize

“…Inviting an exchange of opinions about the current topic of discussion creates a more intimate experience that increases user satisfaction. We have curated a dataset of topic-annotated personal opinion questions, which are inter-weaved throughout the conversation [6]. The personal opinion questions are split into two different strategies, Would You Rather choices (WYR) and open-ended Hypothetical questions (HYP).…”

“…We collected a novel dataset in which we asked crowdsourced workers to provide both questions and answers for specific topics such as Food, Nature, and Astronomy [6]. After filtering out low-quality and extraneous topics there are 635 pairs across 14 topics.…”

“…Existing work also shows the impact of experience with dialogue systems when engaging with a system like Athena [6]. Besides varying levels of experience, there are several user pools with different needs: there are users who have never interacted with Athena vs. repeat users, kids vs. adults, and users with varied backgrounds and interests.…”