User Experience of Alexa when controlling music

Brüggemeier, Birgit; Breiter, Michael; Kurz, Miriam; Schiwy, Johanna

doi:10.1145/3405755.3406122

Cited by 10 publications

(4 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Continued use [33]; Satisfaction [33]; System efectiveness [24]; Tone clarity [8]; Satisfaction [3]; System Usability Scale (SUS) [6,44]; Speech User Interface Service Quality Questionnaire (SUISQ-R) [6]; Document feedback [38]; Voice understandability [10]; Perception of rapport [46]; Group decision performance [46]; Voice performance [11] Social presence [32,33]; Intimacy [25,33]; Closeness (Subjective closeness index) [53];…”

Section: Ivs and IV Frameworkmentioning

confidence: 99%

“…Perceived power [46]; Conformity in decision-making [30]; Helpfulness [25]; [13]; Attractiveness diferences [6]; Self-efciency [7]; Interaction questionnaire [19]; Godspeed questionnaire [19] illustrates how most categories have relied on lab-based studies.…”

Section: Ivs and IV Frameworkmentioning

confidence: 99%

“…They found a lack of consensus on what CUI UX is and how it is measured, leading them to develop a high-level unifying framework. Brüggemeier et al [6] recognized a gap in measuring the UX of commercial voice assistants and then evaluated the validity of several instruments for Amazon Alexa. Nowacki et al [43] proposed a set of heuristics for voice usability and UX, updating previous guidelines in light of changing technologies and conceptualizations of experience.…”

mentioning

confidence: 99%

“…Kocaballi et al [26] reviewed CUIs, which are not necessarily voice-based: other modalities, notably text, are used. Brüggemeier et al [6] narrowly focused on voice assistants, excluding other "bodyless" voice-based possibilities. Nowacki et al [43] focused on heuristics and guidelines rather than measurement.…”

mentioning

confidence: 99%

See 3 more Smart Citations

Measuring Voice UX Quantitatively

Seaborn

Urakami

2021

Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems

View full text Add to dashboard Cite

Computer voice is experiencing a renaissance through the growing popularity of voice-based interfaces, agents, and environments. Yet, how to measure the user experience (UX) of voice-based systems remains an open and urgent question, especially given that their form factors and interaction styles tend to be non-visual, intangible, and often considered disembodied or "body-less. " As a frst step, we surveyed the ACM and IEEE literatures to determine which quantitative measures and measurements have been deemed important for voice UX. Our fndings show that there is little consensus, even with similar situations and systems, as well as an overreliance on lab work and unvalidated scales. In response, we ofer two high-level descriptive frameworks for guiding future research, developing standardized instruments, and informing ongoing review work. Our work highlights the current strengths and weaknesses of voice UX research and charts a path towards measuring voice UX in a more comprehensive way. CCS CONCEPTS• Human-centered computing → Human computer interaction (HCI); Human computer interaction (HCI); Interaction devices; Sound-based input / output.

show abstract

Section: Ivs and IV Frameworkmentioning

confidence: 99%

Section: Ivs and IV Frameworkmentioning

confidence: 99%

mentioning

confidence: 99%

mentioning

confidence: 99%

See 2 more Smart Citations

Measuring Voice UX Quantitatively

Seaborn

Urakami

2021

Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems

View full text Add to dashboard Cite

show abstract

User Experience of Alexa, Siri and Google Assistant When Controlling Music – Comparison of Four Questionnaires

Brüggemeier

Breiter

Kurz

et al. 2020

Lecture Notes in Computer Science

Self Cite

View full text Add to dashboard Cite

We evaluate user experience (UX) when users play and control music with three smart speakers: Amazon's Alexa Echo, Google Home and Apple's Siri on a HomePod. For measuring UX we use five established UX and usability metrics (AttrakDiff, SASSI, SUISQ-R, SUS). We investigated the sensitivity of these five questionnaires in two ways: firstly we compared the UX reported for each of the speakers, secondly we compared the UX of completing easy single tasks and more difficult multi tasks with these speakers. We find that the investigated questionnaires are sufficiently sensitive to show significant differences in UX for these easy and difficult tasks. In addition, we find some significant UX differences between the tested speakers. Specifically, all tested questionnaires, except the SUS, show a significant difference in UX between Siri and Alexa, with Siri being perceived as more user friendly for controlling music. We discuss implications of our work for researchers and practitioners.Speech assistance is a growing market with a 25% yearly growth predicted in the next three years [21]. Speech assistants can be integrated in different devices, like smartphones, personal computers and smart speakers, which are dedicated speakers that can be controlled by voice commands. In our work we focus on smart speakers. Currently one in five Americans over 18 years owns a smart speaker [28], which is a remarkable number, considering that smart speakers were first introduced in 2014 [16]. It means that within six years approximately 53 Million Americans bought a smart speaker, which is a market development comparable to the rapid spread of smart phones [7]. This market trend is not confined to the North American market, but is present throughout the world, in Europe, as well as Asia, Africa and Latin America [32,33,8,17], showing that smart speakers are of broad public interest.The consumer speech assistance market in the English speaking world, as well as in Europe, is dominated by three manufacturers and assistants: Amazon with Alexa, Google with Google Assistant and Apple with Siri [8,36]. These three assistants cover more than 88% of the market in the US [36]. Intuitively, these three assistants are named as the most commonly known Voice User Interfaces (VUIs) [31] and featured as smart speakers in numerous product

show abstract

Success is not Final; Failure is not Fatal – Task Success and User Experience in Interactions with Alexa, Google Assistant and Siri

Kurz

Brüggemeier

Breiter

2021

Lecture Notes in Computer Science

Self Cite

View full text Add to dashboard Cite

Speech assistants exhibit a high error rate with about one in three user requests resulting in an error. Nonetheless, speech assistants are adopted rapidly with about 1.8 billion users expected in 2021. Given the relatively high task failure rate of speech assistants this may be surprising and raises the question how much user experience (UX) is affected by task success in these devices. We measure user experience with four metrics of UX and evaluate task success in interactions with the speech assistants Alexa, Google Assistant, and Siri. We find that task success only explains between 13% and 30% of the variance of UX. This suggests that a majority of UX is not explained by whether an assistant successfully completes tasks. Moreover, we find that the three assistants do not significantly differ in task success rate, but differ in UX, which supports the conclusion that task success and UX possess limited alignment. We discuss our results and point out limitations and potential future work.

show abstract

User Experience of Alexa when controlling music

Cited by 10 publications

References 28 publications

Measuring Voice UX Quantitatively

Measuring Voice UX Quantitatively

User Experience of Alexa, Siri and Google Assistant When Controlling Music – Comparison of Four Questionnaires

Success is not Final; Failure is not Fatal – Task Success and User Experience in Interactions with Alexa, Google Assistant and Siri

Contact Info

Product

Resources

About