Agents for Automated User Experience Testing

Fernandes, Pedro M.; Lopes, Manuel; Prada, Rui

doi:10.1109/icstw52544.2021.00049

Cited by 3 publications

(4 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…37 However, UX assessment is focused almost exclusively on testing with real users. 38 Various UX measurement tools for VUI assessment, such as questionnaires or a combination of methods that collect qualitative and quantitative data, are available. 39,40 Such methods include diary study, 30 a conditional voice recorder, 5 psycho-physiological approaches combined with traditional tools, 41 and the UEQ+ modular framework.…”

Section: User Experience Assessment Methodsmentioning

confidence: 99%

See 1 more Smart Citation

Exploring the context of use for voice user interfaces: Toward context‐dependent user experience quality testing

Klein,

Deutschländer,

Kölln

et al. 2023

J Software Evolu Process

View full text Add to dashboard Cite

Voice user interface (VUI) systems, such as Alexa, Siri, and Google Assistant, are popular and widely available. Still, challenges such as privacy and the ability to have a dialog remain. In the latter example, the user expects a human‐like conversation, that is, that the VUI understands the dialog and its context. However, this VUI feature of context‐aware interaction is rather error prone. For this reason, we intend to explore the VUI context of use and its impact on interaction, that is, relevant user experience (UX). We see a demand for context‐dependent UX measurement because analyzing the context of use and UX assessment are both critical human‐centered design (HCD) methods. Therefore, we examine the VUI context of use by asking users about how, where, and for what they use VUIs, as well as their UX and improvement proposals. We interviewed people with disabilities who rely on VUIs and people without disabilities who use VUIs for convenience or fun. We identified VUI context‐of‐use categories and factors and explored their impacts on relevant UX qualities. Our result is a matrix containing these elements; thus, it provides an overview of the contextual UX of our target group's VUI interaction. We intend to develop a VUI context‐of‐use conceptual structure in the future based on this matrix, which is needed to create an automated context‐dependent UX measurement recommendation tool for VUIs. This conceptual structure could also be useful for automated UX testing in the context of VUI.

show abstract

Section: User Experience Assessment Methodsmentioning

confidence: 99%

“…Interest in automated UX testing is growing, due to, for example, coverage of more users' preferences and acceleration of the UX evaluation process 37 . However, UX assessment is focused almost exclusively on testing with real users 38 …”

Section: Background and Related Workmentioning

confidence: 99%

Exploring the context of use for voice user interfaces: Toward context‐dependent user experience quality testing

Klein,

Deutschländer,

Kölln

et al. 2023

J Software Evolu Process

View full text Add to dashboard Cite

show abstract

“…The UX-aware framework shows that UX requirements are embedded quality requirements that describe the end user's satisfaction or pleasure of using the software. However, UX is generally evaluated based on a prototype(or a release); therefore, productive UX evaluation models could be used early during the requirements development [11]. Regrettably, the unavailability of measurement metrics or a benchmark dataset breaks the early evaluation.…”

Section: Introductionmentioning

confidence: 99%

“…The automation of UX measurement is widely disregarded due to its complexity. In this era, an agent was proposed to automate UX testing for specific predefined tasks of objects finding scenarios in a game application [11].…”

Section: Introductionmentioning

confidence: 99%

Towards Measuring User Experience based on Software Requirements

Atoum¹,

Almalki²,

Alshahrani³

et al. 2021

IJACSA

View full text Add to dashboard Cite

User Experience (UX) provides insights into the users' product perceptions while using or intending to use an application. Software products are known for complexity and changeability, starting from requirements engineering until the product operation. Users often evaluate software UX based on a prototype; however, UX is semantically embedded in the software requirements, a crucial indicator for project success. The problem of current UX evaluation methods is their dependence on the actual involvement of users or experts, a timeconsuming process. First, this paper builds a benchmark dataset of UX based on textual software requirements crowdsourcing several UX experts. Second, the paper develops a machine learning model to measure UX based on the dataset. This research describes the dataset characteristics and reports its statistical internal consistency and reliability. Results indicate a high Cronbach Alpha and a low root mean square error of the dataset. We conclude that the new benchmark dataset could be used to estimate UX instantly without the need for subjective UX evaluation. The dataset will serve as a foundation of UX features for machine learning models.

show abstract