The impact of two different think-aloud instructions in a usability test: a case of just following orders?

Zhao, Tingting; McDonald, Sharon; Edwards, Helen M.

doi:10.1080/0144929x.2012.708786

Cited by 35 publications

(48 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Their explicit instruction led to better quality decision-making in relation to the end result of the search. Zhao et al [9] found that an explicit instruction, used in a usability test, led to no differences in task performance but it did yield more of the type of utterances that evaluators may find useful, but at the cost of increased mental workload.…”

Section: Reactivity and Usability Testingmentioning

confidence: 98%

“…Previous work [4,9] has investigated the effects of explicit instructions but has not tested for reactivity by comparing performance with silent working.…”

Section: Reactivity and Usability Testingmentioning

confidence: 99%

“…It is therefore critical that the techniques we use to measure users' reactions to interactive systems do not, in themselves, create changes in users' behavior. Recent observed differences in how we apply the concurrent thinkaloud protocol [7], a valued technique for usability testing, may threaten test validity by changing what users say and do [6,8,9]. The aim of the present study was to investigate the impact of one reported methodological change in the think-aloud protocol: the use of an explicit instruction.…”

Section: Introductionmentioning

confidence: 94%

See 2 more Smart Citations

The effect of global instructions on think-aloud testing

McDonald

Petrie

2013

Proceedings of the SIGCHI Conference on Human Factors in Computing Systems

Self Cite

View full text Add to dashboard Cite

Verbal protocols are the primary tool for understanding users' task-solving behaviors during usability testing. We investigated whether the classic think-aloud and a thinkaloud with an explicit instruction leads to different tasksolving performance compared to silent working.The results suggest that the classic method had no impact on task performance whereas the explicit instruction led to an increase in within-page and between-page navigation and scrolling activity. The classic method was linked to an increase in mental workload in terms of effort and frustration. The explicit think-aloud led to an increase in mental demand, performance, effort and frustration.

show abstract

Section: Reactivity and Usability Testingmentioning

confidence: 98%

“…Previous work [4,9] has investigated the effects of explicit instructions but has not tested for reactivity by comparing performance with silent working.…”

Section: Reactivity and Usability Testingmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 94%

See 1 more Smart Citation

The effect of global instructions on think-aloud testing

McDonald

Petrie

2013

Proceedings of the SIGCHI Conference on Human Factors in Computing Systems

Self Cite

View full text Add to dashboard Cite

show abstract

“…(18). Simulations were followed by a semi-structured interview in which participants were asked to "think aloud" about their experience and reflections on potential benefits associated with completing the EMA twice daily for a span of 30 days (25,26). Sample questions included, "Describe some of the good things that could happen to someone who participated in a daily diary study of drug use," and "Describe for me some concerns people might have about being given a cell phone to carry with them daily".…”

Section: Methodsmentioning

confidence: 99%

Potential benefits of using ecological momentary assessment to study high-risk polydrug use

Roth

Reed

Goldshear

et al. 2017

mHealth

View full text Add to dashboard Cite

Background: While studies have documented both the feasibility and acceptability of using ecological momentary assessment (EMA) to study drug use, there is little empirical research assessing participants' perceptions of utilizing this technology-driven approach.Methods: Participants were English-speaking persons ≥18 years old who reported injection drug use and sequential (e.g., alcohol followed by opioid use) or simultaneous (i.e., injecting heroin and cocaine in one shot) polydrug use within 30 days recruited in San Diego, CA and Philadelphia, PA. Participants (N=36) completed two cell phone-based EMA simulations assessing mood, drug use, HIV risk behaviors, and daily activities, followed by semi-structured interviews that probed for potential benefits of participation over time.Qualitative analysis involved an iterative process of reviewing texts from the interviews to create a coding framework, which was then applied to all transcripts to identify themes.Results: Findings suggest participants may derive indirect benefits from participation in EMA studies including: improved self-worth from helping others; experiencing increased social support through utilization of the study-provided mobile device for non-research purposes; and most importantly, increased self-reflection, which could lead to therapeutic and intervention-like effects such as decreased substance use or reduced HIV risk. Conclusions:Participants identified a variety of potential benefits from participating in a study that utilizes EMA. This research suggests that benefits are highly salient for individuals involved in studies of polydrug use.

show abstract

“…The concurrent think-aloud has been the primary focus of this interest (Hertzum, Hansen, and Andersen 2009;Olmsted-Hawala et al 2010;McDonald and Petrie 2013;Zhao, McDonald, and Edwards 2014;McDonald, Zhao, and Edwards 2015). The Retrospective Think-aloud (RTA), however, has received less attention (Guan et al 2006;Eger et al 2007;Elling, Lentz, and de Jong 2011).…”

Section: Introductionmentioning

confidence: 99%

Retrospective protocols in usability testing: a comparison of Post-session RTA versus Post-task RTA reports

Willis

McDonald

2016

Behaviour & Information Technology

Self Cite

View full text Add to dashboard Cite

We present the results of a study that compared two placements of the Retrospective Think-aloud (RTA): A Post-session RTA where the think-aloud occurs after all tasks are complete, and a Post-task RTA where the think-aloud is elicited after each task. Data from task performance and verbal measures were collected from 24 participants. The results suggest that in terms of task performance, participants in the Post-session RTA condition performed tasks faster, with fewer errors and fewer clicks than in the Post-task RTA condition. In terms of utterances, participants in the Post-task RTA condition produced significantly more utterances that explained actions, expectations and procedural descriptions than in the Post-session RTA condition.

show abstract

The impact of two different think-aloud instructions in a usability test: a case of just following orders?

Cited by 35 publications

References 26 publications

The effect of global instructions on think-aloud testing

The effect of global instructions on think-aloud testing

Potential benefits of using ecological momentary assessment to study high-risk polydrug use

Retrospective protocols in usability testing: a comparison of Post-session RTA versus Post-task RTA reports

Contact Info

Product

Resources

About