The micro-task market for lemons: data quality on Amazon's Mechanical Turk

Ahler, Douglas J.; Roush, Carolyn E.; Sood, Gaurav

doi:10.1017/psrm.2021.57

Cited by 66 publications

(28 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…These criteria resulted in exclusion of 321 (28.6%) of respondents in one of the experimental conditions, consisting of 65 excluded for both reasons, 184 excluded for the attention check alone, and 62 excluded on the basis of time on task alone. This exclusion rate is consistent with estimates of untrustworthy MTurk workers [ 34 ]. The resulting sample consists of n = 270 for the Male-Narr -Video group, n = 255 for the Female-Narr -Video group, n = 282 for the text group, and n = 393 for the control group.…”

Section: Methodssupporting

confidence: 89%

A randomized controlled trial of a video intervention shows evidence of increasing COVID-19 vaccination intention

Witus

Larson

2022

PLoS ONE

View full text Add to dashboard Cite

Increasing acceptance of COVID-19 vaccines is imperative for public health. Previous research on educational interventions to overcome vaccine hesitancy have shown mixed effects in increasing vaccination intention, although much of this work has focused on parental attitudes toward childhood vaccination. In this study, we conducted a randomized controlled trial to investigate whether vaccination intention changes after viewing an animated YouTube video explaining how COVID-19 mRNA vaccines work. We exposed participants to one of four interventions–watching the video with a male narrator, watching the same video with a female narrator, reading the text of the transcript of the video, or receiving no information (control group). We found that participants who watched the version of the video with a male narrator expressed statistically significant increased vaccination intention compared to the control group. The video with a female narrator had more variation in results. As a whole, there was a non-significant increased vaccination intention when analyzing all participants who saw the video with a female narrator; however, for politically conservative participants there was decreased vaccination intention for this intervention compared to the control group at a threshold between being currently undecided and expressing probable interest. These results are encouraging for the ability of interventions as simple as YouTube videos to increase vaccination propensity, although the inconsistent response to the video with a female narrator demonstrates the potential for bias to affect how certain groups respond to different messengers.

show abstract

Section: Methodssupporting

confidence: 89%

A randomized controlled trial of a video intervention shows evidence of increasing COVID-19 vaccination intention

Witus

Larson

2022

PLoS ONE

View full text Add to dashboard Cite

show abstract

“…The current study utilizes a sample of participants obtained via Amazon Mechanical Turk (MTurk). Previous work has illustrated the imperfect nature of MTurk samples (e.g., Ahler, Roush, & Sood, 2019), and potential misuse of the platforms (e.g., by “bots”) is an important issue not to be overlooked. In the current study, the description of the HIT did not mention aging, though we do note that including IP address monitoring and/or geotagging in future IRB protocols can help identify those who respond untruthfully.…”

Section: Discussionmentioning

confidence: 99%

Remembering proper names as a potential exception to the better-than-average effect in younger and older adults.

Hargis

Whatley

Castel

2020

Psychology and Aging

View full text Add to dashboard Cite

People see themselves as better than average in many domains, from leadership skills to driving ability. However, many people—especially older adults—struggle to remember others’ names, and many of us are aware of this struggle. Our beliefs about our memory for names may be different from other information; perhaps forgetting names is particularly salient. We asked younger and older adults to rate themselves compared with others their age on several socially desirable traits (e.g., honesty); their overall memory ability; and their specific ability to remember scientific terms, locations, and people’s names. Participants demonstrated a better-than-average (BTA) effect in their ratings of most items except their ability to remember names, which both groups rated as approximately the same as others their age. Older adults’ ratings of this ability were related to a measure of the social consequences of forgetting another’s name, but younger adults’ ratings were not. The BTA effect is present in many judgments for both younger and older adults, but people may be more attuned to memory failures when those failures involve social consequences.

show abstract

“…Armitage and Eerola (2020) have shown that the results of music cognition experiments on chord perception carried out in standard lab settings are comparable with those from online experiments that recruit participants using services similar to MTurk. However, study of crowdsourcing platforms has shown that the percentage of non-serious participants and survey bots is relatively large (Ahler et al, 2019;Dennis et al, 2020). Hence, a pre-test and other ways to detect non-serious participants and survey bots were included in our experiment (see Participants and below).…”

Section: Methodsmentioning

confidence: 99%

Relative salience of chord-type and chord-voicing changes: A two-oddball paradigm

Jimenez

Kuusi

Ojala

2021

Psychology of Music

View full text Add to dashboard Cite

Our research project investigated the effect of background and stimuli factors on the relative salience of chord-type and chord-voicing changes. Earlier studies have shown that surface features tend to be easier to perceive than deeper features and that musical training attenuates this general tendency. For further studying how deeper-level and surface-level musical features are perceived, we used a two-oddball paradigm. Each item consisted of a succession of five same-root chords: one chord-type oddball (deeper feature), one voicing oddball (surface feature), and three standards. Participants chose the chord that sounded most different to them. All chord-type pairings formed of major, minor, dominant seventh, major seventh, and minor seventh chords were tested. Chord-type oddball and voicing oddball were chosen equally often, together forming the majority of the responses. Musical training and conceptual knowledge of chords affected the chord-type oddball responses, but not the voicing-oddball responses. However, chord-type oddballs were chosen regardless of the musical training. Chord-type responses were easiest for pairs consisting of a major-based and a minor-based chord and for pairs involving two pitch-class changes. Our results suggest that musical training and conceptual knowledge about chords is not the only factor influencing the relative salience of chord-type changes over voicing changes.

show abstract

The micro-task market for lemons: data quality on Amazon's Mechanical Turk

Cited by 66 publications

References 33 publications

A randomized controlled trial of a video intervention shows evidence of increasing COVID-19 vaccination intention

A randomized controlled trial of a video intervention shows evidence of increasing COVID-19 vaccination intention

Remembering proper names as a potential exception to the better-than-average effect in younger and older adults.

Relative salience of chord-type and chord-voicing changes: A two-oddball paradigm

Contact Info

Product

Resources

About