Exploring the effects of non-monetary reimbursement for participants in HCI research

Abstract

When running experiments within the field of Human Computer Interaction (HCI) it is common practice to ask participants to come to a specified lab location, and reimburse them monetarily for their time and travel costs. This, however, is not the only means by which to encourage participation in scientific study. Citizen science projects, which encourage the public to become involved in scientific research, have had great success in getting people to act as sensors to collect data or to volunteer their idling computer or brain power to classify large data sets across a broad range of fields including biology, cosmology and physical and environmental science. This is often done without the expectation of payment. Additionally, data collection need not be done on behalf of an external researcher; the Quantified Self (QS) movement allows people to reflect on data they have collected about themselves. This too, then, is a form of non-reimbursed data collection. Here we investigate whether citizen HCI scientists and those interested in personal data produce reliable results compared to participants in more traditional lab-based studies. Through six studies, we explore how participation rates and data quality are affected by recruiting participants without monetary reimbursement: either by providing participants with data about themselves as reward (a QS approach), or by simply requesting help with no extrinsic reward (as in citizen science projects). We show that people are indeed willing to take part in online HCI research in the absence of extrinsic monetary reward, and that the data generated by participants who take part for selfless reasons, rather than for monetary reward, can be as high quality as data gathered in the lab and in addition may be of higher quality than data generated by participants given monetary reimbursement online. This suggests that large HCI experiments could be run online in the future, without having to incur the equally large reimbursement costs alongside the possibility of running experiments in environments outside of the lab.

Thinking and Doing: Challenge, Agency, and the Eudaimonic Experience in Video Games

“…Resultant transcripts were over 80,000 words long in total. Participants volunteered their time out of interest for the project rather than for fiscal compensation, which research has suggested leads to higher quality data (Wiseman, Cox, Gould, & Brumby, 2017).…”

Why Do You Need This?

“…Wiseman et al demonstrated the feasibility and effectiveness of HCI research with participation of citizen scientists. They found that data provided in the absence of monetary reward, and contributed for altruistic reasons, can equal or indeed exceed the quality of data generated by paid participants [34]. Their results suggest that citizen science can provide viable solutions for large scale HCI experiments that could be otherwise prohibitively expensive to run.…”

“…Researchers often tell people something about themselves in exchange for their participation in research studies [34], [29]. This kind of quid pro quo is an effective way of recruiting new and curious 'citizen scientists' who help to improve our understanding of the world.…”

Influencing and Measuring Behaviour in Crowdsourced Activities

“…Factors besides money can also be the primary motivation for participants. In previous work we have explicitly compared monetary and non-monetary reimbursement for participants (Wiseman et al, 2017). One of the things we wanted to study was whether participants could be motivated by being given data about themselves as an outcome of a study.…”

“…This has produced some very large snowballed samples, where participants recruit other participants. In Wiseman et al (2017), we compared this kind of data-as-reward approach with other approaches, like simply requesting help for the good of science or in return for payment. We found that that participants in both non-monetary conditions met, and in many cases exceeded, the performance of participants in our control paid condition.…”