Impact and Costs of Incentives to Reduce Attrition in Online Trials: Two Randomized Controlled Trials

Khadjesari, Zarnie; Murray, Elizabeth; Kalaitzaki, Eleftheria; White, Ian R.; McCambridge, Jim; Thompson, Simon G.; Wallace, Paul; Godfrey, Christine

doi:10.2196/jmir.1523

Cited by 105 publications

(142 citation statements)

References 26 publications

Supporting

Mentioning

137

Contrasting

Order By: Relevance

“…185 An online trial to reduce alcohol consumption found that 3-month retention rates were higher for the group who were offered a £10 gift voucher versus nothing, at a cost of £52 per additional response. 258 Further research in this area might compare this with retention rates and costs for other trial designs to find the method that represents best value for money.…”

Section: Recruitment and Retention Costsmentioning

confidence: 99%

Sexual health promotion for young people delivered via digital media: a scoping review

et al. 2015

Self Cite

View full text Add to dashboard Cite

BackgroundYoung people are at risk of poor sexual health and are, therefore, in need of comprehensive, effective sexual health education. Young people are confident and constant users of digital technology, such as the internet and mobile phones, and there are many innovative possibilities for sexual health education involving these technologies.ObjectivesTo summarise evidence on effectiveness, cost-effectiveness and mechanism of action of interactive digital interventions (IDIs) for sexual health; optimal practice for intervention development; contexts for successful implementation; research methods for digital intervention evaluation; and the future potential of sexual health promotion via digital media.DesignLiterature review of evidence on digital interventions for sexual health for young people, integrating the findings with the views of young people, parents and experts in digital media/sexual health. IDIs are defined as digital media programmes that provide health information and tailored decision support, behaviour-change support and/or emotional support. We focus on sexual well-being for young people aged 13–24 years in the UK.ResultsThere are many imaginative IDIs for sexual health promotion, but few interventions address issues that are important to young people, such as sexual pleasure and relationships. It is vital to collaborate with young people and to use Behaviour-Change Theory in designing interventions. We located 19 randomised controlled trials of IDIs for sexual health promotion for young people, finding a moderate effect on sexual health knowledge [standardised mean difference (SMD) 0.54, 95% confidence interval (CI) 0.17 to 0.92], a small effect on confidence (self-efficacy) (SMD 0.11, 95% CI 0.02 to 0.20) and a positive effect on sexual behaviour (odds ratio 1.28, 95% CI 1.01 to 1.61), but no significant effects on safer sex intention or biological outcomes. One study suggests that IDIs may be as good as face-to-face interventions for sexual health knowledge and safer sex intention. There are no existing data on the cost-effectiveness of IDIs for sexual health promotion. The impact of an IDI will be determined by the proportion of the target population reached, intervention efficacy, adoption in a setting, how well it is delivered and maintenance/sustainability. All of these elements must be addressed for IDIs to be successful. More collaboration is needed to capitalise on the knowledge of users and stakeholders, the design and software skills of the commercial sector and the theoretical expertise and evaluation skills of academia.ConclusionsIDIs are effective for knowledge acquisition and sexual behaviour, and could usefully contribute to sexual health education in schools, in clinic settings and online; however, there are obstacles to overcome, such as access to information technology and ensuring the quality and safety of interventions.Future workMore evidence is needed on the best designs for interventions (e.g. choice of behaviour-change mechanisms and interactive features) and the best models of delivery (e.g. setting, modes of delivery, methods of facilitation and support for engagement) to improve sexual behaviour, biological outcomes and sexual well-being in a cost-effective way.FundingThe National Institute for Health Research Public Health Research programme.

show abstract

Section: Recruitment and Retention Costsmentioning

confidence: 99%

Sexual health promotion for young people delivered via digital media: a scoping review

et al. 2015

Self Cite

View full text Add to dashboard Cite

show abstract

“…The larger incentive offered at 12 months appeared to have a positive influence on the response rate (an increase from 36% to 50%), which is in line with studies which tested the effect of larger incentives. 66,89 FEASIBILITY RANDOMISED CONTROLLED TRIAL OF THE MEN'S SAFER SEX WEBSITE NIHR Journals Library www.journalslibrary.nihr.ac.uk…”

Section: Online Outcome Measurementmentioning

confidence: 99%

The Men’s Safer Sex project: intervention development and feasibility randomised controlled trial of an interactive digital intervention to increase condom use in men

Bailey

Webster

Hunter

et al. 2016

Health Technol Assess

Self Cite

View full text Add to dashboard Cite

BackgroundThis report details the development of the Men’s Safer Sex website and the results of a feasibility randomised controlled trial (RCT), health economic assessment and qualitative evaluation.Objectives(1) Develop the Men’s Safer Sex website to address barriers to condom use; (2) determine the best design for an online RCT; (3) inform the methods for collecting and analysing health economic data; (4) assess the Sexual Quality of Life (SQoL) questionnaire and European Quality of Life-5 Dimensions, three-level version (EQ-5D-3L) to calculate quality-adjusted life-years (QALYs); and (5) explore clinic staff and men’s views of online research methodology.Methods(1) Website development: we combined evidence from research literature and the views of experts (n = 18) and male clinic users (n = 43); (2) feasibility RCT: 159 heterosexually active men were recruited from three sexual health clinics and were randomised by computer to the Men’s Safer Sex website plus usual care (n = 84) or usual clinic care only (n = 75). Men were invited to complete online questionnaires at 3, 6, 9 and 12 months, and sexually transmitted infection (STI) diagnoses were recorded from clinic notes at 12 months; (3) health economic evaluation: we investigated the impact of using different questionnaires to calculate utilities and QALYs (the EQ-5D-3L and SQoL questionnaire), and compared different methods to collect resource use; and (4) qualitative evaluation: thematic analysis of interviews with 11 male trial participants and nine clinic staff, as well as free-text comments from online outcome questionnaires.Results(1) Software errors and clinic Wi-Fi access presented significant challenges. Response rates for online questionnaires were poor but improved with larger vouchers (from 36% with £10 to 50% with £30). Clinical records were located for 94% of participants for STI diagnoses. There were no group differences in condomless sex with female partners [incidence rate ratio (IRR) 1.01, 95% confidence interval (CI) 0.52 to 1.96]. New STI diagnoses were recorded for 8.8% (7/80) of the intervention group and 13.0% (9/69) of the control group (IRR 0.75, 95% CI 0.29 to 1.89). (2) Health-care resource data were more complete using patient files than questionnaires. The probability that the intervention is cost-effective is sensitive to the source of data used and whether or not data on intended pregnancies are included. (3) The pilot RCT fitted well around clinical activities but 37% of the intervention group did not see the Men’s Safer Sex website and technical problems were frustrating. Men’s views of the Men’s Safer Sex website and research procedures were largely positive.ConclusionsIt would be feasible to conduct a large-scale RCT using clinic STI diagnoses as a primary outcome; however, technical errors and a poor response rate limited the collection of online self-reported outcomes. The next steps are (1) to optimise software for online trials, (2) to find the best ways to integrate digital health promotion with clinical services, (3) to develop more precise methods for collecting resource use data and (4) to work out how to overcome barriers to digital intervention testing and implementation in the NHS.Trial registrationCurrent Controlled Trials ISRCTN18649610.FundingThis project was funded by the NIHR Health Technology Assessment programme and will be published in full inHealth Technology Assessment; Vol. 20, No. 91. See the NIHR Journals Library website for further project information.

show abstract

“…In this study, it is impossible to determine whether intrinsic motivation to take part was affected by the incentive of 50p per additional session, and it would therefore be prudent to run a similar attrition study but without incentives. Money can be a powerful motivator; for example, (Khadjesari et al, 2011) showed found that offering a £10 Amazon voucher to each participant in a longitudinal study resulted in a 9% increased response rate. In our case, it may simply be that money was the most important factor for taking part, and that the similar attrition rates were driven by the identical incentives.…”

Section: Discussionmentioning

confidence: 99%

No effect of gamification on attrition from a longitudinal cognitive testing study

Lumsden¹,

Skinner²,

Coyle³

et al. 2017

Preprint

View full text Add to dashboard Cite

The prospect of assessing cognition longitudinally is attractive to researchers, health practitioners and pharmaceutical companies alike. However, such repeated-testing regimes place a considerable burden on participants, and with cognitive tasks typically being regarded as effortful and unengaging, these studies may experience high levels of participant attrition. One potential solution is to gamify these tasks to make them more engaging: increasing participant willingness to take part and reducing attrition. However, such an approach must balance task validity with introducing entertaining gamelike elements.We investigated the effects of gamelike features on participant attrition using a betweensubjects, longitudinal online testing study. We used three variants of a common cognitive task, the stop signal task, with a single gamelike feature in each: one variant where points were rewarded for performing optimally, another where the task was given a graphical theme, and a third variant which was a standard stop signal task and served as a control condition. Participants completed four compulsory test sessions over four consecutive days before entering a six-day voluntary testing period where they faced a daily decision to either drop out or continue taking part. Participants were paid for each session they completed.We saw no evidence for an effect of gamification on attrition, with participants dropping out of each variant at equal rates. Our findings raise doubts about the ability of gamification to increase engagement with cognitive testing studies.

show abstract

Impact and Costs of Incentives to Reduce Attrition in Online Trials: Two Randomized Controlled Trials

Cited by 105 publications

References 26 publications

Sexual health promotion for young people delivered via digital media: a scoping review

Sexual health promotion for young people delivered via digital media: a scoping review

The Men’s Safer Sex project: intervention development and feasibility randomised controlled trial of an interactive digital intervention to increase condom use in men

No effect of gamification on attrition from a longitudinal cognitive testing study

Contact Info

Product

Resources

About