Updating the evidence on the effectiveness of the alcohol reduction app, Drink Less: using Bayes factors to analyse trial datasets supplemented with extended recruitment

Abstract

Background: A factorial experiment evaluating the Drink Less app found no clear evidence for main effects of enhanced versus minimal versions of five components but some evidence for an interaction effect. Bayes factors (BFs) showed the data to be insensitive. This study examined the use of BFs to update the evidence with further recruitment. Methods: A between-subject factorial experiment evaluated the main and two-way interaction effects of enhanced versus minimal version of five components of Drink Less. Participants were excessive drinkers, aged 18+, and living in the UK. After the required sample size was reached (n=672), additional data were collected for five months. Outcome measures were change in past week alcohol consumption and Alcohol Use Disorders Identification Test (AUDIT) score at one-month follow-up, amongst responders only (those who completed the questionnaire). BFs (with a half-normal distribution) were calculated (BF<0.33 indicate evidence for null hypothesis; 0.33<BF<3 indicate data are insensitive). Results: Of the sample of 2586, 342 (13.2%) responded to follow-up. Data were mainly insensitive but tended to support there being no large main effects of the enhanced version of individual components on consumption (0.22<BF<0.83) or AUDIT score (0.14<BF<0.98). Data no longer supported there being two-way interaction effects (0.31<BF<1.99). In an additional exploratory analysis, participants receiving four of the components averaged a numerically greater reduction in consumption than those not receiving any (21.6 versus 12.1 units), but the data were insensitive (BF=1.42). Conclusions: Data from extended recruitment in a factorial experiment evaluating components of Drink Less remained insensitive but tended towards individual and pairs of components not having a large effect. In an exploratory analysis, there was weak, anecdotal evidence for a synergistic effect of four components. In the event of uncertain results, calculating BFs can be used to update the strength of evidence of a dataset supplemented with extended recruitment.

A Personalized Approach Bias Modification Smartphone App (“SWiPE”) to Reduce Alcohol Use: Open-Label Feasibility, Acceptability, and Preliminary Effectiveness Study

“…Taken together, these findings suggest that smartphone-delivered ApBM holds promise. However, since controlled trials of delivering ApBM online have found equivalent reductions in active ApBM when compared with sham training and the only prior RCT of an alcohol-reduction app that included ApBM reported null findings [ 28 ], it remains necessary to establish the efficacy of smartphone-delivered ApBM in RCTs. This is a particularly worthwhile investment given the greater convenience, flexibility, and accessibility (eg, notification reminders and immediacy of access) that can be offered via smartphones relative to web-based platforms (eg, via a PC or laptop).…”

“…In the United Kingdom, Crane et al [ 27 ] tested apps containing various combinations of 5 different modules (including an ApBM module) among people drinking at hazardous levels. Despite initially reporting that combining ApBM with normative feedback reduced participants’ weekly alcohol consumption [ 27 ], they later reported a lack of evidence for efficacy after re-analyzing outcomes with a larger sample [ 28 ]. In the Netherlands, Laurens et al [ 29 ] tested an ApBM app with people who were concerned about, or wished to reduce, their drinking.…”

Refining the content and design of an alcohol reduction app, Drink Less, to improve its usability and effectiveness: a mixed methods approach

“…The Bayes factor analysis of additional trial outcome data (Phase 1) indicated that there was support for the null hypothesis that the ‘Identity Change’ module – helping users foster a change from seeing being a “drinker” as a key part of their identity – had no effect on past week alcohol consumption ( Garnett et al , 2019b ). These findings, alongside results from analysing user feedback (Phase 1) and usage data indicating users very rarely viewed these screens, guided the decision to remove the module.…”

“…Bayes factors also allow researchers to ‘top-up’ their results from one trial with additional data collected ( Rouder, 2014 ). In the factorial trial, Bayes factors showed the data to be insensitive so an update was conducted to analyse additional trial outcome data collected through extended recruitment (published in full elsewhere ( Garnett et al , 2019b )). After the required sample size was reached (n = 672), additional data from new participants (n = 1914) were collected over five months (available in Underlying data: https://osf.io/nad4e ) Bayes factors with a half-normal distribution and priors set to the point estimates from the initial 672 participants were calculated.…”

Reducing Risky Alcohol Use via Smartphone App Skills Training Among Adult Internet Help-Seekers: A Randomized Pilot Trial

“…One, targeting internet help-seekers with harmful levels of alcohol use (15 or more drinks/week), did not find any effects over 6 months, although some benefit appeared to occur among those who actually downloaded the app (23). A fourth smartphone app, Drink Less, evaluated among adult help-seekers with at least hazardous alcohol use who all downloaded the app, showed declines in alcohol use over time among all participants, where self-monitoring and feedback combined led to use of more app sessions (24); secondary analysis using Bayes factors showed weak evidence for an interactive combined effect of four components (normative feedback, cognitive bias retraining, self-monitoring and feedback, and action planning) yielding lower alcohol consumption (25).…”