Raising the bar: improving methodological rigour in cognitive alcohol research

Abstract

Background and Aims A range of experimental paradigms claim to measure the cognitive processes underpinning alcohol use, suggesting that heightened attentional bias, greater approach tendencies and reduced cue‐specific inhibitory control are important drivers of consumption. This paper identifies methodological shortcomings within this broad domain of research and exemplifies them in studies focused specifically on alcohol‐related attentional bias. Argument and analysis We highlight five main methodological issues: (i) the use of inappropriately matched control stimuli; (ii) opacity of stimulus selection and validation procedures; (iii) a credence in noisy measures; (iv) a reliance on unreliable tasks; and (v) variability in design and analysis. This is evidenced through a review of alcohol‐related attentional bias (64 empirical articles, 68 tasks), which reveals the following: only 53% of tasks use appropriately matched control stimuli; as few as 38% report their stimulus selection and 19% their validation procedures; less than 28% used indices capable of disambiguating attentional processes; 22% assess reliability; and under 2% of studies were pre‐registered. Conclusions Well‐matched and validated experimental stimuli, the development of reliable cognitive tasks and explicit assessment of their psychometric properties, and careful consideration of behavioural indices and their analysis will improve the methodological rigour of cognitive alcohol research. Open science principles can facilitate replication and reproducibility in alcohol research.

Intoxication without anticipation: Disentangling pharmacological from expected effects of alcohol

“…Finally, it should be noted that the visual prob task produced low reliability levels. While this has also been found in other research in this area (e.g., , future research may benefit from the development of more rigorous cognitive tasks and more appropriate stimuli (Pennington et al, 2021) to verify that the current findings can be replicated using other methodological approaches.…”

How laboratory studies of cigarette craving can inform the experimental alcohol craving literature

“…Better understanding of how attentional biases mediate drinking behavior may help advance efforts to leverage attentional bias interventions to reduce alcohol consumption (cf. Jones & Field, 2021;Pennington et al, 2021).…”

Reshaping the publication process: Addiction Research and Theory joins Peer Community In Registered Reports

“…In their 'manifesto for reproducible science ', Munaf o et al (2017) propose five measures that target these threats with the goal of improving research: (1) methods, (2) reporting and dissemination, (3) reproducibility, (4) evaluation, and (5) incentives. Adoption of open science practices will allow many of these measures to be achieved and calls for their implementation in addiction science are becoming louder (Heirene 2021;Louderback et al 2021;Pennington et al 2021). However, to be successful, academic journals also need to adapt and recentre their focus on scientific rigor rather than the metamorphosis of study results.…”

“…Furthermore, RRs enhance study quality (Soderberg et al 2020), enact higher levels of open data and computational reproducibility (Obels et al 2020), and attract higher altmetric scores and similar-to-slightly greater numbers of citations compared to the general (non-RR) literature (Hummer et al 2017). Despite their substantial benefits to both researchers and journals, ART represents one of only four addiction journals offering this publishing format to date (see Gorman 2019;Pennington et al 2021). As well as our journal guidelines, there are several practical guides for authors of RRs (Kiyonaga and Scimeca 2019;Stewart et al 2020; Center for Open Science 2021).…”