[Commentary] IS THIS THE PATH TO EFFECTIVE PREVENTION?

Hypothesis testing and the EU-Dap evaluation of the Unplugged curriculum

“…Attempting to verify hypotheses is fundamentally at odds with the practice of science (Campbell, 1988;Popper, 1963, ch.1), but unfortunately such an approach is present in much substance abuse prevention research (Gorman, 2005b). It is this failure to critically test hypotheses, rather than a general skepticism about primary prevention, that has led to the growing literature that examines the claims made by substance abuse prevention researchers that their programs are supported by data from "science-based" evaluations (e.g., Gandhi et al, 2007;Gorman, 2005b;Littell, 2005;Midford, 2008).…”

[Commentary] FINDING A PATH TO MORE REASONABLE CONCLUSIONS ABOUT PREVENTION: A RESPONSE TO MIDFORD

“…[1], Dr Midford expresses his 'unease about the paradigm that underpins effort' in the area of universal preventive intervention research. He raises general issues about the role of universal preventive interventions vis-à-vis other strategies that potentially mitigate the consequences of substance misuse, as well as issues about methods for evaluating intervention efficacy.…”

“…In any event, our use of directional hypotheses is transparent; any reader who disagrees with this usage could readily convert the reported P-values accordingly. Also, it is simply inaccurate to state that an alpha was 'set at 0.10' (p. 1169 [1]). Our reports follow the social science standard of reporting P values associated with a range of alpha levels, clearly labeling findings at or below 0.10 as 'marginally significant' (e.g.…”

Further Comments on the Path to Drawing Reasonable Conclusions About Prevention

“…In their response to Midford [1], Spoth et al [2, p. 1171] state that one of the two central questions that the 'critical reader' would ask of their research is: 'What are the most valid and reasonable conclusions to be drawn about intervention efficacy from study observations?'. They had previously drawn the conclusion that the observations they reported demonstrated 'the effectiveness of the tested interventions on life-time or annual prescription drug misuse' [9, p. 7].…”

“…We were interested to see the exchange in Addiction concerning the efficacy of a number of drug prevention programs that have been evaluated by the Partnerships in Prevention Science Institute [1,2], especially as one of the authors drew upon a recent critique that we wrote of the published evaluations of one of these programs [3]. The broader concern underlying our critique was that what has come to be termed 'prevention science' in the United States is not informed by the critical and skeptical approach to hypothesis testing that is the hallmark of science [4,5].…”