Evidence-based Decision Making: Enhancing Systematic Reviews of Program Evaluation Results in Europe

Abstract

Over the last 25 years, meta-analysis has been widely used to study the effects of practical treatment interventions in fields ranging from psychology and education to medicine and public health. The present article first describes the impact of meta-analysis. The article then presents a description of some preliminary results of a study of the main design characteristics of published interventions in Europe. In order to foster both better experimental designs and more systematic reviews in the European context, and to promote collaboration between different countries and research groups, the authors then describe and discuss the Campbell Collaboration (C2), a new international organization aimed at fostering public policies and practices based on systematic reviews of high-quality evidence. Though there is some overlap between the work of the Campbell Collaboration and its sibling organization the Cochrane Collaboration, Campbell focuses more on systematic reviews in the social, behavioral and educational areas. Reviewers in those areas encounter difficulties over the use of non-randomized designs, difficulties in effect-size computation, and the need for new computer software to better serve the needs of meta-analytic reviews in the social sciences.

Adopting an Evidence-Based Approach to Prevent and Intervene against Workplace Bullying Situations

“…Shadish et al [16] analyzed intervention studies in the areas of psychology, education, medicine and public health and showed that only 16% of those studies clearly specified their theoretical orientation and that more than 70% did not offer any information about the theoretical framework in which the intervention was based on. In the case of workplace bullying, results are similar.…”

Assessing the effectiveness of conflict management training in a health sector organization: evidence from subjective and objective indicators

“…The main goal of this study, then, is to assess the effectiveness of conflict management skills training provided to workers from a healthcare organization, at the same time seeking to overcome certain limitations of previous interventions conducted in organizational settings (Shadish, Chacon-Moscoso, & Sanchez-Meca, 2005), such as: (a) lack of information about the theoretical rationale used to underpin and guide the implementation and evaluation of the intervention; (b) lack of (quasi-) experimental designs with baseline data and follow-up periods longer than six months, which complicates causal interpretation and evidence-based practice; and (c) lack of multiple information sources to assess the effectiveness of the intervention.…”

The Development of a Checklist to Enhance Methodological Quality in Intervention Programs

“…There are no clear criteria for choosing the optimal tool to measure methodological quality. This occurs especially since it is common to use non-randomized studies in social sciences (Shadish et al, 2005). This is due to a shortage of instruments that (a) are rigorously developed and (b) have reliability and/or validity evidence with tested R (Crowe and Sheppard, 2011).…”

“…In this case, we did not find reliability and validity evidence (weak R). Furthermore, there was lack of U in social sciences because it is only applicable for randomized control trials (Shadish et al, 2005). Finally, at least two of the items (incomplete outcome data and selective reporting) are difficult to assess (weak F).…”