Avoiding type III error in program evaluation

Dobson, Douglas; Cook, Thomas J.

doi:10.1016/0149-7189(80)90042-7

Cited by 224 publications

(140 citation statements)

References 4 publications

Supporting

Mentioning

136

Contrasting

Unclassified

Order By: Relevance

“…As to the degree of implementation, the emphasis in intervention studies is mostly placed on effectiveness (48). This may increase the risk for type-III-error, namely to assume that there is no effect of an intervention when actually the intervention was not fully implemented as planned (49). To gain insight into the degree of implementation, a process evaluation was conducted alongside the evaluation of the Vital@Work intervention (50).…”

Section: Strijk Et Almentioning

confidence: 99%

Effectiveness of a worksite lifestyle intervention on vitality, work engagement, productivity, and sick leave: results of a randomized controlled trial

Strijk¹,

Proper²,

Mechelen³

et al. 2012

Scand J Work Environ Health

105

View full text Add to dashboard Cite

Good quality effect studies (RCT) concerning specifically older workers are still lacking. Therefore, the results of this study are innovative and provide valuable information for occupational health epidemiology as well as employers. As for policy implications, implementation of worksite yoga facilities could be a useful strategy to promote vitality-related work outcomes, but only if high compliance can be maximized.

show abstract

Section: Strijk Et Almentioning

confidence: 99%

Effectiveness of a worksite lifestyle intervention on vitality, work engagement, productivity, and sick leave: results of a randomized controlled trial

Strijk¹,

Proper²,

Mechelen³

et al. 2012

Scand J Work Environ Health

105

View full text Add to dashboard Cite

show abstract

“…Not recognizing this can lead to what Diane Dobson and Thomas Cook call ''type III errors,'' or evaluating interventions that one assumes were implemented properly when they were actually inappropriately carried out or were never truly implemented. 83 Similar to the need for formative evaluation, collecting and integrating a variety of data can inform program design and revision decisions. The qualitative information from focus groups and interviews provides important input into the selection and revision of change methods, recruitment and retention strategies, and program effects from the consumer's perspective.…”

Section: Promoting Resilience In Children 357mentioning

confidence: 99%

Prevention and Intervention Strategies for Promoting Resilience in Disadvantaged Children

Smokowski

1998

Social Service Review

View full text Add to dashboard Cite

This article addresses the emergence of a resilience-based prevention practice perspective that focuses on positively affecting the development of disadvantaged, at-risk children. Significant progress has been made in understanding risk and resilience processes; however, use of the field's advances in applied settings has lagged. The article will attempt to bridge this gap by reviewing relevant issues in program design, implementation, and evaluation from a resilience perspective. Risk and resilience dynamics are briefly highlighted to illuminate theoretical routes for promoting positive adaptation. Trends in constructing preventive programs are underscored, focusing on ecological routes to behavioral and environmental change. Finally, prevention and early intervention programs for disadvantaged children ages 3 -9 illustrate issues in program conception and effectiveness. Methodological concerns in evaluation of these programs are discussed, and future recommendations are given.In the past 2 decades, enthusiasm for investigating childhood resilience and preventive interventions has developed on parallel trajectories. Both areas have enjoyed growing public attention and have begun to coalesce fragmentary research attempts into cohesive bodies of research. Yet growth has not occurred without stumbling blocks. The study of resilience has struggled to emerge from the overarching shadow of pathology, and prevention programs have been accused of lacking strong theoretical foundations or rigorous evaluation. 1 With common roots in epidemiological mapping of risk processes, these estranged disciplines are just beginning to rediscover each other and intertwine more effectively to reinforce practice applications and

show abstract

“…As a consequence, potentially effective programs may be discontinued due to findings of no or negative effects, when the results are not the consequence of an intervention not being effective, but rather that the program was not delivered properly, or at all (Dobson and Cook 1980, p. 270). In the context of quantitative impact evaluations, we are often concerned with type I and type II errors, and in a similar vein measuring something that does not exist has been described as committing a type III error (Scanlon et al 1977, p. 36, cited in Dobson andCook 1980). …”

Section: Methodsological Concernsmentioning

confidence: 99%

Systematic reviews: from ‘bare bones’ reviews to policy relevance

Snilstveit

2012

Journal of Development Effectiveness

View full text Add to dashboard Cite

Systematic reviews: from 'bare bones' reviews to policy relevance Birte SnilstveitTo cite this article: Birte Snilstveit (2012) Systematic reviews: from 'bare bones' reviews to policy relevance, Journal of Development Effectiveness, 4:3, 388-408, DOI: 10.1080/19439342.2012 Theory-based systematic reviews, which summarise evidence on what works, when and why, strive for greater policy relevance. Reviews that answer these questions adopt a mixed methods approach and draw on a range of study types. Answering the 'what works' and 'what doesn't' questions means drawing on effectiveness studies, conducted to standards of high-quality impact evaluation. But in formulating answers to the 'when' and 'why' questions requires a broader range of evidence from both quantitative and qualitative research. Based on a review of the methodological literature in this field and the experience of 3ie's systematic reviews program, this article provides an outline of how a theory-based approach to systematic reviews, including appropriate quantitative and qualitative evidence, can be operationalised. We propose an approach based on three principles: (1) develop a program theory for the intervention; (2) adopt a mixed methods approach and include a broader range of evidence; and (3) maintain the rigour and transparency that characterise systematic reviews. The approach translates into two broad options. Effectiveness plus reviews focus on providing a detailed causal chain analysis by drawing on a program theory and additional data collection on context and intervention implementation. And effectiveness plus with a parallel review component, which is designed to answer specific research questions related to effectiveness, adopts separate inclusion criteria, reflecting the type of studies appropriate for answering those questions.

show abstract

Avoiding type III error in program evaluation

Cited by 224 publications

References 4 publications

Effectiveness of a worksite lifestyle intervention on vitality, work engagement, productivity, and sick leave: results of a randomized controlled trial

Effectiveness of a worksite lifestyle intervention on vitality, work engagement, productivity, and sick leave: results of a randomized controlled trial

Prevention and Intervention Strategies for Promoting Resilience in Disadvantaged Children

Systematic reviews: from ‘bare bones’ reviews to policy relevance

Contact Info

Product

Resources

About