Outcomes for implementation science: an enhanced systematic review of instruments using evidence-based rating criteria

Abstract

BackgroundHigh-quality measurement is critical to advancing knowledge in any field. New fields, such as implementation science, are often beset with measurement gaps and poor quality instruments, a weakness that can be more easily addressed in light of systematic review findings. Although several reviews of quantitative instruments used in implementation science have been published, no studies have focused on instruments that measure implementation outcomes. Proctor and colleagues established a core set of implementation outcomes including: acceptability, adoption, appropriateness, cost, feasibility, fidelity, penetration, sustainability (Adm Policy Ment Health Ment Health Serv Res 36:24–34, 2009). The Society for Implementation Research Collaboration (SIRC) Instrument Review Project employed an enhanced systematic review methodology (Implement Sci 2: 2015) to identify quantitative instruments of implementation outcomes relevant to mental or behavioral health settings.MethodsFull details of the enhanced systematic review methodology are available (Implement Sci 2: 2015). To increase the feasibility of the review, and consistent with the scope of SIRC, only instruments that were applicable to mental or behavioral health were included. The review, synthesis, and evaluation included the following: (1) a search protocol for the literature review of constructs; (2) the literature review of instruments using Web of Science and PsycINFO; and (3) data extraction and instrument quality ratings to inform knowledge synthesis. Our evidence-based assessment rating criteria quantified fundamental psychometric properties as well as a crude measure of usability. Two independent raters applied the evidence-based assessment rating criteria to each instrument to generate a quality profile.ResultsWe identified 104 instruments across eight constructs, with nearly half (n = 50) assessing acceptability and 19 identified for adoption, with all other implementation outcomes revealing fewer than 10 instruments. Only one instrument demonstrated at least minimal evidence for psychometric strength on all six of the evidence-based assessment criteria. The majority of instruments had no information regarding responsiveness or predictive validity.ConclusionsImplementation outcomes instrumentation is underdeveloped with respect to both the sheer number of available instruments and the psychometric quality of existing instruments. Until psychometric strength is established, the field will struggle to identify which implementation strategies work best, for which organizations, and under what conditions.Electronic supplementary materialThe online version of this article (doi:10.1186/s13012-015-0342-x) contains supplementary material, which is available to authorized users.

Validation of the Brazilian Version of WHODAS 2.0 in Patients with Mental Disorders: Should the 12-Item Scale be an Alternative to 36-Item Scale in DSM-5?

“…The patients who refused to participate or dropped out of the study tend to be different from those who agreed and remained [12][13][14]; however this situation should not be considered as selection bias, because the viability and acceptability of the instrument determine its clinical universe [15,16]. Based on the results, it was seen that the Brazilian Portuguese version of the WHODAS 2.0 PRO measure met all the statistical requirements to be considered valid and reliable (reproducible and responsive), and can be used safely in Brazil in patients with mental disorders and can even distinguish them from patients with non-psychiatric illnesses.…”

The Cost to Successfully Apply for Level 3 Medical Home Recognition

“…Work is currently underway to develop, validate, and disseminate such instruments within the developing field of implementation science. 30,31 As health care organizations continue to strive to achieve the Institute for Healthcare Improvement's triple aim, these instruments will be critical to the rigorous evaluation of the sustainability of innovations and emerging interventions.…”

Measuring the Implementation of Behavioral Intervention Technologies: Recharacterization of Established Outcomes (Preprint)

“…Moreover, the overlap between the conceptual and practical aspects of usability and implementation outcomes highlights what others have written about overlap within implementation outcomes themselves, mainly that there may be theoretical divisions between these outcomes but little empirical evidence or practical means to measure distinctions between them. [60] Again, further empirical work in this area is needed.…”