Scales to Assess the Quality of Randomized Controlled Trials: A Systematic Review

Abstract

Background and PurposeThe methodological quality of randomized controlled trials (RCTs) is commonly evaluated in order to assess the risk of biased estimates of treatment effects. The purpose of this systematic review was to identify scales used to evaluate the methodological quality of RCTs in health care research and summarize the content, construction, development, and psychometric properties of these scales. MethodsExtensive electronic database searches, along with a manual search, were performed. ResultsOne hundred five relevant studies were identified. They accounted for 21 scales and their modifications. The majority of scales had not been rigorously developed or tested for validity and reliability. The Jadad Scale presented the best validity and reliability evidence; however, its validity for physical therapy trials has not been supported. Discussion and ConclusionMany scales are used to evaluate the methodological quality of RCTs, but most of these scales have not been adequately developed and have not been adequately tested for validity and reliability. A valid and reliable scale for the assessment of the methodological quality of physical therapy trials needs to be developed. T he medical literature is an important resource to guide clinical decision making and research. The evaluation of the methodological quality of studies is an essential step in the process of selecting the best clinical literature. According to Verhagen et al, 1 assessment of methodological quality involves evaluation of internal validity (the degree to which a study's design, conduct, and analysis have minimized biases) and external validity (the extent to which the results of a study can be generalized outside the experimental situation) as well as statistical analysis of primary research. Taken together, these validity constructs are important in determining the methodological quality of primary research. Khan et al 2 pointed out that some reasons for performing quality assessment include: to determine a minimum quality threshold for the selection of the primary studies for a systematic review; to explore differences in quality as an explanation for heterogeneity in study results; to weigh the results in proportion to the quality in metaanalysis; and, more importantly, to guide interpretation of findings, help determine the strength of inferences, and guide recommendations for future research and clinical practice.The assessment of the quality of controlled trials is essential because variations in the quality of trials can affect the conclusions about the existing evidence. 3 In a review of trials evaluating primarily medical treatments, Moher and colleagues 4,5 demonstrated that trials that did not include features such as blinding and allocation concealment tended to report an exaggerated treatment effect compared with trials that did include these features. These facts emphasize the importance of methodological quality assessment in order to provide accurate information on therapeutic effects.Trial quality can be divided ...

Effectiveness of Manual Therapy Combined With Physical Therapy in Treatment of Patellofemoral Pain Syndrome: Systematic Review

“…37 Furthermore, it has been proposed as a valid tool to be used by researchers to assess study protocols and to perform analysis based on the quality of trials. 21 It has been tested for reliability in different settings (intraclass correlation coefficient = 0.98), 37 and it presented the best validity evidence when compared with other scales.…”

The influence of home exercise programs for patients with non-specific or specific neck pain: a systematic review of the literature

“…14 In particular, this tool has strong face and content validity, as well as reproducibility (agreement/reliability). 27 Based on the application of this tool in the current literature, previous researchers determined the cut-off percentage values as ,50% indicating poor quality, 50-80% indicating moderate quality, and .80% indicating good quality.…”

Commentary Re: Botox Article by Berger & Knoll IPH 19(5):243–244 on “The efficacy of botulinum toxin type A in managing chronic musculoskeletal pain: a systematic review and meta analysis”

“…It is known that a wide range of scales and check-lists are employed in systematic reviews for assessing qualities of RCTs with Jadad scale being one of them (Juni et al 2001;Olivo et al 2008). We acknowledge that the scale is not without its flaws.…”

“…However, Jadad scale is easy to apply and assess important attributes of a RCT, such as randomization, blinding, and attrition biases (Jadad et al 1996;Juni et al 2001). Furthermore, it has been developed on the basis of rigorous validity and reliability testing (Olivo et al 2008). This is in contrast to other scales, most of which have not been appropriately evaluated (Olivo et al 2008).…”

“…Furthermore, it has been developed on the basis of rigorous validity and reliability testing (Olivo et al 2008). This is in contrast to other scales, most of which have not been appropriately evaluated (Olivo et al 2008).…”