Assessment of Methodologic Quality of Randomized Trials of Interventional Techniques: Development of an Interventional Pain Management Specific Instrument

Abstract

Background: A major component of a systematic review is an assessment of the methodological quality and bias of randomized trials. The most commonly utilized methodological quality assessment and bias assessment for randomized trials is by the Cochrane Review Group. While this is not a “gold standard,” it is an indication of the current state-of-the-art review methodology. There is, however, no specific instrument to assess the methodological quality of manuscripts published for interventional techniques. Objectives: Our objective was to develop an instrument specifically for interventional pain management, to assess the methodological quality of randomized trials of interventional techniques. Methods: Item generation for the instrument was based on a definition of quality, to the extent to which the design and conduct of the trial were congruent with the objectives of the trial. Applicability was defined as the extent to which the trial produced procedures could be applied with contemporary interventional pain management techniques. Multiple items based on Cochrane review criteria were utilized along with specific requirements for interventional techniques. Results: A total of 22 items were developed which formed IPM-QRB or Interventional Pain Management Techniques – Quality Appraisal of Reliability and Risk of Bias Assessment tool. This included 9 of the 12 items from the Cochrane review criteria with deletion of some items that were repetitive or duplicate, and the addition of 13 new items. The results were compared for inter-rater reliability of Cochrane review criteria and IPM-QRB, and interinstrument reliability. The assessment was performed in multiple stages with an initial learning curve. The final assessment was for 4 randomized controlled trials (RCTs) utilizing both Cochrane review criteria and IPM-QRB criteria. The inter-rater agreement for Cochrane review criteria with overall intra-class correlation coefficient was 0.407 compared to an intra-class correlation coefficient of 0.833 for IPM-QRB criteria. The inter-rater agreement was superior for IPM-QRB criteria compared to Cochrane review criteria despite twice the items of Cochrane review criteria as IPM-QRB criteria with the detailed nature of assessment. Limitations: Limited validity or accuracy assessment of the instrument and the large number of items to be scored. Conclusion: We have developed a new comprehensive instrument to assess the methodological quality of randomized trials of interventional techniques. This instrument is superior to Cochrane review methodology criteria in that it provides more extensive and specific information for interventional techniques that will be useful in assessing the methodologic quality and bias of interventional techniques. Key words: Methodological quality assessment, evidence-based medicine, comparative effectiveness research, Cochrane Reviews, interventional techniques, risk of bias assessment

The role of percutaneous neurolysis in lumbar disc herniation: systematic review and meta-analysis

“…Of the 6 manuscripts meeting inclusion criteria [ 25 , 31 , 33 , 37 , 38 , 40 ], there was only one placebo-controlled RCT [ 25 ]. Appendix Tables 4 and 5 show the methodologic quality assessment and risk of bias of the one RCT utilizing the Cochrane review criteria and the IPM-QRB criteria respectively [ 27 , 28 ]. Assessment by the Cochrane review criteria and IPM-QRB of this RCT showed high quality [ 25 ].…”

“…Qualitative analysis was performed utilizing a modified approach for the grading of evidence [ 28 ] with moderate (level II) evidence from one relevant high-quality RCT and 5 relevant moderate-quality observational studies. All of the studies consistently showed improvement in patients undergoing neurolysis at 6 months and one-year follow-up periods.…”

“…The quality of each RCT was assessed using the Cochrane review rating system ( Appendix Table 1 ) [ 27 ] and Interventional Pain Management Techniques – Quality Appraisal of Reliability and Risk of Bias Assessment Tool (IPM-QRB) for RCTs ( Appendix Table 2 ) [ 28 ]. Non-randomized or observational studies were assessed by Interventional Pain Management Techniques – Quality Appraisal of Reliability and Risk of Bias Assessment for nonrandomized or observational studies (IPM-QRBNR) ( Appendix Table 3 ) [ 29 ].…”

The American Society of Pain and Neuroscience (ASPN) Best Practices and Guidelines for the Interventional Management of Cancer-Associated Pain

“…Quality of evidence for each treatment class was graded from I-III as detailed in (Table 1) and a degree of recommendation was provided on a scale from A to D, or as insufficient (I), according to the United States Preventive Services Task Force (USPSTF) criteria (Table 2). [23][24][25]…”

Is Epidural Injection of Sodium Chloride Solution a True Placebo or an Active Control Agent? A Systematic Review and Meta-Analysis

“…The quality of each individual article used in this analysis was assessed by Cochrane review (76) criteria and Interventional Pain Management techniques--Quality Appraisal of Reliability and Risk of Bias Assessment (IPM-QRB) for randomized trials (77).…”