Indirect Treatment Comparison/Network Meta-Analysis Study Questionnaire to Assess Relevance and Credibility to Inform Health Care Decision Making: An ISPOR-AMCP-NPC Good Practice Task Force Report

Abstract

Despite the great realized or potential value of network meta-analysis of randomized controlled trial evidence to inform health care decision making, many decision makers might not be familiar with these techniques. The Task Force developed a consensus-based 26-item questionnaire to help decision makers assess the relevance and credibility of indirect treatment comparisons and network meta-analysis to help inform health care decision making. The relevance domain of the questionnaire (4 questions) calls for assessments about the applicability of network meta-analysis results to the setting of interest to the decision maker. The remaining 22 questions belong to an overall credibility domain and pertain to assessments about whether the network meta-analysis results provide a valid answer to the question they are designed to answer by examining 1) the used evidence base, 2) analysis methods, 3) reporting quality and transparency, 4) interpretation of findings, and 5) conflicts of interest. The questionnaire aims to help readers of network meta-analysis opine about their confidence in the credibility and applicability of the results of a network meta-analysis, and help make decision makers aware of the subtleties involved in the analysis of networks of randomized trial evidence. It is anticipated that user feedback will permit periodic evaluation and modification of the questionnaire.

Bibliographic study showed improving statistical methodology of network meta-analyses published between 1999 and 2015

“…In recent years, the methodology of NMA has been further refined and many tutorials and guidance papers have been published [12,[27][28][29][30][31]. Efthimiou inconsistency [33,34].…”

Additional considerations are required when preparing a protocol for a systematic review with multiple interventions

“…Also, in a NMA the impact of potential effect modifiers needs to be evaluated in the context of the transitivity assumption. The plausibility of transitivity can be judged by comparing the distribution of the potential effect modifiers across the available direct comparisons in the network (10,22,46). Therefore, data extraction on these characteristics is necessary rather than optional.…”

“…The validity of NMA relies on the transitivity assumption. As discussed above, the assumption of transitivity can be assessed by comparing the distribution of the potential effect modifiers across the direct comparisons in the network (10,46). However, the utility of this approach is sometimes limited by the fact that effect modifiers are under-reported, or that very few studies are available for each direct comparison.…”

“…Such reviews, which we refer to as comparative effectiveness reviews (CER), may include a data synthesis component, network meta-analysis (NMA). The popularity of the technique prompted the rapid methodological development of NMA, the availability of technical guidance, statistical tutorials (3)(4)(5)(6)(7), and recommendations for the proper conduct and reporting of CER (8)(9)(10)(11)(12). Good practice requirements of a standard systematic review apply equally to a CER, although several practices need to be adapted to accommodate the complexity imposed by comparing more than two interventions at a time.…”

FDAMA Section 114: Why the Renewed Interest?

“…34 Methodology standards and criteria for decision makers to use when assessing the available CER data are under development by organizations such as the CER Collaborative, of which the AMCP is a member. [35][36][37] This type of effort supports managed care pharmacists in sorting through the relevance and credibility of the information communicated to them in published studies and through promotional vehicles.…”