Letter to the Editor. Validity of the results of network meta-analysis

Abstract

Validity of the results of network meta-analysisTO THE EDITOR: We read the article by Shaffer et al. with interest (Shaffer A, Naik A, Bederson M, et al. Efficacy of deep brain stimulation for the treatment of anorexia nervosa: a systematic review and network meta-analysis of patient-level data. Neurosurg Focus. 2023;54[2]:E5). 1 The authors compared the efficacy of different stimulation targets for deep brain stimulation (DBS) in patients with anorexia nervosa (AN). The authors used patient-level data to identify stimulation targets with the greatest evidence for efficacy in increasing BMI. The authors also performed a network meta-analysis to compare different targets. The authors included 11 studies, including several case reports. A total of 36 patients were included in the network meta-analysis. We commend the authors for trying to present some evidence for the surgeons/physicians dealing with this difficult disease. However, the results of the network meta-analysis need to be analyzed critically.Network meta-analysis, or multiple arm meta-analysis, is a statistical method for simultaneously comparing multiple treatment arms. There are a few basic assumptions of a network meta-analysis that have to be met for the results of a network meta-analysis to be valid. One of these is exchangeability, which means that the included trials have very similar methodologies and there are no systematic differences among the included comparisons, except their compared interventions. [2][3][4] This can be ensured only if randomized controlled trials are included. However, most of the studies included in this network meta-analysis were nonrandomized trials. Moreover, the total sample size of the study was 36, which is not optimal for conducting a network meta-analysis. 3,4 The authors used the frequentist approach to perform network meta-analysis, which is susceptible to small sample size. These shortcomings make the results of this network meta-analysis unreliable.The authors have done a systematic review of the literature and presented the available evidence. We want the readers of this journal to be aware of these shortcomings of the network meta-analysis done because the conclusion of this study-"this study identified the [subcallosal cingulate cortex] as the most supported region for BMI change"-can be misleading. This may stop physicians from trying other targets for DBS in patients with AN, thus negatively impacting further research in the field.