A Methodological and Reporting Quality Assessment of Systematic Reviews/Meta-Analyses about Chinese Medical Treatment for Gastroesophageal Reflux Disease

Abstract

Objective. To access the methodological and reporting quality of systematic reviews (SRs)/meta-analyses (MAs) about Chinese medical treatment for gastroesophageal reflux disease (GERD). Methods. The PubMed, Wanfang Data, China National Knowledge Infrastructure (CNKI), Chinese Science and Technology Periodical Database (VIP), Chinese Biomedical (CBM), Web of Science, and Cochrane Library databases were searched from inception to June 2020. Two researchers independently screened the literature considering the eligibility criteria. Overview Quality Assessment Questionnaire (OQAQ), Assessment of Multiple Systematic Reviews 2 (AMSTAR 2), and Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines were used to assess the methodological and reporting quality of the included reports. The Grading of Recommendations Assessment, Development and Evaluation (GRADE) system was used to evaluate the level of evidence in each report. Results. Thirty-three SRs/MAs met the inclusion criteria. The OQAQ results showed that defects in the methodological quality of 17/32 reports were major, with scores of 3 points. Analyzing a single item as the object, search strategies (item 2), and risk of bias in individual studies (item 4) was considered poor. The AMSTAR 2 results showed that 25.4% of the items were not reported, and 7.8% of the items were only partially reported. The overall assessment of AMSTAR 2 showed the majority of systematic reviews and meta-analyses were of low/very low (31/33, 93.9%) methodological quality, with a lack of protocol registration and excluded study list. The PRISMA results showed that 19.9% of items were not reported, and 15.2% of items were only partially reported, due to a lack of protocol registration and study selection methods. The methodological and reporting quality of the included studies was generally poor. Evidence evaluation with GRADE showed that most (31/33) of the included studies had low or very low levels of evidence. Conclusion. The methodological and reporting quality of SRs/MAs about Chinese medical treatment for GERD is generally poor. The main problems included incomplete search strategies, risk of bias in individual studies, the lack of protocol registration and excluded study list, and incorrect study selection methods.

Prevalence and methodological quality of systematic reviews in Korean medical journals

“…Item 10 (reporting funding sources of primary studies) was reported in fewer than 5% of SRs. This finding is similar to the proportions of 0%-30.3% reported in previous studies [6,[21][22][23]25]. Industry-funded studies sometimes show favor toward sponsors, due to which they are less likely to be published than independently funded studies [9,25,26].…”

“…Similarly low adherence to items 2 and 7 was reported in previous studies [6,[21][22][23]. Regarding item 2, protocol registration, only 14.3% of SRs in major KMJs registered or published their study protocol prior to conducting the SRs.…”

“…Significant contributors to low confidence were less than 50% adherence to domains, such as protocol registration (item 2), explanation of study designs selection (item 3), provision of a list of excluded studies (item 7), reporting of funding sources of primary studies (item 10), risk of bias interpretation (item 13), and failure to adhere to critical domains. Most previous studies using AMSTAR showed methodologically moderate quality overall [7,[13][14][15][16][17][18][19], whereas studies using AMSTAR 2 showed confidence levels ranging from low to critically low [5,6,8,[20][21][22][23]. Unlike AMSTAR, AMSTAR 2 does not classify quality based on a total score, but divides the items into 7 critical and 9 non-critical domains according to importance, and then classifies the quality according to each item rating [9].…”

“…Regarding item 2, protocol registration, only 14.3% of SRs in major KMJs registered or published their study protocol prior to conducting the SRs. Previous studies assessing the methodological quality of SRs in clinical areas have reported adherence to this item as 0% [ 6 , 22 ]. A written protocol with independent verification is recommended to reduce the risk of bias that may creep in during the process of conducting research [ 9 ].…”

Predictors of Higher Quality of Systematic Reviews Addressing Nutrition and Cancer Prevention

“…Despite the fact that several previous studies reported insufficient assessment of RoB of primary studies included in SR/MAs [ 21 , 28 , 29 , 30 , 31 ], we did not identify any studies that would formally test such a variable as a predictor of SR quality. However, previous studies showed that involvement of a ‘research methodologist’ predicts a higher quality of SRs.…”