The quality of systematic reviews and other synthesis in the time of COVID-19

Abstract

COVID-19 research has been produced at an unprecedented rate and managing what is currently known is in part being accomplished through synthesis research. Here we evaluated how the need to rapidly produce syntheses has impacted the quality of the synthesis research. Thus, we sought to identify, evaluate and map the synthesis research on COVID-19 published up to 10 July 2020. A COVID-19 literature database was created using pre-specified COVID-19 search algorithms carried out in eight databases. We identified 863 citations considered to be synthesis research for evaluation in this project. Four-hundred and thirty-nine reviews were fully assessed with A MeaSurement Tool to Assess systematic Reviews (AMSTAR-2) and rated as very low-quality (n = 145), low-quality (n = 80), medium-quality (n = 208) and high-quality (n = 151). The quality of these reviews fell short of what is expected for synthesis research with key domains being left out of the typical methodology. The increase in risk of bias due to non-adherence to systematic review methodology is unknown and prevents the reader from assessing the validity of the review. The responsibility to assure the quality is held by both producers and publishers of synthesis research and our findings indicate there is a need to equip readers with the expertise to evaluate the review conduct before using it for decision-making purposes.

Herbal Medicine and COVID-19: Characteristics and Methodological Quality of Published Reviews

“…As our study was not limited to systematic reviews only and had a different aim, our approach to analysis was more similar to another scoping review on the quality of systematic reviews and other synthesis research during the COVID-19 era. 5 Compared to the overall quality of general reviews on COVID-19 assessed by Baumeister et al, 5 the herbal medicine reviews included here were found to be of lower quality. In our study, one of the most evident reasons for rating down the quality of reviews is the lack of a prior protocol, an exclusion criterion for the previous evaluation by Baumeister et al, as these were deemed too low quality to be included in their review.…”

“…4 Accordingly, there has been a surge in herbal medicine review publications, which raises concerns about the quality of those reviews. 5 To date, several scoping and systematic reviews assessing the quality of COVID-19 reviews have been published, [5][6][7][8][9] of which three were about or related to herbal medicine. [7][8][9] These were interventional reviews performed to collate and assess systematic reviews, with the common primary objective of summarising the effectiveness of herbal medicine for COVID-19 while considering the quality of the evaluated systematic reviews.…”

Most published systematic reviews of remdesivir for COVID-19 were redundant and lacked currency

“…Although several meta-research studies have explored different aspects of COVID-19 research, such as volume, characteristics, and quality [ 2 , 4 , 73 , 74 ], this is the first study to our knowledge that has focused on the extent of duplication and currency among published systematic reviews of one intervention. In addition, looking at which studies were included in the reviews allowed us to contrast the utility of systematic reviews with living guidelines for informing up-to-date decisions.…”

A Comprehensive Meta-Analysis of COVID-19 in Latin America

“…Study design was based on a retrospective observational study. We combined the endpoints of the collected data from 8 different countries to detect the possible associations or trends between mortality and risk factors [16][17][18]. The adopted statistical methods were weighted descriptive statistics, t-test, independent samplestest and linear regression analysis as appropriate, where weights were associated with total number of cases in each country.…”