Analysis of COVID-19 Guideline Quality and Change of Recommendations: A Systematic Review

Abstract

Background. Hundreds of coronavirus disease 2019 (COVID-19) clinical practice guidelines (CPGs) and expert consensus statements have been developed and published since the outbreak of the epidemic. However, these CPGs are of widely variable quality. So, this review is aimed at systematically evaluating the methodological and reporting qualities of COVID-19 CPGs, exploring factors that may influence their quality, and analyzing the change of recommendations in CPGs with evidence published. Methods. We searched five electronic databases and five websites from 1 January to 31 December 2020 to retrieve all COVID-19 CPGs. The assessment of the methodological and reporting qualities of CPGs was performed using the AGREE II instrument and RIGHT checklist. Recommendations and evidence used to make recommendations in the CPGs regarding some treatments for COVID-19 (remdesivir, glucocorticoids, hydroxychloroquine/chloroquine, interferon, and lopinavir-ritonavir) were also systematically assessed. And the statistical inference was performed to identify factors associated with the quality of CPGs. Results. We included a total of 92 COVID-19 CPGs developed by 19 countries. Overall, the RIGHT checklist reporting rate of COVID-19 CPGs was 33.0%, and the AGREE II domain score was 30.4%. The overall methodological and reporting qualities of COVID-19 CPGs gradually improved during the year 2020. Factors associated with high methodological and reporting qualities included the evidence-based development process, management of conflicts of interest, and use of established rating systems to assess the quality of evidence and strength of recommendations. The recommendations of only seven (7.6%) CPGs were informed by a systematic review of evidence, and these seven CPGs have relatively high methodological and reporting qualities, in which six of them fully meet the Institute of Medicine (IOM) criteria of guidelines. Besides, a rapid advice CPG developed by the World Health Organization (WHO) of the seven CPGs got the highest overall scores in methodological (72.8%) and reporting qualities (83.8%). Many CPGs covered the same clinical questions (it refers to the clinical questions on the effectiveness of treatments of remdesivir, glucocorticoids, hydroxychloroquine/chloroquine, interferon, and lopinavir-ritonavir in COVID-19 patients) and were published by different countries or organizations. Although randomized controlled trials and systematic reviews on the effectiveness of treatments of remdesivir, glucocorticoids, hydroxychloroquine/chloroquine, interferon, and lopinavir-ritonavir for patients with COVID-19 have been published, the recommendations on those treatments still varied greatly across COVID-19 CPGs published in different countries or regions, which may suggest that the CPGs do not make sufficient use of the latest evidence. Conclusions. Both the methodological and reporting qualities of COVID-19 CPGs increased over time, but there is still room for further improvement. The lack of effective use of available evidence and management of conflicts of interest were the main reasons for the low quality of the CPGs. The use of formal rating systems for the quality of evidence and strength of recommendations may help to improve the quality of CPGs in the context of the COVID-19 pandemic. During the pandemic, we suggest developing a living guideline of which recommendations are supported by a systematic review for it can facilitate the timely translation of the latest research findings to clinical practice. We also suggest that CPG developers should register the guidelines in a registration platform at the beginning for it can reduce duplication development of guidelines on the same clinical question, increase the transparency of the development process, and promote cooperation among guideline developers all over the world. Since the International Practice Guideline Registry Platform has been created, developers could register guidelines prospectively and internationally on this platform.

COVID-19 vaccine guidelines was numerous in quantity but many lack transparent reporting of methodological practices

“…The number is close to the number of clinical practice guidelines of COVID-19. [16] The majority of these guidelines were published in peer-reviewed journals in English by developers from high-income countries such as Canada and the United States. There is a lack of guidelines for COVID-19 vaccines from low- and middle-income countries and published in languages other than English and a lack of recommendations for these countries.…”

“… [7] To address this problem, practice guidelines have been developed by many organizations and institutions aiming to translate the research findings into recommendations for practice and to provide a guidance for the appropriate use of COVID-19 vaccination. [8] , [9] , [10] , [11] , [12] , [13] , [14] , [15] , [16] There are already many different types of guidelines at present, some of the guidelines are brief, others lengthy, however, the overall picture of the vaccine guidelines is unclear. To our knowledge so far, no study has comprehensively summarized the characteristics of these guidelines.…”

Evaluation of the quality of COVID-19 guidance documents in anaesthesia using the Appraisal of Guidelines for Research and Evaluation II instrument

“…Another potential improvement to guidance document development is a prospective register which could reduce duplication and foster greater collaboration. 18 This study demonstrates that while the general quality of guidelines in anaesthesia is recognized as poor, the guidance documents produced during COVID-19 offered an even lower standard, despite readily available aids such as appraisal tools. Based on the findings of this study, a suggested ideal COVID-19 guidance document in anaesthesia would utilise an appraisal tool during development, take a collaborative approach with multi-institutional and international involvement and adhere to the requirements of AGREE II, particularly Domains 1,3,4 and 6.…”

“…16 This has been amplified during the COVID-19 era due to lack of an existing evidence base, purported absence of rigour in some guideline development strategies, use of non-standard document formats, and use of confusing terminology. 17 , 18 Several publications have detailed the quality of anaesthesia guidelines and the quality of COVID-19 recommendations in other fields of medicine. 15 , 17 , 19 One small anaesthesia study offered an early indication as to the standard of COVID-19 guidance.…”

Guidelines for the prevention and management of children and adolescents with COVID-19

“…Since the beginning of the COVID-19 outbreak, close to 100 international and national clinical practice guidelines (CPGs) for the management of adult COVID-19 patients have been developed [ 3 ]. However, there are so far only few evidence-based guidelines specifically focusing on pediatric COVID-19 [ 4 ].…”