Quality and Reporting Completeness of Systematic Reviews and Meta-Analyses in Dermatology

Abstract

We sought to assess the quality of dermatological systematic reviews (SRs) and identify factors that predict high methodological quality. We searched for all SRs published in 2017 using PubMed, Epistemonikos, and the Cochrane Database of SRs. We included studies identified as SRs or meta-analysis in the title or abstract and dealing with a dermatological topic. Study selection and data extraction were carried out and Preferred Reporting Items for SRs and Meta-Analyses and rating by A MeaSurement Tool to Assess SRs 2 were used independently by two authors. On the basis of A MeaSurement Tool to Assess SRs 2, confidence in SRs results was classified as high, moderate, low, or very low. We included 732 studies. We described a random sample of 140. The overall rating of confidence in the results according to a tool called A MeaSurement Tool to Assess SRs 2 was high or moderate for nine reviews (6%). A total of 20 reviews (15%) had a registered protocol. Independent factors associated with moderate or high rating of A MeaSurement Tool to Assess SRs 2 were publication in a journal where Preferred Reporting Items for SRs and Meta-Analyses was mandatory (OR [95% confidence interval] ¼ 27.0 [1.4e528]) and journal impact factor (OR of 1.9 [1.3e3]) for each increase in one more point. The observation that 90% of published dermatology SRs are of very low quality is alarming. Review registration in the International Prospective Register of SRs and full reporting according to Preferred Items for SRs and Meta-Analyses should be mandatory for publication. This study is registered in the International Prospective Register of SRs (CRD42018093856).

Avoidable research waste in dermatology: what is the problem?

“…Despite my enthusiasm for systematic reviews, it is with regret that I now think they have spun completely out of control and have become a ‘sausage machine’ 7 . Around 90% of dermatology systematic reviews are of poor quality 8 . Redundant duplication is also rife – there are now over 20 network meta‐analyses in the field of systemic treatments for psoriasis – is that a good use of resources?…”

“…7 Around 90% of dermatology systematic reviews are of poor quality. 8 Redundant duplication is also rifethere are now over 20 network meta-analyses in the field of systemic treatments for psoriasis is that a good use of resources?…”

Quality of systematic reviews on timing of complementary feeding for early childhood allergy prevention

“…SRs provide summaries of the accumulated evidence accounting for risk of bias (RoB) and assess the certainty of the evidence, which in turn aids translation into clinical practice. It has been shown, however, that the methodological quality of SRs examining the same primary studies varies, [ 25 , 27 ] which may lead to conflicting and misleading evidence syntheses, impeding the translation of knowledge to practice [ 25 ]. Consequently, substantial efforts have been made to develop tools that assist in the systematic assessment of the methodological quality of SRs and RoB in SRs.…”

Are Dermatology Systematic Reviews Spinning Out of Control?

“…The field has become a “sausage machine” industry that undermines the value of systematic reviews in providing a summary of the best evidence to inform patient care [8]. A recent review of 140 dermatology systematic reviews found that around 90% were low quality [9], suggesting that almost anyone can get a publication these days with the words “systematic review” in the title. In his article entitled “The Mass Production of Redundant, Misleading, and Conflicted Systematic Reviews and Meta-analyses,” Ioannidis [10] comments that: “Instead of promoting evidence-based medicine and health care, these instruments often serve mostly as easily produced publishable units or marketing tools.” Sadly, the same has happened to systematic reviews in dermatology, and the sin of spin can be appended to the seven top sins of dermatology systematic reviews that now haunt me (Table 1).…”