Cochrane and non-Cochrane systematic reviews in leading orthodontic journals: a quality paradigm?

Abstract

The aims of this study were to assess and compare the methodological quality of Cochrane and non-Cochrane systematic reviews (SRs) published in leading orthodontic journals and the Cochrane Database of Systematic Reviews (CDSR) using AMSTAR and to compare the prevalence of meta-analysis in both review types. A literature search was undertaken to identify SRs that consisted of hand-searching five major orthodontic journals [American Journal of Orthodontics and Dentofacial Orthopedics, Angle Orthodontist, European Journal of Orthodontics, Journal of Orthodontics and Orthodontics and Craniofacial Research (February 2002 to July 2011)] and the Cochrane Database of Systematic Reviews from January 2000 to July 2011. Methodological quality of the included reviews was gauged using the AMSTAR tool involving 11 key methodological criteria with a score of 0 or 1 given for each criterion. A cumulative grade was given for the paper overall (0-11); an overall score of 4 or less represented poor methodological quality, 5-8 was considered fair and 9 or greater was deemed to be good. In total, 109 SRs were identified in the five major journals and on the CDSR. Of these, 26 (23.9%) were in the CDSR. The mean overall AMSTAR score was 6.2 with 21.1% of reviews satisfying 9 or more of the 11 criteria; a similar prevalence of poor reviews (22%) was also noted. Multiple linear regression indicated that reviews published in the CDSR (P < 0.01); and involving meta-analysis (β = 0.50, 95% confidence interval 0.72, 2.07, P < 0.001) showed greater concordance with AMSTAR.

Methodological quality assessment of paper-based systematic reviews published in oral health

“…However, an established threshold for assigning descriptive quality or an overall quality score has not yet been established; therefore, a Bhigher^AMSTAR score, rather than a Blower^AMSTAR score, may help to reveal methodologically sound systematic reviews. Conversely, we identified some selected reviews, which showed that 20 % of included SR satisfied nine or more of the 11 AMSTAR criteria [17,18]. While it was out of the scope of this review to explore reasons for differences in methodological quality of SR across dental disciplines, the present results may serve as reference for further methodological development of SR published in these fields.…”

“…The three least common topics were restorative dentistry (n=15; 2.3 %), pain (n=11; 1.7 %), and other unclassified studies (n=25; 3.9 %). Although the median number of studies per SR was [15][16][17][18][19][20][21][22][23][24][25][26][27][28], the majority (70.2 %, n=450) included fewer than 24 studies in their review, while 2.0 % (n=13) included more than 100 studies. Most studies failed to describe study characteristics (88.2 %, n = 567), with terms such as Badults^(6.5 %, n = 42) or Bpaediatrics^(5.3 %, n = 34) populations used sparingly among the studies.…”

A PRISMA assessment of the reporting quality of systematic reviews in orthodontics

“…10 Most reviews were published in AJODO (31.2%), in Angle Orthodontist (29.4%), and on the CDSR (23.9%). In terms of adherence to individual criteria, the overall performance was assessed as good for reporting of the abstract, explanation of rationale for the review, listing of information sources and criteria for study selection, analysis of risk of bias (or quality) of primary studies, presentation of results from individual studies, summary of the evidence, and reporting of both limitations and conclusions.…”

“…Discrepancies between the authors in the grading of individual criteria were resolved by joint discussion. AMSTAR scores for the included studies have been reported in a prior study 10 ; in this study, the relationship between overall reporting (PRISMA) and quality (AMSTAR) scores for each review was to be assessed.…”

“…9 Previous studies have been undertaken on orthodontic systematic reviews to investigate the quality of the research using AMSTAR criteria. 10,11 It is unclear, however, whether reporting of systematic reviews is necessarily likely to reflect improved quality of the review process. The aims of this study were to assess the reporting quality of Cochrane and nonCochrane systematic reviews published in leading orthodontic journals and the Cochrane Database of Systematic Reviews (CDSR) with reference to PRISMA guidelines.…”