Pure or Tarnished: Are Systematic Reviews Blind or Biased?

Abstract

EditorialPure or Tarnished: Are Systematic Reviews Blind or Biased?I T SEEMS that the current mindset of the government, payors, and various watchdog groups is to hold the medical and scientific communities to practices and treatments that are "evidence based." This concept is certainly laudable and is definitely not new. We live in an age of rapidly changing technologies, with vast amounts of information being offered via the scientific literature, the lay press, infomercials, the Internet, and various other sources. New therapies, techniques, and technologies are touted, and patients are enticed by the purported benefits. The payors and government are focusing on outcomes, quality, and the ability of treatments to add value. These efforts are truly something that should be embraced and nurtured by all of us, particularly since the cost and complexity of care continues to rise, and since therapies oftentimes carry a significant risk of morbidity and mortality. Indeed, the basis for the randomized controlled trial is to compare alternate modalities, ideally in a double-blind fashion, and thereby to determine the safety and efficacy of the particular strategy. We would thus attempt to determine whether A is "better" than B, and whether C is an effective alternative.We assume that the scientific trials, the interpretation of the data, and its ultimate publication are factual and without bias. However, it is increasingly clear that our assumption is not always correct. This writer has brought these issues to this column before. 1-4 The credibility issue and the issues of editorial and scientific bias merit our careful consideration and attention as we ourselves produce, read, and interpret the literature or offer various treatments to our patients. This task is further compounded by the fact that there may well be several seemingly beneficial treatment strategies for a particular problem. Still worse is the conflicting nature of the literature itself. Studies that report positive results are not infrequently trumped by studies that demonstrate the diametric opposite. Ioannidis 5 found that only 44% of highly cited original clinical research studies were replicated, while 16% were contradicted and an additional 16% had found effects that were stronger than those of subsequent studies. His further analysis demonstrated that 5 of 6 (83%) of highly cited nonrandomized studies were contradicted or had found stronger effects compared to 9 of 39 (23%) of large randomized controlled trials. 5 Ioannidis concluded that contradiction and initially stronger effects are not unusual in highly cited research of clinical interventions and their outcomes. 5 We are driven to consider centralized systematic reviews conducted and published by reputable agencies and groups as being our resource for complete, accurate, transparent and unbiased analysis of data. These groups cull the world literature for Type 1 studies and other high quality data. They then construct their recommendations based on the results of their defined criteria. Th...

Phototherapy promotes healing of chronic diabetic leg ulcers that failed to respond to other therapies

“…Even though others have offered detailed comments, analysis and meta-analysis of the literature [74][75][76][77][78][79][80][81][82], the strong positive findings of our study warrant further discussion of the clinical value of phototherapy, given the widespread perception that the literature is replete with contradictory data. For example, treatment of nine patients with 12 ''minor postsurgical wounds'' using 300 mW cm À2 irradiance and 9 J cm À2 fluence of 830 nm light was reported to be ineffective in promoting wound closure or reducing the pain caused by the incisions [83].…”

Phototherapy Improves Healing of Chronic Venous Ulcers

“…Therefore, a brief comment seems warranted considering this aspect of our results, even though others have offered detailed comments and analysis of the literature in the past. [54][55][56][57] Lagan et al 58 used 830-nm light to treat nine patients with 12 "minor postsurgical wounds" using 300 mW cm Ϫ2 irradiance and 9 J/cm Ϫ2 fluence; they found no difference in wound closure and no difference in pain levels between patients in the treatment group and controls. Our finding that small-sized ulcers or minor wounds tend to heal properly regardless of phototherapy support their results.…”

Editorial: Working with the Evidence

“…The issues regarding conflicts and bias in randomized trials and the evidence based literature have been frequent topics considered in this column. [2][3][4][5][6][7][8] We have examined bias in study development, conduct, and reporting, and have echoed the concerns of Bjordal and others who have demonstrated that systematic reviews themselves are biased in their analysis of the "evidence base" and the subsequent recommendations promulgated as a result thereof. 5 The topics continue to receive notoriety as authors have considered "gizmo idolatry," 9 the influence of "industry influence," 10 the "quality" of RCTs of efficacy research in complementary and alternative medicine, 11 and the assessment of physician and patient preferences on research.…”