Validity of original studies in health technology assessment reports: Significance of standardized assessment and reporting

Ackermann

Amelung

2007

BMC Health Serv Res

Self Cite

115

Background: Interest in assessing the value of health-care services in Germany has considerably increased since the foundation of the Institut für Qualität und Wirtschaftlichkeit im Gesundheitswesen, IQWiG (Institute for Quality and Efficiency in Health Care). The practical application of value assessment illustrates how problematic the process can be. In all decisions made for the provision of health care, data concerning the measurable dimensions (quantity and quality of efficacy and effectiveness, validity of the results and costs) flow into a complex and not yet standardized decision-making process concerning public financing. Some of these decisions are based on data of uncertain validity, unknown reproducibility and unclear appropriateness.

Section: Appendixmentioning

confidence: 99%

The value of health care – a matter of discussion in Germany

Ackermann

Amelung

2007

BMC Health Serv Res

Self Cite

115

“…Despite measuring different things, both instruments recognized the serious weakness of the appraised key publications. Second, despite the weakness mentioned in the core parts of the HTA reports, which are read by scientists, the authors of both reports recommended the use of the procedure in the summaries of their HTA reports, which are read by policy makers (Porzsolt 2005). This asymmetric distribution of information generates problems.…”

Section: Introductionmentioning

confidence: 99%

Section Original article

Costa²,

Thomaz³

2009

J Public Health

Self Cite

Background Considerable time and energy are expended in the scientific community to discuss the validity, importance, and applicability of the results of clinical trials. Depending on the goals, perspectives, and other motivating factors, protagonists and skeptics come to different conclusions, even when using the same methods and tools for critical appraisal. The aim of this study was to complement existing methods and tools with minor modifications to provide a prototype instrument that generates commonly accepted versions of critical appraisals. Methods As a pilot experiment, one university-based and one industry-based referee independently completed the twin assessment of five trials published in well-recognized journals. They identified the study questions, defined the simplest, i.e., ideal, study designs to answer these questions, and checked eight validity criteria. Identical positive or negative answers of both referees increased or decreased the validity score. A maximum of two disagreements (0 score) was allowed. This procedure, which had been tested by two referees in a pilot experiment, was repeated with 19 third-year medical students and their supervisor at the Universidade Federal Fluminense, Niterói/RJ, Brasil. Four students each played the roles of the industry-based and university-based referees and finally recorded their consensus. Results The two referees of the pilot experiment agreed in all but one answer to the five investigated publications. The points of criticism differed in various papers. The consensus reached by the students considerably differed from the consensus reached by the referees. Conclusions A consensus score generated by two referees or by two groups of students is feasible, but the achieved result is not necessarily reproducible. The critical appraisal of the study question in connection with the applied study design deserves special attention. It is time consuming but possible to identify and describe the possible flaws in the design conduct and report of clinical trials, but it is unlikely to reach a reproducible interpretation. These data indicate the problems with even evidence-based assessments and appraisals: the assessments may well be reproducible, but not the appraisals. Quality scores that include also the appraisal may therefore be interpreted with caution. Appraisals or quality scores may be used for interim decisions until data are available that confirm under real-world conditions what was predicted by the results generated under ideal but artificial conditions of a clinical trial.

“…Wir haben mit unserer Erweiterung des klassischen Fünf-Schritt-Modells von Sackett et al auf ein Sechs-Schritt-Mobut to convince critical members of society with additional data which have been obtained under everyday conditions. andere Aussagen zu finden sind als innerhalb desselben HTA Report, der von Wissenschaftlern gelesen und verwendet wird[25]. Diese Art vermeidbarer Fehler belastet das Vertrauen in evidenzbasierte Aussagen.…”

unclassified

Folgt auf die „Evidence-based Medicine“ eine „Confidence-based Medicine“?

Fangerau

2010

Med Klin

Self Cite

In an appeal concerning accusations of defamation, the England and Wales Court of Appeal Decisions determined that evidence-based statements are to be judged as opinions and not statements of fact. Since the authors consider it probable that this legal judgment will exert influence on physicians' decisions about the provision of health care services, they have compiled the implications of the judgment and discuss its consequences. The own analyses and considerations lead to the conclusion that confidence-based medicine follows evidence-based medicine. This extension is necessary because evidence-based medicine has not been able to generate the required trust. Therefore, it will be demanded to underpin the existing concept with additional data. These data will be necessary because it is no longer sufficient to convince scientists with data which are obtained under ideal conditions, but to convince critical members of society with additional data which have been obtained under everyday conditions.