Clinical Practice Guidelines and Consensus Statements in Oncology – An Assessment of Their Methodological Quality

Abstract

BackgroundConsensus statements and clinical practice guidelines are widely available for enhancing the care of cancer patients. Despite subtle differences in their definition and purpose, these terms are often used interchangeably. We systematically assessed the methodological quality of consensus statements and clinical practice guidelines published in three commonly read, geographically diverse, cancer-specific journals. Methods Consensus statements and clinical practice guidelines published between January 2005 and September 2013 in Current Oncology, European Journal of Cancer and Journal of Clinical Oncology were evaluated. Each publication was assessed using the Appraisal of Guidelines for Research and Evaluation II (AGREE II) rigour of development and editorial independence domains. For assessment of transparency of document development, 7 additional items were taken from the Institute of Medicine’s standards for practice guidelines and the Journal of Clinical Oncology guidelines for authors of guidance documents.MethodsConsensus statements and clinical practice guidelines published between January 2005 and September 2013 in Current Oncology, European Journal of Cancer and Journal of Clinical Oncology were evaluated. Each publication was assessed using the Appraisal of Guidelines for Research and Evaluation II (AGREE II) rigour of development and editorial independence domains. For assessment of transparency of document development, 7 additional items were taken from the Institute of Medicine's standards for practice guidelines and the Journal of Clinical Oncology guidelines for authors of guidance documents.FindingsThirty-four consensus statements and 67 clinical practice guidelines were evaluated. The rigour of development score for consensus statements over the three journals was 32% lower than that of clinical practice guidelines. The editorial independence score was 15% lower for consensus statements than clinical practice guidelines. One journal scored consistently lower than the others over both domains. No journals adhered to all the items related to the transparency of document development. One journal’s consensus statements endorsed a product made by the sponsoring pharmaceutical company in 64% of cases.ConclusionGuidance documents are an essential part of oncology care and should be subjected to a rigorous and validated development process. Consensus statements had lower methodological quality than clinical practice guidelines using AGREE II. At a minimum, journals should ensure that that all consensus statements and clinical practice guidelines adhere to AGREE II criteria. Journals should consider explicitly requiring guidelines to declare pharmaceutical company sponsorship and to identify the sponsor’s product to enhance transparency.

Clinical Practice Guidelines for Rare Diseases: The Orphanet Database

“…Based on the AGREE II criteria, some insufficiencies were frequently noticed, especially regarding rigour of development and editorial independence. This observation has been described by others for some RDs [31] but is not specific to this field, as also reported for more common diseases [2, 32, 33]. Variable quality was also noticed depending on the publication medium.…”

Is there expert consensus on expert consensus?

“…Rigor of development score for consensus statements over three cancer journals (Current Oncology, European Journal of Cancer and Journal of Clinical Oncology) was one-third lower than that of evidence-based recommendations. Editorial independence score was also 15% lower for consensus statements [7]. Strategies to deal with the problem inherent to consensus methodology rely on formalized processes in which all panelists contribute equally.…”

What is the role of consensus statements in a risk society?

“…The effectiveness of consensus as policy tools is highly contingent on their quality. Consensus statements are frequently criticised in the literature with respect to the process of their development, the nature of the consensus they achieve and their impact on public and policy perceptions of the role of science (Lenzer 2013;Jacobs et al 2014;Mahawar et al 2015). First, expertise is often linked with ideological and/or financial interests (Lenzer 2013).…”

“…Appraisal tools exist (e.g. the AGREE and AGREE II tools (Burgers et al 2004;Fervers et al 2005;de Haas et al 2007;Brouwers et al 2010;Shimbo et al 2010;Langton et al 2011;Hogeveen et al 2012;Jacobs et al 2014)), but they typically lack guidance around the processes of stakeholder participation, deliberation, and decision-making, focusing more on the methodology of reviewing evidence and the nature of recommendations. In a review of ten consensus statements, Lenzer (2013) raised concerns about panel member independence in eight.…”