Do the American Society of Clinical Oncology Value Framework and the European Society of Medical Oncology Magnitude of Clinical Benefit Scale Measure the Same Construct of Clinical Benefit?

Abstract

Purpose Whether the ASCO Value Framework and the European Society for Medical Oncology (ESMO) Magnitude of Clinical Benefit Scale (MCBS) measure similar constructs of clinical benefit is unclear. It is also unclear how they relate to quality-adjusted life-years (QALYs) and funding recommendations in the United Kingdom and Canada. Methods Randomized clinical trials of oncology drug approvals by the US Food and Drug Administration, European Medicines Agency, and Health Canada between 2006 and August 2015 were identified and scored using the ASCO version 1 (v1) framework, ASCO version 2 (v2) framework, and ESMO-MCBS by at least two independent reviewers. Spearman correlation coefficients were calculated to assess construct (between frameworks) and criterion validity (against QALYs from the National Institute for Health and Care Excellence [NICE] and the pan-Canadian Oncology Drug Review [pCODR]). Associations between scores and NICE/pCODR recommendations were examined. Inter-rater reliability was assessed using intraclass correlation coefficients. Results From 109 included randomized clinical trials, 108 ASCOv1, 111 ASCOv2, and 83 ESMO scores were determined. Correlation coefficients for ASCOv1 versus ESMO, ASCOv2 versus ESMO, and ASCOv1 versus ASCOv2 were 0.36 (95% CI, 0.15 to 0.54), 0.17 (95% CI, -0.06 to 0.37), and 0.50 (95% CI, 0.35 to 0.63), respectively. Compared with NICE QALYs, correlation coefficients were 0.45 (ASCOv1), 0.53 (ASCOv2), and 0.46 (ESMO); with pCODR QALYs, coefficients were 0.19 (ASCOv1), 0.20 (ASCOv2), and 0.36 (ESMO). None of the frameworks were significantly associated with NICE/pCODR recommendations. Inter-rater reliability was good for all frameworks. Conclusion The weak-to-moderate correlations of the ASCO frameworks with the ESMO-MCBS, as well as their correlations with QALYs and with NICE/pCODR funding recommendations, suggest different constructs of clinical benefit measured. Construct convergent validity with the ESMO-MCBS did not increase with the updated ASCO framework.

How Can Experiments Play a Greater Role in Public Policy? 12 Proposals from an Economic Model of Scaling

“…Maybe the original study was a false positive, whereby the received evidence was not yet actionable? Perhaps there was a "voltage effect" (see, e.g., Kilbourne et al, 2007;Supplee and Meyer, 2015;Supplee and Metz, 2015;Gottfredson et al, 2015;Cheng et al, 2017) because Greta's school district changed core components of the program or failed to hire high quality instructors. The term "voltage effect" or "voltage drop" describes the phenomenon of the measured program benefit in the original research study being significantly larger than the measured benefit when that program is implemented at larger scale.…”

Value assessment of oncology drugs using a weighted criterion‐based approach

“…Our clinical benefit scores only weakly correlated with ESMO‐MCBS scores. Although some data suggested weak to moderate correlation between ASCO and ESMO framework scores, many are increasingly recognizing that value depends on the perspective of the assessor and the unique clinical context being considered . Consequently, multiple perspectives of value are important in decisions regarding funding and sustainability.…”

“…Although some data suggested weak to moderate correlation between ASCO and ESMO framework scores, many are increasingly recognizing that value depends on the perspective of the assessor and the unique clinical context being considered. [29][30][31][32] Consequently, multiple perspectives of value are important in decisions regarding funding and sustainability. The ESMO-MCBS scale also places less importance on RR and PFS for treatments that demonstrate an improvement in OS.…”

The “American (cancer) patients first” plan to reduce drug prices—A critical assessment

“…Consequently, the ASCO document has no tangible effect. Schnipper and colleagues updated the ASCO value framework, which was evaluated in three follow‐up studies that could not substantiate the benefit of this value framework . The other societies (ASH, ACS, LLS) had vague (LLS) or no positions on the issue (ACS, ASH).…”