Decomposing clinical practice guidelines panels' deliberation into decision theoretical constructs

Abstract

Rationale, Aims and Objectives The development of clinical practice guidelines (CPG) suffers from the lack of an explicit and transparent framework for synthesising the key elements necessary to formulate practice recommendations. We matched deliberations of the American Society of Haematology (ASH) CPG panel for the management of pulmonary embolism (PE) with the corresponding decision‐theoretical constructs to assess agreement of the panel recommendations with explicit decision modelling. Methods Five constructs were identified of which three were used to reformulate the panel's recommendations: (1) standard, expected utility threshold (EUT) decision model; (2) acceptable regret threshold model (ARg) to determine the frequency of tolerable false negative (FN) or false positive (FP) recommendations, and (3) fast‐and‐frugal tree (FFT) decision trees to formulate the entire strategy for management of PE. We compared four management strategies: withhold testing versus d‐dimer → computerized pulmonary angiography (CTPA) (‘ASH‐Low’) versus CTPA→ d‐dimer (‘ASH‐High’) versus treat without testing. Results Different models generated different recommendations. For example, according to EUT, testing should be withheld for prior probability PE < 0.13%, a clinically untenable threshold which is up to 15 times (2/0.13) below the ASH guidelines threshold of ruling out PE (at post probability of PE ≤ 2%). Three models only agreed that the ‘ASH low’ strategy should be used for the range of pretest probabilities of PE between 0.13% and 13.27% and that the ‘ASH high’ management should be employed in a narrow range of the prior PE probabilities between 90.85% and 93.07%. For all other prior probabilities of PE, choosing one model did not ensure coherence with other models. Conclusions CPG panels rely on various decision‐theoretical strategies to develop its recommendations. Decomposing CPG panels' deliberation can provide insights if the panels' deliberation retains a necessary coherence in developing guidelines. CPG recommendations often do not agree with the EUT decision analysis, widely used in medical decision‐making modelling.

Improving methods of clinical practice guidelines: From guidelines to pathways to fast‐and‐frugal trees and decision analysis to develop individualised patient care

“…13 By proposing FFT heuristics as a theoretical framework for constructing clinical pathways, we provided a theoretical structure that allows quantitative analysis of the accuracy of clinical management strategies. 12,14 FFT draws its theoretical robustness by relating to signal detection theory, evidence accumulation theory, and the threshold model to help improve decision-making. 12,14,15 By classifying disease or prediction, FFT can also be considered as a predictive model.…”

“…12,14 FFT draws its theoretical robustness by relating to signal detection theory, evidence accumulation theory, and the threshold model to help improve decision-making. 12,14,15 By classifying disease or prediction, FFT can also be considered as a predictive model. 16 This allows individualised prediction of outcomes and management according to a series of responses along the FFT decision tree, taking the heterogeneity of patients' risk into account.…”

“…16 This allows individualised prediction of outcomes and management according to a series of responses along the FFT decision tree, taking the heterogeneity of patients' risk into account. 12,14,16 Predictive models can individualise the prediction of outcomes, but because they do not take the consequences of the treatment into account, they cannot assess the clinical utility of the competing treatment alternatives. 17 A variant of FFT, FFT that includes a decision threshold (FFTT).…”

“…10,11,[53][54][55][56] The latter constraints are particularly evident in busy, attention and time-limited clinical practice, making FFT an ideal vehicle for implementing guidelines. 14 By treating FFT as a predictive model, we respond to one of the oldest critiques of EBM guidelines-the recommendations refer to "average" rather than individual patients. 36 Our analytical framework successfully addresses this criticism by linking FFT/FFTT within gDCA framework, allowing us to (a) compare effects among multiple treatments and (b) tailor treatment strategies toward predicted risks of clinical events such as HIT and benefits and harms of treatment.…”

“…FFTs often yield more accurate diagnostic and prognostic classification than complex machine learning or multivariate regression models 13 . By proposing FFT heuristics as a theoretical framework for constructing clinical pathways, we provided a theoretical structure that allows quantitative analysis of the accuracy of clinical management strategies 12,14 . FFT draws its theoretical robustness by relating to signal detection theory, evidence accumulation theory, and the threshold model to help improve decision‐making 12,14,15 …”