Clinical Usefulness of the Framingham Cardiovascular Risk Profile Beyond Its Statistical Performance: The Tehran Lipid and Glucose Study

Abstract

The utility of a risk function in clinical practice is an important concept that has received insufficient attention. The authors evaluated the clinical usefulness of the Framingham risk function (FRF) for cardiovascular disease in a Middle Eastern population (2,640 men and 3,584 women aged 30-74 years) free of cardiovascular disease at baseline in 1999. They calculated the net benefit fraction for treatment of subjects with an estimated 10-year risk of ≥10% and also ≥20%, where the net benefit fraction is a weighted sum of true-positive and false-positive rates divided by incidence, as estimated by Kaplan-Meier analysis. The authors drew a decision curve by plotting the net benefit fraction against a wide range of risk thresholds for treatment. The cumulative incidence of cardiovascular disease was 7.6% and 12.3% in women and men, respectively. The FRF had a C index of 0.832 in women and 0.785 in men with a reasonable calibration. On the basis of the net benefit fraction, about 50% of the incidence in men and women could be appropriately treated by using the 10% threshold; however, the FRF was not useful at the 20% threshold, especially in women. In both genders, usefulness of the FRF was as good as the function derived directly from Tehrani data with the same variables; however, it could be useful in low thresholds for treatment.

A new approach to test validity and clinical usefulness of the 2013 ACC/AHA guideline on statin therapy: A population-based study

“…We also calculated Net Benefit Fraction (NBF) as an index for clinical usefulness of statin therapy [9]. Net benefit is defined as (TPs − w × FPs) / N, when TP is true-positive decision, FP is false-positive decision, N is the total number of study population, and w is the harm-to-benefit ratio of the treatment which is equal to the odds of the risk threshold for treatment (Pt / (1 − Pt)) [17,18]; in this way, at the threshold of 5%, the FP is valued as 5.3% of the TP and at the threshold of 7.5% this value would be 8.1%.…”

“…Net benefit is defined as (TPs − w × FPs) / N, when TP is true-positive decision, FP is false-positive decision, N is the total number of study population, and w is the harm-to-benefit ratio of the treatment which is equal to the odds of the risk threshold for treatment (Pt / (1 − Pt)) [17,18]; in this way, at the threshold of 5%, the FP is valued as 5.3% of the TP and at the threshold of 7.5% this value would be 8.1%. Net Benefit Fraction is net benefit divided by the incidence rate of the outcome; indeed, NBF is true-positive rate (sensitivity) penalized by false-positive rate and means the net fraction of the incidence that could be predicted and potentially prevented [9]. We computed NBF considering the risk threshold of 5% and 7.5%, for initiation of moderate-and high-intensity statin therapies, respectively.…”

“…Moreover, assessing the clinical usefulness of the guideline considering the harm, for false-positive, and benefit, for true-positive predictions is more important, which has not got enough attention. We have previously determined the validity and clinical usefulness of the Framingham risk equation in the prediction of CVDs and starting statin therapy [9]; however, in the present study we started with assessing the validity of the new risk prediction equation, followed by determining the validity and clinical usefulness of the guideline [10], and ended with estimating the net number needed to treat (NNT) for statin therapy based on the guideline.…”

Re: "Clinical Usefulness of the Framingham Cardiovascular Risk Profile beyond Its Statistical Performance: The Tehran Lipid and Glucose Study"

“…The first is to stratify any analysis by important patient characteristics that affect risk threshold-for example, gender as in the study by Khalili et al (1). The second is to report the net benefit fraction together with its constituents, the TPR and FPR, over a range of risk thresholds.…”

“…Khalili et al (1) evaluated the clinical utility of the Framingham cardiovascular risk model in a Middle Eastern cohort study. We commend the authors for their use of highly relevant measures of model performance, the net benefit (2, 3) and net benefit fraction (4,5).…”