Clinical judgment and statistics. Lessons from a simulated randomized trial in coronary artery disease.

Abstract

A simulated randomized clinical trial in coronary artery disease was conducted to illustrate the need for clinical judgment and modern statistical methods in assessing therapeutic claims in studies of complex diseases. Clinicians should be aware of problems that occur when a patient sample is subdivided and treatment effects are assessed within multiple prognostic categories. In this example, 1073 consecutive, medically treated coronary artery disease patients from the Duke University data bank were randomized into two groups. The groups were reasonably comparable and, as expected, there was no overall difference in survival. In a subgroup of 397 patients characterized by three-vessel disease and an abnormal left ventricular contraction, however, survival of group 1 patients was significantly different from that of group 2 patients. Multivariable adjustment procedures revealed that the difference resulted from the combined effect of small imbalances in the distribution of several prognostic factors. Another subgroup was identified in which a significant survival difference was not explained by multivariable methods. These are not unlikely examples in trials of a complex disease. Clinicians must exercise careful judgment in attributing such results to an efficacious therapy, as they may be due to chance or to inadequate baseline comparability of the groups.

Beyond randomized clinical trials: applying clinical experience in the treatment of patients with coronary artery disease.

“…Several examples now exist documenting the ability of observational studies to achieve the same results as randomized controlled trials in well-characterized diseases. 23 24 Nevertheless, in each new situation the completeness with which important baseline differences have been characterized must be questioned. No single observational treatment comparison can be regarded as definitive.…”

On Problems Related to Subgroup Analyses in Clinical Trials

“…In this paper, we want to illustrate the probable consequences of an extensive use of multiple significance tests in several subgroups of a study population by means of a 'hypothetical clinical trial' in breast cancer patients treated in a uniform way. A similar approach had been formerly presented by Lee [1] in patients with coronary heart disease.…”

Subgroup Interactions in the Heart and Estrogen/Progestin Replacement Study

“…8 We conducted a total of 172 subgroup analyses to determine the effect of treatment in 86 subgroups after 1 year and after the entire follow-up. Under the null hypothesis that the treatment effect does not differ by subgroup, we would expect Ϸ4 (5%) of 86 interactions to be nominally significant at PϽ0.05 by chance alone, both for the 1-year and overall follow-up periods.…”