Getting it wrong most of the time? Comparing trialists’ choice of primary outcome with what patients and health professionals want

Abstract

Background Randomised trials support improved decision-making through the data they collect. One important piece of data is the primary outcome — so called because it is what the investigators decide is the most important. Secondary outcomes provide additional information to support decision-making. We were interested in knowing how important patients and healthcare professionals consider the outcomes (especially the primary outcome) measured in a selection of published trials. Methods The work had three stages: (1) We identified a body of late-stage trials in two clinical areas, breast cancer management and nephrology. (2) We identified the primary and secondary outcomes for these trials. (3) We randomly ordered these outcomes and presented them to patients and healthcare professionals (with experience of the clinical area), and we asked them to rank the importance of the outcomes. They were not told which outcomes trial authors considered primary and secondary. Results In our sample of 44 trials with 46 primary outcomes, 29 patients, one patient representative and 12 healthcare professionals together ranked the primary outcome as the most important outcome 13/46 times or 28%. Breast cancer patients and healthcare professionals considered the primary outcome to be the most important outcome for 8/21 primary outcomes chosen by trialists. For nephrology, the equivalent figure was 5/25. The primary outcome appeared in a respondent’s top 5 ranked outcomes 151/178 (85%) times for breast cancer and 225/259 (87%) times for nephrology even if the primary was not considered the most important outcome. Conclusions The primary outcome in a trial is the most important piece of data collected. It is used to determine how many participants are required, and it is the main piece of information used to judge whether the intervention is effective or not. In our study, patients and healthcare professionals agreed with the choice of the primary outcome made by trial teams doing late-stage trials in breast cancer management and nephrology 28% of the time.

Patient and Public Involvement (PPI) in outcome selection in breast cancer and nephrology trials

“…Our original study found that patients and healthcare professionals disagreed with the trial team’s choice of primary outcome 72% of the time [ 1 ]. It does not matter how robust the methodology of a trial is or how experienced the research team is, if the outcome that is most important to people living with a disease (and those treating it) is not measured, or not measured to a sufficient degree of certainty, then it is quite possible that the results will be considered irrelevant.…”

“…The methods used to randomly select the original 44 trials have been described previously [ 1 ]. For the current study, the protocol for each of the 44 trials was sought.…”

“…In a recently published study, we investigated how important patients ( n = 30) and healthcare professionals ( n = 12), with experience in the clinical areas of breast cancer and nephrology, consider the outcomes (particularly the primary outcome) measured in a random selection of 20 breast cancer and 24 nephrology published randomised controlled trials [ 1 ]. Primary outcomes are deemed the most important outcomes by trial investigators, and the primary outcome is used to determine the sample size for the trial [ 2 ] as well as being the main way to judge whether the intervention is effective or not.…”

“…Primary outcomes are deemed the most important outcomes by trial investigators, and the primary outcome is used to determine the sample size for the trial [ 2 ] as well as being the main way to judge whether the intervention is effective or not. Secondary outcomes aid with decision-making by providing additional information but are considered less important [ 1 ].…”

“…We found that according to the patients and healthcare professionals, teams conducting breast cancer and nephrology trials got their choice of primary outcome wrong (72% of the time) more often than they got it right (28% of the time) [ 1 ]. A Patient and Public Involvement (PPI) representative, co-author of this letter (LL), asked (on Twitter) whether PPI contributors had been involved in the design of the original trials and by extension the outcome selection.…”