Potential problems in the use of patient reported outcome measures (PROMs) and reporting of PROM data in sports science

Krogsgaard, Michael; Brodersen, John; Jensen, Jonas; Hansen, Christian Fugl; Comins, Jonathan

doi:10.1111/sms.13888

Cited by 11 publications

(18 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Generally, there is little focus on whether the PROM that is used in a study is adequate or not, 8 despite the impact of results obtained by use of PROMs. An adequate PROM is a measurement tool that, first of all, actually measures what it claims to measure.…”

Section: Introductionmentioning

confidence: 99%

Are adequate PROMs used as outcomes in randomized controlled trials? an analysis of 54 trials

Hansen

Jensen

Brodersen

et al. 2021

Scandinavian Med Sci Sports

Self Cite

View full text Add to dashboard Cite

Results by patient‐reported outcome measures (PROMs) from randomized controlled trials (RCTs) in musculoskeletal research often influence healthcare strategies. We aimed to evaluate to which extent these RCTs use adequate PROMs, and how this influences the results and conclusions. We identified RCTs of sports research relevance with PROMs as primary outcomes published in 13 preselected journals between January 1, 2008, and November 1, 2019; all journals regularly publish results from musculoskeletal research. Five journals have a high impact factor (>15), and eight with lower impact factors are widely read journals. It was assessed whether the RCTs had used PROMs with high content validity and whether the most adequate PROMs were used (ie, the most well developed and well validated for the patients enrolled in the study). We registered journal impact factor, year of publication, existence of a registered protocol, and whether the study showed significant difference between interventions. A total of 54 RCTs with 56 primary outcomes comprising 26 different PROMs were identified. For 13 RCTs (24%), a protocol was not published. In only 24 of RCTs (44%), the most appropriate PROM had been used as primary outcome, independent of a registered protocol, ranking of the journal, and year of publication. In seven cases, PROMs were used to evaluate a condition that they had not been developed for. RCTs that used the most adequate PROM showed significantly more often (46%) difference in outcomes in contrast to RCTs that used inadequate PROMs (22%) (P = 0.0483). In conclusion, in the majority of RCTs, the most adequate PROM had not been used. Studies, in which the most adequate PROM had been used as outcome, were significantly more likely to show significant difference between interventions. The extent to which protocols were not available was surprisingly high. Journals should request that adequate PROMs are used in RCTs, and if this is not the case that it is discussed how it might influence the results and conclusions. Likewise, it should be requested that a protocol is published or registered.

show abstract

Section: Introductionmentioning

confidence: 99%

Are adequate PROMs used as outcomes in randomized controlled trials? an analysis of 54 trials

Hansen

Jensen

Brodersen

et al. 2021

Scandinavian Med Sci Sports

Self Cite

View full text Add to dashboard Cite

show abstract

“…In an article in this series, to be published in June, 2 14 potential problems in the use of PROMs were identified by a panel (three full professors, two associate research professors, one postdoctoral research fellow, and two research assistants/medical students). Based on the general principles for PROMs, for instance described by US Food and Drug Administration (FDA) 3 and the COnsensus‐based Standards for the selection of health status Measurement Instruments (COSMIN) group, 4 the consensus group deliberated on what types of potential problems they had encountered in health science papers they had read or analyzed in the course of their research careers or had contributed to personally.…”

Section: Methodsmentioning

confidence: 99%

“…Five of the potential problems described in this article of the current series on PROMs 2 were not used in the analyses for the present article for the following reasons:…”

Section: Methodsmentioning

confidence: 99%

“…While several types of such potential problems have been reported, 2 it was hypothesized for the current study that in the majority of musculoskeletal studies, PROMs are used adequately, meaning that data are reported in a way that minimizes bias. Furthermore, to the extent that such problems occur, we hypothesized that PROMs are used more adequately in (a) studies published in journals with high ranking, (b) studies where there exists a published study protocol, and (c) more recent studies (as general knowledge of PROMs is expected to have increased).…”

Section: Introductionmentioning

confidence: 99%

“…The validity of results obtained from studies using PROMs as outcomes may be undermined if the PROMs are not used adequately and PROM data are not reported in a way that minimizes bias. In article six of this series, 2 we discussed various issues in relation to the use of PROMs, the reporting of PROM data, and the problems this may create.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Are PROMs used adequately in sports research? An analysis of 54 randomized controlled trials with PROMs as endpoint

Jensen

Hansen

Brodersen

et al. 2020

Scandinavian Med Sci Sports

Self Cite

View full text Add to dashboard Cite

Deviations from adequate use and reporting of PROMs may be problematic and misleading. The aim of this study was to investigate the extent of such problems in randomized clinical trials (RCTs). RCTs involving sports medicine research that used PROMs as primary outcomes were identified in 13 preselected journals. The articles were reviewed for nine potential problems related to how the PROM was used and how the data had been reported. The potential problems were as follows: aggregating subscale scores; combining patient‐reported scores with physical, clinical, or para‐clinical measures; using a PROM to diagnose or evaluate the individual patient; using a PROM for one leg or arm; selectively excluding domains or items; constructing a PROM for the specific occasion; mixing PROM formats (ie, digital, paper, telephone, e‐mail, in person); ambiguous instructions for how the PROM should be completed; and recall bias. As covariates, we registered journal impact factor, year of publication, and existence of a registered protocol. In 29 (53.7%) of 54 identified RCTs, at least one potential problem was identified, the most common being aggregation of domain scores. This was not different with a published protocol or dependent on journal rankings, except for exclusion of domains, which was most common in high‐ranking journals. Aggregation of domain scores was significantly less common in recently published articles compared with older articles (P = .03). Potential problematic use of PROMs and reporting of PROM data are common in RCTs, also in high‐ranking journals, but less so in more recent articles.

show abstract

High recall bias in retrospective assessment of the pediatric International Knee Documentation Committee Questionnaire (Pedi-IKDC) in children with knee pathologies

Macchiarola

Pirone

Grassi

et al. 2022

Knee Surg Sports Traumatol Arthrosc

View full text Add to dashboard Cite

Purpose The increasing incidence of knee injuries among children is well known by sports physicians. Papers dealing with this topic have often collected patient-reported outcomes measures (PROMs) in a retrospective manner; this limitation could lead to a misinterpretation of the results, because pediatric patients might not remember their preoperative conditions adequately. This study aims to evaluate the reliability and the reproducibility of the IKDC pediatric score when administered retrospectively at a 12-month follow-up. Methods From September 2018 and June 2019, all patients aged 7-18 scheduled for surgery due to diferent knee pathologies in a single center were considered eligible. Parents were contacted by phone for consent. An open-source platform was implemented to collect the responses: two surveys were created (Q1, Q2). They included general information and the Pedi-IKDC score. Q1 was completed prospectively, while Q2 was completed 12 months after surgery. The two questionnaires were identical, and patients were carefully advised to complete Q2 recalling their health status before surgery. ICC and the concordance correlation coeicient (ρ c ) were used to assess the reproducibility between the prospective and recalled scores. Results Sixty-six patients responded to Q1 and Q2, and the mean age was 12.9 ± 2.2 years at Q1 and 14.1 ± 2.2 years at Q2. The mean time between Q1 and Q2 was 14.1 ± 2.1 months. Between prospective-IKDC and recall-IKDC, the ICC coeicient was "poor" at 0.32 (CI 0.09 to 0.5) and the ρ c was "poor" at 0.4 (CI 0.29 to 0.51). Mean prospective-IKDC was 76.8 ± 23.52 mean recalled-IKDC was 60.4 ± 11.5 (P < 0.0001), while mean diference was -16.3 ± 2.09. Simple linear regression models showed that Δ-IKDC is independently associated with age at Q1 (R 2 = 0.2676; P0.0001) and prospective-IKDC (R 2 = 0.653; P < 0.0001). Conclusions Retrospective collection of the Pedi-IKDC score is not reliable and has high recall bias. This should be avoided in children with knee conditions. Level of evidence III.

show abstract

Potential problems in the use of patient reported outcome measures (PROMs) and reporting of PROM data in sports science

Cited by 11 publications

References 31 publications

Are adequate PROMs used as outcomes in randomized controlled trials? an analysis of 54 trials

Are adequate PROMs used as outcomes in randomized controlled trials? an analysis of 54 trials

Are PROMs used adequately in sports research? An analysis of 54 randomized controlled trials with PROMs as endpoint

High recall bias in retrospective assessment of the pediatric International Knee Documentation Committee Questionnaire (Pedi-IKDC) in children with knee pathologies

Contact Info

Product

Resources

About