The use and quality of reporting of propensity score methods in multiple sclerosis literature: A review

Abstract

Background: Propensity score (PS) analyses are increasingly used in multiple sclerosis (MS) research, largely owing to the greater availability of large observational cohorts and registry databases. Objective: To evaluate the use and quality of reporting of PS methods in the recent MS literature. Methods: We searched the PubMed database for articles published between January 2013 and July 2019. We restricted the search to comparative effectiveness studies of two disease-modifying therapies. Results: Thirty-nine studies were included in the review, with most studies (62%) published within the past 3 years. All studies reported the list of covariates used for the PS model, but only 21% of studies mentioned how those covariates were selected. Most studies used PS matching (72%), followed by PS adjustment (18%), weighting (15%), and stratification (3%), with some overlap. Most studies using matching or weighting reported checking post-PS covariate imbalance (91%), although about 45% of these studies relied on p values from various statistical tests. Only 25% of studies using matching reported calculating robust standard errors for the PS analyses. Conclusions: The quality of reporting of PS methods in the MS literature is sub-optimal in general, and in some cases, inappropriate methods are used.

Early High Efficacy Treatment in Multiple Sclerosis Is the Best Predictor of Future Disease Activity Over 1 and 2 Years in a Norwegian Population-Based Registry

“…This score has not yet been validated, and we would like to validate it in a new population. We could have used propensity score analyses to control for confounding, but propensity score matching does not yield different estimates compared to conventional multivariate methods (54) and is often used inappropriately in MS research (55).…”

Recommendations for the use of propensity score methods in multiple sclerosis research

“…14 In MS, a recent review found that an average of eight covariates (range: 3-16) were selected to construct the PS, with the following baseline covariates most frequently used: age, sex, disease duration, number of relapses in the 12 months prior to baseline, Expanded Disability Status Scale (EDSS) score at baseline, and previous treatments. 5 However, only 21% of the reviewed studies reported how the list of covariates was determined. 5 A practical issue with covariate selection for the PS is that relevant confounders are sometimes unavailable in the data source, which violates the condition of no unmeasured confounding.…”

“…5 However, only 21% of the reviewed studies reported how the list of covariates was determined. 5 A practical issue with covariate selection for the PS is that relevant confounders are sometimes unavailable in the data source, which violates the condition of no unmeasured confounding. We provide examples and discuss mitigation strategies in the Supplemental Material, section 2.…”

“…12,15 Estimation of the propensity score Once a set of covariates is selected, different modeling approaches can be used to arrive at the final PS model, with the overarching goal of achieving covariate balance after PS matching or weighting (see section "Assessment of balance"). Logistic regression is the most popular approach among researchers in MS. 5 Kainz et al 16 recommend an iterative procedure to guide the construction of the PS model. First, all selected covariates should be included as main effects in the regression model, along with biologically plausible nonlinear (e.g.…”

“…Such details were seldom explicitly reported in MS studies. 5 Logistic regression requires the PS model to be correctly specified (i.e. the appropriate covariate transformations, nonlinear terms, and interaction terms are included) for the estimated PS to possess the balancing property.…”