Psychometric Assessment of an Item Bank for Adaptive Testing on Patient-Reported Experience of Care Environment for Severe Mental Illness: Validation Study (Preprint)

Abstract

BACKGROUND The care environment significantly influences the experiences of patients with severe mental illness (SMI) and the quality of their care. While a welcoming and stimulating environment enhances patient satisfaction and health outcomes, psychiatric facilities often prioritize staff workflow over patient needs, leading to a prison-like atmosphere and limited social interactions. Addressing these challenges is crucial to improve patient experiences and outcomes in mental health care. OBJECTIVE This study aimed to establish an item bank (PREMIUM-CE) and develop a computerized adaptive test (CAT) to measure the care environment experience of adult patients with schizophrenia, bipolar disorder, or major depressive disorder. METHODS Psychometric analyses included assessments of item response theory (IRT) model assumptions, IRT model fit, differential item functioning (DIF), item bank validity, and CAT simulations. RESULTS In this multicenter cross-sectional study, 498 patients were recruited from outpatient and inpatient settings. The 13-item bank was sufficiently unidimensional, had no item bias, and fit the IRT model well (RMSEA = 0.088, 95% CI [0.073–0.103], CFI = 0.967, TLI = 0.961). CAT simulations demonstrated that assessments could be performed with only 7 items, with correlations of 0.98 between CAT and the full item bank. CONCLUSIONS The PREMIUM-CE item bank and its CAT version have shown excellent psychometric properties, making them reliable measures for evaluating the patient experience of the care environment among adults with SMI in both out- and in-patient settings. These measures are a valuable addition to the existing landscape of patient experience assessment, capturing what truly matters to patients and enhancing the understanding of their care experiences. CLINICALTRIAL The trial registration is NCT02491866. The assessment protocol was approved by the relevant ethical review board (CPP-Sud Méditerranée V, n°2014-A01152-45). All data are collected anonymously.