The accurate identification of persons at risk of exiting permanent supportive housing could help maximize client success and minimize attrition and premature exits from such housing. Thus, in the present study, we developed and tested multivariable prediction models of negative and positive exits from the U.S. Department of Housing and Urban Development-Veterans Affairs Supportive Housing (HUD-VASH) program using logistic regression and random forests. We compared the performance of these models with clinical predictions made by HUD-VASH program case managers. We selected a cohort of all 92,196 Veterans who entered HUD-VASH nationwide between October 1, 2014 and September 30, 2019, 70% of whom were randomly selected to serve as the development cohort and the remaining 30% of whom served as the validation cohort. Negative and positive exits were measured until September 30, 2019. A subset of 1,264 Veterans was used to compare performance of models with clinical judgment. Predictor variables included sociodemographic characteristics, health and behavioral health diagnoses, homeless/housing history, and VA service utilization history. Performance of models and clinical judgment were assessed using an array of metrics including area under the receiver operating characteristic curve (AUROC), sensitivity, specificity, and positive predictive value. The logistic regression and random forest models had similar, modest performance in predicting negative and positive exits. These models were substantially more sensitive, yet far less specific in predicting exits than clinician ratings. Study findings highlight the challenges and tradeoffs in using actuarial models or case manager predictions to target interventions to Veterans at risk of exiting HUD-VASH.
Public Policy Relevance StatementThe U.S. Department of Housing and Urban Development-Veterans Affairs Supportive Housing (HUD-VASH) program provides permanent supportive housing (PSH) to roughly 90,000 veterans nationwide and is a key part of the federal government's efforts to end homelessness among Veterans. Preventing negative exits or facilitating positive exits from HUD-VASH may be a desirable goal to ensure that the program is as effective as possible.Our study provides new evidence about the performance of actuarial models and clinician judgment in predicting exits from the program, and thus has implication for continued policy and programmatic efforts to address homelessness among veterans.