The productivity of mental health care: an instrumental variable approach

Abstract

Background Like many other medical technologies and treatments, there is a lack of reliable evidence on treatment effectiveness of mental health care. Increasingly, data from non‐experimental settings are being used to study the effect of treatment. However, as in a number of studies using non‐experimental data, a simple regression of outcome on treatment shows a puzzling negative and significant impact of mental health care on the improvement of mental health status, even after including a large number of potential control variables. The central problem in interpreting evidence from real‐world or non‐experimental settings is, therefore, the potential ‘selection bias’ problem in observational data set. In other words, the choice/quantity of mental health care may be correlated with other variables, particularly unobserved variables, that influence outcome and this may lead to a bias in the estimate of the effect of care in conventional models. Aims of the Study This paper addresses the issue of estimating treatment effects using an observational data set. The information in a mental health data set obtained from two waves of data in Puerto Rico is explored. The results using conventional models—in which the potential selection bias is not controlled—and that from instrumental variable (IV) models—which is what was proposed in this study to correct for the contaminated estimation from conventional models—are compared. Methods Treatment effectiveness is estimated in a production function framework. Effectiveness is measured as the improvement in mental health status. To control for the potential selection bias problem, IV approaches are employed. The essence of the IV method is to use one or more instruments, which are observable factors that influence treatment but do not directly affect patient outcomes, to isolate the effect of treatment variation that is independent of unobserved patient characteristics. The data used in this study are the first (1992–1993) and second (1993–1994) wave of the ongoing longitudinal study Mental Health Care Utilization Among Puerto Ricans, which includes information for an island‐wide probability sample of over 3000 adults living in poor areas of Puerto Rico. The instrumental variables employed in this study are travel distance and health insurance sources. Results It is very noticeable that in this study, treatment effects were found to be negative in all conventional models (in some cases, highly significant). However, after the IV method was applied, the estimated marginal effects of treatment became positive. Sensitivity analysis partly supports this conclusion. According to the IV estimation results, treatment is productive for the group in most need of mental health care. However, estimations do not find strong enough evidence to demonstrate treatment effects on other groups with less or no need. The results in this paper also suggest an important impact of the following factors on the probability of improvement in mental health status: baseline mental health status,...

Substance Use, Education, Employment, and Criminal Activity Outcomes of Adolescents in Outpatient Chemical Dependency Programs

“…The expected direction of this bias cannot be determined a priori. 26 Adolescents with high motivation, for instance, may be more likely to initiate and comply with treatment. These adolescents, however, would also be more likely to decrease substance use, attend school, respect the law, or achieve other successful outcomes in various life areas.…”

“…Treatment would appear to have a smaller impact on school attendance, criminal activity, or employment than the true effect. 26,42 IV estimation attempts to remove the unobserved variation potentially correlated with the error term in Eqs. 1A and 1B from the main explanatory variable.…”

“…Other studies have successfully used geographic distance or geographic location as instrumental variables. 26,39,40 Mapquest's driving directions function estimated the distance between the treatment center and the zip code of the adolescent's residence in miles and the travel time in minutes. The average of the distance to the treatment facility from the adolescent's home zip code and to the home zip code from the treatment facility was calculated.…”

“…IV estimation has been applied to evaluate traditional medical interventions such as treatment of acute myocardial infarction 24 as well as various mental health and behavioral interventions. [25][26][27][28] To the authors' knowledge, this is the first study to use an IV technique to evaluate outcomes of a CD program for adolescents. Our results demonstrate the importance of incorporating estimation techniques that address selection into program evaluations of nonrandomized trials.…”

Estimation of Antipsychotic Effects on Hospitalization Risk in a Naturalistic Study With Selection on Unobservables

“…Methods for obtaining estimates from nonexperimental data that are free of selection bias caused by unobservables have recently been applied in several mental health treatment effectiveness studies (Crown et al, 1998a;1998b;Foster, 2000;Hylan et al, 1998;Lu, 1999). These methods generally involve the estimation of two different multiple regression models, with one model predicting the observed treatment choice and the second model predicting the outcome of interest (e.g., hospitalization).…”