Objective-The inability to blind research participants to their experimental conditions is the Achilles' heel of mental health services research. When one experimental condition receives more disappointed participants, or more satisfied participants, research findings can be biased in spite of random assignment. The authors explored the potential for research participants' preference for one experimental program over another to compromise the generalizability and validity of randomized controlled service evaluations as well as cross-study comparisons.Method-Three Cox regression analyses measured the impact of applicants' service assignment preference on research project enrollment, engagement in assigned services, and a service-related outcome, competitive employment.Results-A stated service preference, referral by an agency with a low level of continuity in outpatient care, and willingness to switch from current services were significant positive predictors of research enrollment. Match to service assignment preference was a significant positive predictor of service engagement, and mismatch to assignment preference was a significant negative predictor of both service engagement and employment outcome.Conclusions-Referral source type and service assignment preference should be routinely measured and statistically controlled for in all studies of mental health service effectiveness to provide a sound empirical base for evidence-based practice.Participants in research on mental health services are rarely blind to their experimental assignments, and so attrition caused by disappointment in service assignment is a wellrecognized risk. Researchers can minimize overall attrition, as well as the threat of differential attrition (1), by screening out applicants with a priori service preferences. However, some applicants may choose not to disclose their preference to ensure that they have a chance to be assigned to a favored program, or they may feel they do not have a personal preference over and above the obvious one posed by the study design. For instance, when the experimental comparison is between a new intervention and services as usual, many applicants may prefer the new program even before they learn much about it.Unfortunately, random assignment will not always equalize preexisting study enrollee characteristics across experimental groups (2), and this will always be the case when study
NIH-PA Author ManuscriptNIH-PA Author Manuscript NIH-PA Author Manuscript enrollees tend to prefer one service condition over another. Common sense tells us that service assignment preference must first be balanced within the total enrollee group for it to be distributed equitably across experimental conditions. For example, if 60% of enrollees have a preference for condition A and 40% have a preference for condition B, then, with true equivalence across conditions, service A would have 60% pleased and 40% disappointed assignees, while service B would have 40% pleased and 60% disappointed assignees. To the extent that pa...