Financial incentives for quality improvement and feedback on specific clients are two approaches to improving the quality of treatment for individuals with substance use disorders. We examined the impacts of these interventions in Washington State by randomizing outpatient substance use treatment agencies into intervention and control groups. From October 2013 through December 2015, agencies could earn financial incentives for meeting performance goals incorporating both achievement relative to a benchmark and improvement from agencies’ own baselines. Weekly feedback was e-mailed to agencies in the alert or alert plus incentives arms. Difference-in difference regressions controlling for client and agency characteristics showed that none of the interventions significantly affected client engagement after outpatient admissions, overall or for sub-groups based on race/ethnicity, age, rural residence, or agency baseline performance. Treatment agencies offered insights related to several themes: delivery system context (e.g., agency time and resources needed during transition to a managed behavioral healthcare system), implementation (e.g., data lag), agency issues (e.g., staff turnover), and client factors (e.g., motivation). Interventions took place during a time of Medicaid expansion and planning for statewide integration of mental health and substance use disorder treatment into a managed care model, which may have resulted in agencies not responding to the interventions. Moreover, incentives and alerts at the agency-level may not be effective when factors are at play beyond the agency’s control.