Development of a Measure to Assess the Implementation of Children’s Systems of Care: The Systems of Care Implementation Survey (SOCIS)

Abstract

The children's system of care framework has been extensively implemented in the U.S. Since its inception in 1993, the Comprehensive Community Mental Health Services for Children and Their Families Program has invested in excess of $1 billion supporting the development of systems of care in 164 grantee sites across the country. Despite these efforts to implement children's systems of care nationally, little is known about the extent to which the principles and values actually have been put into practice outside of the funded grantee sites. This paper describes the development of the Systems of Care Implementation Survey, a measure designed specifically for the first ever study assessing the level of implementation of factors contributing to effective children's systems of care in a nationally representative sample of counties throughout the U.S.

Multilevel Confirmatory Factor Analysis of the Systems of Care Implementation Survey (SOCIS)

“…Face validity was established for the 14 SOC factors in an earlier phase of the SOCIS instrument development by obtaining agreement among a panel of children's mental health services experts that the items were appropriate and important components for the construct in question. 2 Construct validity refers to the instrument-derived factor's ability to be related to other constructs that are part of the nomological network of theory-based constructs known to be associated with the construct in question. A particular form of construct validity, concurrent validity, refers to the SOCIS instrument's ability to be associated with other county-level variables that were measured at the same time and, based on existing theory, were expected to be related to the SOC factors.…”

“…As noted by Boothroyd et al, although a number of local communities have measured their development of systems of care, there has been no measurement of the status of SOC and its development on a national level. 2 Assessment of the current level of SOC implementation throughout the United States would provide a better understanding of the nation's capacity to deliver comprehensive services to children and their families, while at the same time supplying information for policy-makers' future planning to expand the amount and distribution of those areas needed to move local communities forward in providing a national network of SOC.…”

“…The conceptualization that is examined in the current study specifies 14 different factors or constructs and although this factor structure represents one version of the state-ofthe-art, other theoretical perspectives might have selected a different mix (see related article in this issue for a detailed rationale for construct selection). 2 The 14 factors selected to represent the SOC were: (a) family choice and voice, (b) individualized, comprehensive, and culturally competent treatment, (c) outreach and access to care, (d) transformational leadership, (e) theory of change, (f) implementation plan, (g) local population of concern, (h) interagency and cross-sector collaboration, (i) values and principles, (j) comprehensive financing plan, (k) skilled provider network, (l) performance measurement system, (m) provider accountability, and (n) management and governance (see Table 1 for more specific components of each factor). Second, a SOC is a complex organizational system that includes both a large number of factors that need to be measured and, because SOC is a communityor organizational-level concept, must be measured at the group rather than the individual level.…”

“…The initial steps in constructing the SOCIS have been described in detail elsewhere. 2 The purpose of the present study was to provide psychometric information, particularly with regard to the construct validity of the SOCIS scales as measured by the scales' factorial and concurrent validities. Multilevel confirmatory factor analysis (MCFA) and multilevel regression analysis were used to test these indices of internal and external validity for the 14 SOCIS scales used to measure those constructs believed to be essential to the operation of a community's SOC.…”

Levels of System of Care Implementation: A National Benchmark Study

“…Other articles in this special section 25,26 provide an in-depth description of the sampling procedures and the psychometric properties of the survey used for the current study, the SOCIS.…”