The Program Environment Scale: Assessing Client Perceptions of Community‐Based Programs for the Severely Mentally Ill

Abstract

The Program Environment Scale (PES) was developed for use with clients of community-based programs for the severely mentally ill. It is intended to fill the gap in available tools for assessing clients' perceptions of program functioning as it affects their "quality of life" in a program. Formal pretests were conducted with 121 clients at 12 randomly selected programs near Washington, DC. The final field test used a revised form (29 domains; 129 items) with 221 clients in 22 programs selected randomly throughout the U.S., including Clubhouse, day treatment, psychosocial rehabilitation, and social club programs. Twenty-three subscales met at least five of eight psychometric criteria for internal consistency and discriminant validity. A 24th subscale was retained because of its substantive importance. Successful subscales cover program atmosphere and interactions (program cares about me, energy level, friendliness, openness, staff-client and client-client respect, reasonable rules, availability of positive physical contact, protection from bad touch, staff investment in their jobs, and confidentiality), client empowerment/staff-client equality (program and treatment empowerment, egalitarian space use), and service components (support for paid work, work importance, emergency access, family activities, housing, public benefits, community activities, medications, substance abuse, and continuity). Subscale validity is indicated by associations of specific service offerings with scores on scales measuring client perceptions of those services, and by an ability to differentiate among program models (i.e., Clubhouses, day treatment programs, and psychosocial rehabilitation programs look different from each other). Subscale scores were not influenced by client characteristics (gender, race, age, diagnosis, number of hospitalizations, length of time in program). The final scale has 97 items and takes about 25 minutes to complete. The PES succeeds in measuring different aspects of programs as clients perceive them. In the programs we visited, directors felt the PES covers the important things they want to know about how clients perceive their program. The PES should become a useful tool both for researchers interested in how client responses to programs may influence their therapeutic outcomes, and for practitioners interested in improving their clients' program experiences and/or increasing convergence of staff and client views of their program.

Doing the Difficult and Dangerous: The Community Program Practice Scale

“…Four studies included statewide samples of programs. In one case the CPPS was included in a study with a parallel client survey in a national random sample of 22 programs that included clubhouse, day treatment, psychosocial rehabilitation and social club (drop-in center) programs (Burt et al, 1998). Many other programs were added by administrators or university people who were interested in seeing the range of practices on a common scale for all of the programs in a catchment area or large city.…”

“…Fidelity measures are also essential in dismantling studies aimed at assessing which of a model's practices are critical to effectiveness and cost-effectiveness of services. The second type of tool is the cross-model practices measure, which assesses a variety of service practices across a range of program models (e.g., Burt, Duke, & Hargreaves, 1998).…”

Developing an instrument for assessing fidelity to the intervention in the Critical Time Intervention – Task Shifting (CTI-TS) - preliminary report

“…Content analysis can identify problems in terminology, description and item scores, assess the adequacy of the scale of responses, observe if there are important topics identified in practice that were not included in the scale, as well as increasing the expertise of the performance evaluator. Psychometric analysis determines the reliability and validity of an instrument [17]. This is a limitation of the present article, since the psychometric properties of the fidelity scale have not been tested so far, as the pilot study in Brazil has not yet started.…”

“…The fidelity scale and assessment sheets developed should be tested in a pilot, after proper training of raters in the method [17].…”

Factors Influencing Staff Perceptions of the Organizational Environment of Clubhouses

“…The Program Environment Scale (PES; Burt et al 1998) was used to assess perceptions of the clubhouse that included the program's climate, quality of interactions, clubhouse member empowerment and equity, and service components. The PES is a cross-model practice measure that has been used in assessing common elements across a variety of community based psychiatric programs (Hargreaves et al 2007).…”