A Comparison of Two Item-Selection Methodologies for Measuring Change in University Counseling Center Clients

American Journal of Evaluation

2004

Self Cite

Despite evidence that the choice of dependent measures can significantly influence design sensitivity, many evaluators default to traditional measures that may be insensitive to intervention effects. This paper describes an innovative set of test development guidelines designed to select items and create aggregate scales that are better able to detect program effects. The application of these Intervention Item Selection Rules (IISRs) is illustrated during the initial development of an outcome measure, completed by teachers, for elementary age children receiving psychosocial services from community mental health agencies. The major scale formed with these change-sensitive items displayed a larger effect size and an adequate reliability estimate.

Section: Discussionmentioning

confidence: 99%

Section: Alternative Procedures For Scale Development: the Iisrsmentioning

confidence: 99%

Improving Design Sensitivity through Intervention-Sensitive Measures

American Journal of Evaluation

2004

Self Cite

“…Intervention-sensitive items should remain relatively stable over time when no intervention is present (i.e., be reliable), but should evidence change in response to an intervention (i.e., evidence state effects). Research with progress monitoring and outcome measures developed using change-sensitive criteria indicates that change-sensitive measures possess different psychometric characteristics compared with tests developed through traditional approaches (Meier, 2000(Meier, , 2004Meier, McDougal, & Bardos, 2008;Weinstock & Meier, 2003). In general, scores on treatment-sensitive measures show larger effect sizes, when completed by individuals receiving psychosocial interventions, than scores on tests developed through traditional procedures.…”

Section: Focusing Methods Effects: Designing Studies and Measures To ...mentioning

confidence: 99%

Is it possible to integrate psychotherapy theory on the basis of empirical findings? The disruptive role of method effects.

Journal of Psychotherapy Integration

2018

Self Cite

Most approaches to psychotherapy integration assume that any integrated theory will rest on a solid empirical basis. Recent reconceptualizations of the research process, however, indicate that knowledge of research findings alone cannot provide such a foundation. Most critically, method effects influence the results of every study to the extent that the validity of theoretical propositions is tenuous without reference to the underlying methodological conditions. While researchers vary methodology during prestudies to enhance the possibility of finding statistically significant results, the details of such prestudies are typically unreported, and methodological conditions in general are underreported in all psychological research. In psychotherapy outcome studies, altering methodological conditions changes findings and subsequent conclusions. Assessment method, source of data, and the specific operation chosen, for example, have been demonstrated to influence degree of apparent change. Additional examples illustrate how differences in measures' change sensitivity can interact with participant characteristics to produce a dispersion of Effect Sizes (ESs) in response to psychosocial interventions. Prevailing approaches to psychotherapy integration, focusing on theoretical propositions based on empirical findings without reference to methodology, have produced conflicting results and little consensus beyond recognition of the need for additional efforts at integration. Adding a methodological emphasis should enhance detection of constructs, detection of relations among constructs, and replication of psychotherapy research findings to strengthen subsequent work on psychotherapy integration.

“…NA states such as depression and anxiety are present with many clients at the beginning of therapy and tend to decrease across sessions if therapy is effective; research indicates that client improvement occurs with verbalization of affect and affective processing (Greenberg, Korman, & Paivio, 2001). Krampen (2010); Vermeersch, Lambert, and Burlingame (2000); Vermeersch et al (2004); Weinstock and Meier (2003) reported the amount of change from counseling at the item level on several outcome measures. Assessing a broad range of problem domains with clients who received services at university counseling centers, outpatient clinics, private practitioners, and employee assistance programs, these studies found that scores on depression and anxiety items demonstrated larger effect sizes (ES) than items assessing other domains.…”

Section: Affect and Therapeutic Progressmentioning

confidence: 99%

Elevated trait anxiety: Obstacle to progress with female clients?

Professional Psychology: Research and Practice

2019

Self Cite

Contemporary clinicians employ progress-monitoring (PM) systems to differentiate between clients who are improving and those failing to make progress in psychotherapy. Once failing clients are identified, however, PM systems typically provide little feedback about how to adjust the provided therapy. The next step in the development of PM systems is to identify constructs that influence client progress and methods to assess these markers. Through a reanalysis of a previous study of psychotherapy progress involving 383 session ratings for 81 outpatient clients, this study examined the effect of trait anxiety on the amount and rate of client change across early sessions. Findings indicated that women with high trait anxiety evidenced smaller effect sizes and less likelihood of improvement on a measure of negative affect. Findings have implications for identification of high-trait-anxiety clients who evidence less improvement as well as treatment strategies with such individuals. Public Significance StatementTherapists are increasing their use of measures that assess client progress. This study investigates whether a measure of worry, particularly in women, can detect slow progress.