The Validity of Immediate and Delayed Self‐reports in Training Evaluation: An Exploratory Field Study

Abstract

Deux Ctudes parallkles furent conduites pour Ctudier la validit6 de l'auto-Cvaluation d'un apprentissage, l'une utilisant un programme d'aide a l'entretien structurC de sClection pour Ctudiants dipl8mCs, l'autre un Requests for reprints should be sent to Paul Taylor, Department of Psychology, Private Bag 3105, University of Waikato, Hamilton, New Zealand. 0 1998 International Association of Applied Psychology 460 TAYLOR ET AL.and (3) analysing within-person (as well as across-person) correlations. The timing of self-report administration was experimentally manipulated to determine whether completion of self-reports after training generates less valid ratings than waiting until trainees have had an opportunity to apply newly learned skills. Results indicated limited support for the use of self-report instruments, regardless of when the questionnaires were administered. Although effect size estimates based on self-reports were similar to those derived from objective measures, acrossand within-person correlations were weak and inconsistent.

Evaluation of Ongoing Crisis Intervention Team (CIT) Training for Law Enforcement Using the ECHO Model

“…The number of CIT ECHO sessions completed was not controlled for in the analyses, and undoubtedly the impact of CIT ECHO would vary among LEOs who were “exposed” to CIT ECHO for the first time compared to those who had multiple exposures. Finally, all of the data collected in this evaluation were self-report, and self-reported data are subject to biases and limitations (Taylor et al 1998 ). Given these limitations, and the promising results of this exploratory evaluation, a more scientifically rigorous evaluation is warranted.…”

Coworker health awareness training: An evaluation

“…Another significant limitation of this study is the fact that all of the measures used were self‐report (Taylor, Lamers, Vincent, & O'Driscoll, ). Given that one of the primary goals of this training was to help employees learn how to recognize warning signs among coworkers and better support coworkers who may be struggling with a mental health problem, it is unfortunate that we were limited to self‐reported data among participants.…”

Effects of a Virtually Abroad Program on Students' Cross-Cultural Competence

“…Such objective measures have difficulty demonstrating validity, more specifically, criterion validity (Howard, 1994). An example of a study where self-report research failed to show strong validity is in the case of Taylor, Lamers, Vincent, and O'Driscoll (1998), in which a longitudinal research design was used to assess through self-report, respondents' learning. There were weak correlations within subjects from Time 1 to Time 2 -the time lapsed between the beginning of the training course and the conclusion -as well as weak correlations across individuals between Time 1 and Time 2.…”