A Preliminary Assessment of the Validity and Reliability of the Performance Diagnostic Checklist-Human Services

The Performance Diagnostic Checklist–Human Services (1.1): An initial assessment of validity and reliability

“…We found no meaningful difference in accurate domain identification across vignettes. The overall mean accuracy score for this study was slightly higher than the corresponding score obtained by Cymbal et al (2020; M = 86.11%) when participants completed the original PDC‐HS but lower than those obtained by Wilder et al (2019; M = 97.06). However, the vignettes and scripted responses from the Wilder et al (2019) study were highly simulated, which may have contributed to the inflation of those scores relative to the scores obtained in the present study and by Cymbal et al Both Cymbal et al and the current study involved video vignettes designed to better reflect real‐world assessment conditions.…”

“…We created three different video vignettes; each was approximately 11 min and included a performance concern across one or more PDC‐HS (1.1) domains (see Supporting Information A, B, and C for transcripts of each vignette). As in Wilder et al (2019) and Cymbal et al (2020), each vignette was structured as an interview between a consultant using the PDC‐HS (1.1) and a supervisor answering questions based on the performance concern of an employee. Each vignette also included a 1‐to 2‐min observation of the employee demonstrating the performance deficit; this was meant to allow scoring of questions that required an observation component to be verified.…”

“…One limitation noted in the studies by Wilder et al (2019) and Cymbal et al (2020) was that participants were not required to select an intervention after completing the PDC‐HS. This is a necessary step in the performance analysis process.…”

The Performance Diagnostic Checklist‐Human Services: A brief review

“…Recently, Wilder, Lipschultz, Gehrman, Ertel, and Hodges () assessed the reliability and validity of the PDC‐HS by having participants use the tool to score videos simulating a performance problem. Each of three videos indicated a different problematic domain.…”

“…One particular aspect of reliability that needs further examination is the extent to which multiple supervisors agree when completing a PDC‐HS on a performance problem exhibited by the same employee. Although Wilder et al () examined the interrater reliability of the PDC‐HS, this study was conducted in an analog setting using videos of performance problems. Research on the interrater reliability of the tool when used with actual supervisors commenting on actual performance problems in real‐world settings is needed.…”