Expert and Target Scoring: Their relation, corresponding test instructions, and their effects on the construct validity of the video‐based social understanding test (VSU)

Abstract

This study investigated the relation between expert and target scoring of a video-based social understanding test (VSU) under two different types of instructions (internal and observer). The effects of the scoring methods and instructions on the VSU's construct validitywere also examined. A total of 529 pilot applicants completed the VSU (some with internal and some with observer instructions), cognitive ability and knowledge tests, and a personality questionnaire. A subsample (n = 132) completed the VSU again with the other instructions and participated in an assessment center (AC). The two scores were moderately correlated; correlations decreased when the instructions were considered. Neither expert nor target scores showed convergent validity with AC variables; none of the scoringinstruction combinations showed significant associations with the remaining measures.Note: *p < .05. **p < .01. GPA = grade point average (corresponding to the final grade of the university entrance diploma); inf = information; observ = observer; instr = instructions. a The sample size deviates from the maximum possible sample size since some of the applicants did not have the university entrance diploma when they applied.Note: *p < .05. **p < .01. S1 = Study 1; S2 = Study 2; T = target scoring; E = expert scoring; I = internal instructions; O = observer instructions; Inf. = information; GPA = grade point average (corresponding to the final grade on the university entrance diploma). a The sample size deviates from the maximum possible sample size because some of the applicants did not have the university entrance diploma when they applied.