A Signal Detection Approach in a Multiple Cohort Study: Different Admission Tools Uniquely Select Different Successful Students

Abstract

Using multiple admission tools in university admission procedures is common practice. This is particularly useful if different admission tools uniquely select different subgroups of students who will be successful in university programs. A signal-detection approach was used to investigate the accuracy of Secondary School grade point average (SSGPA), an admission test score (ACS), and a non-cognitive score (NCS) in uniquely selecting successful students. This was done for three consecutive first year cohorts of a broad psychology program. Each applicant's score on SSGPA, ACS, or NCS alone-and on seven combinations of these scores, all considered separate "admission tools"-was compared at two different (medium and high) cut-off scores (criterion levels). Each of the tools selected successful students who were not selected by any of the other tools. Both sensitivity and specificity were enhanced by implementing multiple tools. The signal-detection approach distinctively provided useful information for decisions on admission instruments and cut-off scores.

The added value of signal detection theory as a method in evidence-informed decision-making in higher education: A demonstration

“…This information may suggest evidence-informed adjustments on how to use selection instruments. Importantly, we address accuracy (validity) of selection instruments at a level of decision-making regarding admitting or rejecting specific applicants, that is at an individual level (van Ooijen-van der Linden et al, 2018). In a wider context, the selection procedure as a whole should be evaluated not only on validity and reliability of the selection instruments but also additional criteria such as fairness, legality, educational impact etc.…”

“…In school diagnostics, SDT analyses can be used to detect academic impediments (Smolkowski and Cummings, 2015). Even though its application in other fields is well established, the use of SDT in higher education is surprisingly rare (for an example on applying SDT in undergraduate admissions, see van Ooijen-van der Linden et al, 2017Linden et al, , 2018.…”