A TAXONOMY OF EXPECTANCY TABLES¹

Abstract

Educational Testing ServiceDuring the past 60 years, correlation and regression have come to occupy a central position in measurement and research.Results obtained by these methods are particularly important in the evaluation of tests and in the use of test scores.Psychologists and educational researchers use these methods with confidence based on familiarity.Many persons concerned with research and testing, however, find results expressed in these terms difficult or impossible to interpret, and prefer to have results expressed in a more concrete form. This is particularly true if they are to make decisions on the basis of the results.In response to this need, expectancy tables have been developed.Walter V. Bin~ham was a particularly forceful advocate of their use for th~s purp~" This paper will offer a tentative classification of expectancy tables, together with some comments on the characteristics of various approaches.The somewhat form~ consideration of the possibilities may serve to clarify issues and may suggest suitable ways of dealing with specific problems.

A Note on Presenting What Predictive Validity Numbers Mean

“…The idea that expectancy tables, or other methods that go beyond simply reporting the correlation coefficient, may more clearly show the value of selection tests is not a new one (Taylor & Russell, 1939;Brogden, 1946;Schrader, 1965;Rosenthal & Rubin, 1982), but it is an idea that must be regularly rediscovered as test critics continue to focus on variance accounted for (Fairtest, 2007;Kidder & Rosner, 2002). In a recent exchange in Science magazine, responding to a metaanalysis on the predictability of graduate outcomes from test scores (Kuncel & Hezlett, 2007), two of three letters explicitly questioned the value of a 0.40 correlation (Brown, 2007;Sherley, 2007).…”

“…Two-and-a-half times as likely to excel may carry a very different message than more than half of the variance being unexplained. The methods in this study were clearly inspired by previous work with expectancy tables (e.g., Schrader, 1965), but they do modestly extend this work by dealing with the question of incremental validity, by their graphical presentation, and by aggregating data across a diverse set of departments.…”

“…Readers do not understand what an additional 10% of the variance means, but 10% of anything sounds quite unimportant. Since at least 1965, there have been clear warnings that even trained social scientists may be severely underestimating the practical importance of apparently small amounts of explained variance (Schrader, 1965;Rosenthal & Rubin, 1982).…”

“…For example, if 60% of applicants succeed before the test is used, and only 10% of applicants are to be selected, a test with a predictive validity coefficient of only 0.35 will raise the proportion succeeding to 82%. Schrader (1965) used a related approach with expectancy tables to show how the standing on the predictor, in terms of fifths, relates to standing on the criterion, also in terms of fifths. So, for example, if the correlation of predictor and criterion is 0.30, 10% of the students in the bottom fifth on the predictor will be in the top fifth on the criterion whereas 33% of students on the top fifth of the predictor will be in the top fifth on the criterion.…”

How language proficiency contributes to Chinese students’ academic success in Korean universities

“…However, even a small correlation can indicate a meaningful relationship (Cho & Bridgeman, 2012;Rosenthal & Rubin, 1982;Sackett et al, 2008;Schrader, 1965). This should be kept in mind when interpreting the correlations between TOPIK scores and indicators of academic success obtained in the present study.…”