Using Just Noticeable Differences to Interpret Test of Spoken English Scores

Abstract

This study explored the value of obtaining a Just Noticeable Difference (JND) -the difference in scores needed before observers discern a difference in examinees' English proficiency --for the current Test of Spoken English (TSE~) as a means of interpreting scores in practical terms, using college students' ratings of their international teaching assistants' English proficiency and adapting classical psychophysical methods. The test's concurrent validity against these ratings was also appraised. Three estimates of the JND were obtained. They varied considerably in size, but all were substantial when compared with the standard deviation of the TSE scores, the test's standard error of measurement, and guidelines for the effect size for mean differences. The TSE test correlated moderately with the rating criterion. The JND estimates appear to be meaningful and useful in interpreting the practical significance of TSE scores, and the test has some concurrent validity. ETS administers the TOEFL program under the general direction of a Policy Council that was established by, and is affiliated with, the sponsoring organizations. Members of the Policy Council represent the College Board, the GRE Board, and such institutions and agencies as graduate schools of business, junior and community colleges, nonprofit educational exchange agencies, and agencies of the United States government.A continuing program of research related to the TOEFL test is carried out under the direction of the TOEFL Research Committee. Its six members include representatives of the Policy Council, the TOEFL Committee of Examiners, and distinguished English as a second language specialists from the academic community. The Committee meets twice yearly to review and approve proposals for testrelated research and to set guidelines for the entire scope of the TOEFL research program. Members of the Research Committee serve three-year terms at the invitation of the Policy Council; the chair of the committee serves on the Policy Council.Because the studies are specific to the test and the testing program, most of the actual research is conducted by ETS staff rather than by outside researchers. Many projects require the cooperation of other institutions, however, particularly those with programs in the teaching of English as a foreign or second language. Representatives of such programs who are interested in participating in or conducting TOEFL-related research are invited to contact the TOEFL program office. All TOEFL research projects must undergo appropriate ETS review to ascertain that data confidentiality will be protected.Current (1996-97) members of the TOEFL Research Committee are:

Validating the Revised Test of Spoken English Against a Criterion of Communicative Success

“…TSE scores also relatedstrongly (r = -.68) to students' ratingsof the degree to whichan instructor's pronunciation interfered with studentcomprehension: the higherthe TSE score,the less the interference due to pronunciation. More.recently, Stricker(1997) computed a correlation of .38 between TSE scores (fromthe actual revised test) and studentratingsofteaehing assistants' abilityto communicate in English.…”