Drawing on experience between 2000 and 2007 in developing a validity argument for the high‐stakes Test of English as a Foreign Language™ (TOEFL®), this paper evaluates the differences between the argument‐based approach to validity as presented by Kane (2006) and that described in the 1999 AERA/APA/NCME Standards for Educational and Psychological Testing. Based on an analysis of four points of comparison—framing the intended score interpretation, outlining the essential research, structuring research results into a validity argument, and challenging the validity argument—we conclude that an argument‐based approach to validity introduces some new and useful concepts and practices.

No abstract

The planned introduction of a computer-based Test of English as a Foreign Language (TOEFL) test raises concerns that language proficiency will be confounded with computer proficiency, introducing construct-irrelevant variance to the measurement of examinees' English-language abilities. We administered a questionnaire focusing on examinees' computer familiarity to 90,000 TOEFL test takers. A group of 1,200 "low-computer-familiar" and "highcomputer-familiar" examinees from 12 international sites worked through a computer tutorial and a set of 60 computer-

A continuing program of research related to the TOEFL test is carried out under the direction of the TOEFL Research Committee. Its six members include representatives of the Policy Council, the TOEFL Committee of Examiners, and distinguished English as a second language specialists from the academic community. The Committee meets twice yearly to review and approve proposals for testrelated research and to set guidelines for the entire scope of the TOEFL research program. Members of the Research Committee serve three-year terms at the invitation of the Policy Council; the chair of the committee serves on the Policy Council.Because the studies are specific to the test and the testing program, most of the actual research is conducted by ETS staff rather than by outside researchers. Many projects require the cooperation of other institutions, however, particularly those with programs in the teaching of English as a foreign or second language. Representatives of such programs who are interested in participating in or conducting TOEFL-related research are invited to contact the TOEFL program office. All TOEFL research projects must undergo appropriate ETS review to ascertain that data confidentiality will be protected.Current members of the TOEFL Research Committee are: Web Site: http://www.toefl.orgii An international study of this scale and complexity could not have been completed without the collaboration and assistance of many professionals from ETS, Sylvan Learning Systems, Inc., and the TOEFL overseas representatives. We wish to thank ETS staff in TOEFL Program Direction, Test Development, Research, and Operations Management whose support and financial commitments made this project possible. We also wish to acknowledge the TOEFL Research Committee for their encouragement and support throughout the study.Some individuals, because of their special contributions, deserve mention. Charles Lewis, Donald Rock, Henry Braun, and Samuel Messick provided invaluable advice on the design of the study. Patricia Santiago and Terrie Mansmann provided guidance with operational issues. Test developers Felicia DeVincenzi, Philip Everson, Susan Nissan, Cynthia Potter, and Mary Schedl were responsible for developing the test items used in the study and reviewed the tutorials at various stages of development. The tutorial development teamworked on an accelerated schedule to develop and package the tutorial and test items. Thomas Florek advised on all equipment needs, oversaw the loading of the computer programs at Sylvan, made a site visit, and was on call to troubleshoot technical problems during data collection.Sylvan Learning Systems, Inc. staff provided inestimable support with all logistics of the data collection. In retrospect we could not have managed the international computer administrations without Sylvan's partnership. Brett Lundeen negotiated contracts and arrangements with TOEFL overseas representatives. Kent Weatherley coordinated and oversaw all Sylvan activities. Sylvan's test administrators were extremely accommodat...

This article reports the results of a study of the effectiveness of computer‐assisted language learning (CALL) in the acquisition of English as a second language by Arabic‐ and Spanish‐speaking students in an intensive program. The study also examined two student variables—time spent using and attitude toward the CALL lessons—as well as four cognitive/affective characteristics—field independence, ambiguity tolerance, motivational intensity, and English‐class anxiety. English proficiency was measured by the TOEFL and an oral test of communicative competence. Results indicated that the use of CALL lessons predicted no variance on the criterion measures beyond what could be predicted by the cognitive/affective variables. In addition, it was found that time spent using and attitude toward CALL were significantly related to field independence and motivational intensity. These results indicate that (a) certain types of learners may be better suited to some CALL materials than other students and (b) it is necessary to consider many learner variables when researching the effectiveness of CALL.