Validation of two measures for assessing English vocabulary knowledge on web-based testing platforms: Long-form assessments

Abstract

The goal of the current work was to develop and validate web-based measures for assessing English vocabulary knowledge. Two existing paper-and-pencil assessments, the Vocabulary Size Test (VST) and the Word Familiarity Test (WordFAM), were modified for web-based administration. In Experiment 1, participants (n = 100) completed the web-based VST. In Experiment 2, participants (n = 100) completed the web-based WordFAM. Results from these experiments confirmed that both tasks (1) could be completed online, (2) showed expected sensitivity to English frequency patterns, and (3) revealed high split-half reliability, suggesting that stable vocabulary assessment could be achieved with fewer test items. Based on the results of Experiments 1 and 2, two “brief” versions of the VST and WordFAM were developed. Each version consisted of approximately half of the items from the full assessment, with novel items presented across the two brief versions of each assessment. In Experiment 3, participants (n = 85) completed one brief version of both the VST and WordFAM at session one, followed by the other brief version of each task at session two. The results showed high test-retest reliability for both the VST (r = 0.68) and WordFAM (r = 0.82). The two brief assessments also showed moderate convergent validity (ranging from r = 0.38 to r = 0.59) indicative of construct validity for each assessment. This work provides open-source vocabulary knowledge assessments with normative data that researchers and clinicians can use to foster high quality data collection in web-based environments.