Validation of two measures for assessing English vocabulary knowledge on web-based testing platforms: brief assessments

Abstract

Two measures for assessing English vocabulary knowledge, the Vocabulary Size Test (VST) and the Word Familiarity Test (WordFAM), were recently validated for web-based administration. An analysis of the psychometric properties of these assessments revealed high internal consistency, suggesting that stable assessment could be achieved with fewer test items. Because researchers may use these assessments in conjunction with other experimental tasks, the utility may be enhanced if they are shorter in duration. To this end, two “brief” versions of the VST and the WordFAM were developed and submitted to validation testing. Each version consisted of approximately half of the items from the full assessment, with novel items across each brief version. Participants (n = 85) completed one brief version of both the VST and the WordFAM at session one, followed by the other brief version of each assessment at session two. The results showed high test-retest reliability for both the VST (r = 0.68) and the WordFAM (r = 0.82). The assessments also showed moderate convergent validity (ranging from r = 0.38 to 0.59), indicative of assessment validity. This work provides open-source English vocabulary knowledge assessments with normative data that researchers can use to foster high quality data collection in web-based environments.

The Web-based Open-access Reliable Decision on Synonyms (WORDS) English Vocabulary Test

“…Translating traditional lab-based vocabulary tests (e.g., Baddeley, Emslie, & Smith, 1992;Raven et al, 1989) into web-based versions can be difficult due to copyright issues (e.g., Shipley, Gruber, Martin, & Klein, 2009), requirements for experienced administrators (e.g., Wiig, Semel, & Secord, 2013), long completion times (e.g., Brown, Fishco, & Hanna, 1993; I. S. P. Nation & Beglar, 2007;Wiig et al, 2013), or low reliability (e.g., Shipley, 1940;see Vermeiren et al, 2023 about the reliability of the Shipley test). Several online and freely accessible vocabulary tests for English native speakers are now available and deploy different formats from yes/no questions testing pure recognition of words (e.g., Meara & Miralpeix, 2016) to multiple-choice questions requiring differentiation between several definition/description options (e.g., Drown, Giovannone, Pisoni, & Theodore, 2023a;2023b).…”

The Web-based Open-access Reliable Decision on Synonyms (WORDS) English Vocabulary Test

“…Translating traditional lab-based vocabulary tests (e.g., Baddeley, Emslie, & Smith, 1992 ; Raven et al, 1989 ) into web-based versions can be difficult due to copyright issues (e.g., Shipley, Gruber, Martin, & Klein, 2009 ), requirements for experienced administrators (e.g., Wiig, Semel, & Secord, 2013 ), long completion times (e.g., Brown, Fishco, & Hanna, 1993 ; I. S. P. Nation & Beglar, 2007 ; Wiig et al, 2013 ), or low reliability (e.g., Shipley, 1940 ; see Vermeiren et al, 2023 about the reliability of the Shipley test). Several online and freely accessible vocabulary tests for English native speakers are now available and deploy different formats from yes/no questions testing pure recognition of words (e.g., Meara & Miralpeix, 2016 ) to multiple-choice questions requiring differentiation between several definition/description options (e.g., Drown, Giovannone, Pisoni, & Theodore, 2023a ; 2023b ). A few self-administered and automated vocabulary tests with different formats (e.g., lexical decision, synonym matching, definition matching) have been demonstrated to have high correlations with another in-person vocabulary assessment requiring a trained administrator (the Peabody Picture Vocabulary Test) ( Harel et al, 2024 ).…”