Validation of two measures for assessing English vocabulary knowledge on web-based testing platforms: long-form assessments

Abstract

The goal of the current work was to develop and validate web-based measures for assessing English vocabulary knowledge. Two existing paper-and-pencil assessments, the Vocabulary Size Test (VST) and the Word Familiarity Test (WordFAM), were modified for web-based administration. In Experiment 1, participants (n = 100) completed the web-based VST. In Experiment 2, participants (n = 100) completed the web-based WordFAM. Results from these experiments confirmed that both tasks (1) could be completed online, (2) showed expected sensitivity to English frequency patterns, (3) exhibited high internal consistency, and (4) showed an expected range of item discrimination scores, with low frequency items exhibiting higher item discrimination scores compared to high frequency items. This work provides open-source English vocabulary knowledge assessments with normative data that researchers can use to foster high quality data collection in web-based environments.

The Web-based Open-access Reliable Decision on Synonyms (WORDS) English Vocabulary Test

“…Translating traditional lab-based vocabulary tests (e.g., Baddeley, Emslie, & Smith, 1992;Raven et al, 1989) into web-based versions can be difficult due to copyright issues (e.g., Shipley, Gruber, Martin, & Klein, 2009), requirements for experienced administrators (e.g., Wiig, Semel, & Secord, 2013), long completion times (e.g., Brown, Fishco, & Hanna, 1993; I. S. P. Nation & Beglar, 2007;Wiig et al, 2013), or low reliability (e.g., Shipley, 1940;see Vermeiren et al, 2023 about the reliability of the Shipley test). Several online and freely accessible vocabulary tests for English native speakers are now available and deploy different formats from yes/no questions testing pure recognition of words (e.g., Meara & Miralpeix, 2016) to multiple-choice questions requiring differentiation between several definition/description options (e.g., Drown, Giovannone, Pisoni, & Theodore, 2023a;2023b).…”

The Web-based Open-access Reliable Decision on Synonyms (WORDS) English Vocabulary Test

“…Translating traditional lab-based vocabulary tests (e.g., Baddeley, Emslie, & Smith, 1992 ; Raven et al, 1989 ) into web-based versions can be difficult due to copyright issues (e.g., Shipley, Gruber, Martin, & Klein, 2009 ), requirements for experienced administrators (e.g., Wiig, Semel, & Secord, 2013 ), long completion times (e.g., Brown, Fishco, & Hanna, 1993 ; I. S. P. Nation & Beglar, 2007 ; Wiig et al, 2013 ), or low reliability (e.g., Shipley, 1940 ; see Vermeiren et al, 2023 about the reliability of the Shipley test). Several online and freely accessible vocabulary tests for English native speakers are now available and deploy different formats from yes/no questions testing pure recognition of words (e.g., Meara & Miralpeix, 2016 ) to multiple-choice questions requiring differentiation between several definition/description options (e.g., Drown, Giovannone, Pisoni, & Theodore, 2023a ; 2023b ). A few self-administered and automated vocabulary tests with different formats (e.g., lexical decision, synonym matching, definition matching) have been demonstrated to have high correlations with another in-person vocabulary assessment requiring a trained administrator (the Peabody Picture Vocabulary Test) ( Harel et al, 2024 ).…”