Moving the field of vocabulary assessment forward: The need for more rigorous test development and validation

Schmitt, Norbert; Nation, Paul; Kremmel, Benjamin

doi:10.1017/s0261444819000326

Cited by 113 publications

(72 citation statements)

References 36 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…However, the instability of the relative weights of these influences means that it is not feasible to prescribe target responses, for example as benchmarks of proficiency, or to reliably predict WA behaviour from specified variables (or vice versa). A further warning is sounded by Schmitt, Nation & Kremmel (2019) who, noting that test revisions should not be uncritically welcomed, call for revised versions to be subjected to the same validation criteria as new tests. The pattern of iterative refinements to test formats that we have seen in relation to stereotype and WAF measures in particular are cause for concern in this regard.…”

Section: Resultsmentioning

confidence: 99%

Word association research and the L2 lexicon

Fitzpatrick

Thwaites

2020

Lang. Teach.

View full text Add to dashboard Cite

Since its modern inception in the late nineteenth century, research on word associations has developed into a large and diverse area of study, including work with both applied linguistic and psycholinguistic orientations. However, despite significant recent interest in the use of word association to investigate second language (L2) vocabulary knowledge and testing, there has until now been no systematic attempt to review the wider word association research tradition for the benefit of L2-oriented researchers and practitioners. This paper seeks to address this, drawing together linguistic research from the past 150 years, with a focus on research published since 2000. We evaluate the current state of L2 word association research, before identifying methodological and theoretical themes from a broader range of disciplinary approaches. Emerging from this, new paradigms are identified which have potential to catalyse a new phase of work for second language word association scholars, and which indicate priority foci for future work.

show abstract

Section: Resultsmentioning

confidence: 99%

Word association research and the L2 lexicon

Fitzpatrick

Thwaites

2020

Lang. Teach.

View full text Add to dashboard Cite

show abstract

“…There are other aspects that are important but were not examined (e.g., spoken forms, word parts, grammatical functions), and future research should include measures of size and depth by using various instruments (e.g., Godfroid, 2020, for offline and online measures to cover Nation, 2020, vocabulary elements) to examine their relationships. Tests should be developed, and test validation should be conducted, by following the principles summarized by Schmitt et al (2020).…”

Section: Resultsmentioning

confidence: 99%

“…This line of research, which should also include vocabulary constructs (i.e., the vocabulary knowledge and ability that tests are intended to measure and what is actually measured; see Chapelle, 1998), has hitherto been limited. The necessity of investigating the quality of vocabulary tests and their constructs has been emphasized by Schmitt et al (2020), who stated that L2 vocabulary fields need step-by-step test development and validation of vocabulary tests to allow for the meaningful interpretation and application of test scores.…”

Section: Introductionmentioning

confidence: 99%

Structural Equation Modeling of Vocabulary Size and Depth Using Conventional and Bayesian Methods

Koizumi

In’nami

2020

Front. Psychol.

View full text Add to dashboard Cite

In classifications of vocabulary knowledge, vocabulary size and depth have often been separately conceptualized (Schmitt, 2014). Although size and depth are known to be substantially correlated, it is not clear whether they are a single construct or two separate components of vocabulary knowledge (Yanagisawa and Webb, 2020). This issue has not been addressed extensively in the literature and can be better examined using structural equation modeling (SEM), with measurement error modeled separately from the construct of interest. The current study reports on conventional and Bayesian SEM approaches (e.g., Muthén and Asparouhov, 2012) to examine the factor structure of the size and depth of second language vocabulary knowledge of Japanese adult learners of English. A total of 255 participants took five vocabulary tests. One test was designed to measure vocabulary size in terms of the number of words known, while the remaining four were designed to measure vocabulary depth in terms of word association, polysemy, and collocation. All tests used a multiple-choice format. The size test was divided into three subtests according to word frequency. Results from conventional and Bayesian SEM show that a correlated two-factor model of size and depth with three and four indicators, respectively, fit better than a single-factor model of size and depth. In the two-factor model, vocabulary size and depth were strongly correlated (r = 0.945 for conventional SEM and 0.943 for Bayesian SEM with cross-loadings), but they were distinct. The implications of these findings are discussed.

show abstract

“…Therefore, answering VLT items correctly suggests learners have only a very basic receptive knowledge of the form-meaning connection for the target words. There are also disadvantages with the multiple-choice format of the VLT, which allows for guessing and of testing strategies such as the process of elimination, something that could exaggerate responses and needs consideration when reporting results (Gyllstad et al, 2015;Pecorari et al, 2019;Schmitt et al, 2019). Concerns have been raised because the revised VLT (Schmitt et al, 2001) has not been updated since 2001 (Schmitt et al, 2019).…”

Section: Measuring Academic English Vocabulary Knowledgementioning

confidence: 99%

Mind the gap: Academic vocabulary knowledge as a predictor of English grades

Skjelde

Coxhead

2020

ADNO

View full text Add to dashboard Cite

Knowledge of academic English vocabulary is necessary for Norwegian speaking upper secondary students for the completion of their English course and to qualify for university studies. General academic vocabulary occurs in academic texts across disciplines (for example, furthermore, research). The focus of this article is meaning-recognition knowledge of written academic English vocabulary and associations with English course grades. The academic sections from versions one and two of the Vocabulary Levels Test (VLT) (Nation, 1990; Schmitt et al., 2001) were administered to 134 Norwegian-speaking students aged 15 - 17. Participants were first-year upper secondary school students taking the final obligatory English course provided in the Norwegian educational context. On average, learners had high levels of meaning-recognition knowledge (80.93%). However, 58.21% failed to reach minimum mastery levels of 52/60 for the test, and results varied considerably (SD = 8.31). A Spearman correlation revealed significant associations between VLT scores and English course grades. Compared to students who did not reach a mastery level, the odds of receiving a higher English grade were nine times greater for students with a maximum of two mistakes on the academic section of the VLT, and four times greater for students who reached the minimum mastery level. Findings indicate a need for these learners to attain a greater understanding of academic vocabulary in English and suggest the need for a principled focus on academic vocabulary acquisition. Keywords: academic vocabulary; academic achievement; vocabulary testing; second language learning; mastery levels «Mind the gap» - Akademisk ordforråd som prediktor for karakterer i engelsk SammendragKunnskap om akademiske ord er nødvendig for at norske elever i videregående skole skal kunne gjennomføre engelskfaget og for at de skal kunne kvalifisere seg til høyere utdanning. Akademiske ord omfatter ord som forekommer i vitenskapelige tekster fra ulike akademiske disipliner (for eksempel dessuten, forskning). Fokuset i denne artikkelen er forholdet mellom elevers kunnskap om skriftlige akademiske ord og karakterene deres i engelsk. 134 norsktalende elever på Vg1 studiespesialiserende mellom 15 og 17 år tok Nations (1990) VLT, den akademiske delen fra versjon en og to av Schmitt et al. (2001) reviderte test. Resultatene viser at elevene i gjennomsnitt hadde en høy VLT- poengsum (80.93%), men 58.21% ikke klarte å oppnå den laveste anbefalte poengsummen på 52/60 og det var en høy grad av variasjon i resultatene (SD = 8.31). En Spearman-korrelasjon viste signifikante forbindelser mellom resultatene fra VLT og elevenes engelskkarakterer. Sannsynligheten for å ha en høyere engelskkarakter var ni ganger større for elever med maksimum to feil på VLT- testen og fire ganger større for de som oppnådde minimumskravet på 52/60. Funnene indikerer at disse elevene har behov for å utvikle en bedre forståelse av akademiske ord, og de antyder dermed også et behov for fokus på utvikling av akademisk ordforråd i engelskundervisningen. Nøkkelord: akademisk ordforråd, testing av vokabular, engelsk som andrespråk, språkundervisning

show abstract

Moving the field of vocabulary assessment forward: The need for more rigorous test development and validation

Cited by 113 publications

References 36 publications

Word association research and the L2 lexicon

Word association research and the L2 lexicon

Structural Equation Modeling of Vocabulary Size and Depth Using Conventional and Bayesian Methods

Mind the gap: Academic vocabulary knowledge as a predictor of English grades

Contact Info

Product

Resources

About