Examining the Yes/No vocabulary test: some methodological issues in theory and practice

Abstract

This article evaluates the characteristics of the Yes/No test as a measure for receptive vocabulary size in second language (L2). This evaluation was conducted both on theoretical grounds as well as on the basis of a large corpus of data collected with French learners of Dutch. The study focuses on the internal qualities of the format in comparison with other more classical test formats. The central issue of determining a meaningful test score is addressed by providing a theoretical framework distinguishing discrete from continuous models. Correction formulae based on the discrete approach are shown to differ when applied to the Yes/No test in comparison with Multiple Choice (MC) or True/False formats. Correction formulae based on the continuous approach take the response bias into account but certain underlying assumptions need to be validated. It is shown that both correction schemes display several shortcomings and that most of the data relative to the reliability of the Yes/No test presented in the literature are overestimated. Finally, several future research options are proposed in order to attain a straightforward but reliable and valid instrument for measuring receptive vocabulary size.

Identifying academically at-risk students in an English-as-a-Lingua-Franca university setting

“…The mean difference was statistically significant, t(175) ¼ 2.38, p ¼ .018, r ¼ .593. The false alarm rate is high when compared to previous research, where it ranged from 5% (Mochida & Harrington, 2006;Harrington & Carey, 2009) to 20% (Beeckmans, Eyckmans, Janssens, Dufranne, & Van de Velde, 2001;Cameron, 2002).…”

Conventional Expressions as a Pragmalinguistic Resource: Recognition and Production of Conventional Expressions in L2 Pragmatics

“…Despite the fact that vocabulary studies have observed that learners tend to be conservative in their judgments about whether they know words and underestimate rather than overestimate their knowledge (Meara), it is possible that learners are not as conservative in their estimate of familiarity for expressions. In fact, studies by Eyckmans, Van de Velde, and colleagues suggest that learners significantly overestimate their knowledge on yes/no tests even for vocabulary items (Beeckmans, Eyckmans, Janssens, Dufranne, & Van de Velde, 2001;Eyckmans, Van de Velde, Van Hout, & Boers, 2007), so the finding for expressions may not be unusual. In addition, the use of a written format for conversational expressions might have led to an exaggeration of self-reported recognition.…”

What vocabulary size tells us about pronunciation skills: Issues in assessing L2 learners

“…Clearly, the trustworthiness of learners' self-evaluations is debatable (Ross, 1998), so that these results may not be sufficient to cast doubt on the applicability of Dialang vocabulary test as a quick evaluation of learners' level in the four skills, but the limited applicability of vocabulary tests has already been pointed out by other researchers (cf. Beeckmans et al, 2001). More specifically, Eyckmans (2004) reported the failure of Dialang vocabulary test to account for the effects of cognates, something that may clearly play a relevant role for Italian learners of L2 French.…”