Measuring incremental changes in word knowledge: Experimental validation and implications for learning and assessment

Abstract

Knowledge of words is essential to success in early education, and it provides an important foundation for learning throughout adulthood (NRP, 2000;Stahl & Fairbanks, 1986). For example, Feingold (1983) found that vocabulary scores in high school predict 35%-40% of the variance in college GPA, which suggests that the breadth of a person's vocabulary is critical to academic success. Vocabulary also plays an important role in the mastery of particular subjects: From astronomy to zoology, practically every domain requires a thorough understanding of words and their meanings.Given the importance of word knowledge, a fundamental question is how word meanings are acquired. When and how are specific cognitive and linguistic processes engaged during meaning acquisition? What conditions lead to successful word learning? These questions open up a rich set of topics with relevance for semantic theory, language learning, and instructional tools to support vocabulary development.In the present article, we describe a new method for the assessment of word knowledge, the Markov estimation of semantic association (MESA), and show how this method can be used to expose the trajectory of word learning over time, providing a sensitive tool for studying psychological processes in word learning. We describe results using MESA scores from an experiment that was designed to elicit changes in word knowledge across multiple sentence contexts. After learners had read each sentence, they were asked to generate the meaning of the target word. The accuracy of the meaning derivation was scored using a continuous scale, in which accuracy was a function of how closely the response resembled the correct (dictionary) meaning. Because the definition-scoring measure that is generated by MESA is continuous, it can represent gradual changes in knowledge. We discuss how the trajectory of word learning is affected by the manipulation of instructional contexts, with a focus on the robustness of learning in the presence of misleading contexts (i.e., malapropisms). We further consider how these effects may vary as a function of individual differences in vocabulary and reading-comprehension skill. The Incremental Nature of Word LearningOne important aspect of word learning is that knowledge of word meanings typically accrues gradually, over The goal of this study was to test a new technique for assessing vocabulary development. This technique is based on an algorithm for scoring the accuracy of word definitions using a continuous scale (Collins-Thompson & Callan, 2007). In an experiment with adult learners, target words were presented in six different sentence contexts, and the number of informative versus misleading contexts was systematically manipulated. Participants generated a target definition after each sentence, and the definition-scoring algorithm was used to assess the degree of accuracy on each trial. We observed incremental improvements in definition accuracy across trials. Moreover, learning curves were sensitive to the proportion of misleadin...

Effects of induced orthographic and semantic knowledge on subsequent learning: a test of the partial knowledge hypothesis

“…These findings replicate prior work that demonstrates efficient and effective learning of word meanings from single-sentence contexts, particularly when learners are highly engaged and when the contexts are sufficiently directive to support lexical inferencing (Frishkoff, et al, 2008, 2011). Further, they contrast with studies of “incidental” CWL, where students are typically engaged in text comprehension rather than lexical inferencing and where the text itself provides unreliable or too few cues to support robust learning (Beck, et al, 2013).…”

Predicting Robust Vocabulary Growth from Measures of Incremental Learning

“…We assume, with many others, that growth of word knowledge is incremental (e.g., Frishkoff et al, 2008; Frishkoff, Perfetti, & Collins-Thompson, 2009; Reichle & Perfetti, 2003; Stahl, 2003) and that word knowledge itself is often passive, unstable, and partial (Brown, Frishkoff, & Eskenazi, 2005; Durso & Shore, 1991). Theories of partial word knowledge differ in detail, but each captures the idea that word knowledge develops along several dimensions: familiarity with word forms increases, different aspects of meaning (both denotative and connotative) are revealed through exposure to words in different contexts, and associations between form and meaning are strengthened.…”

“…In general terms, we know the answer: Learners require multiple, high-quality encounters with words in a variety of meaningful contexts (Beck, McKeown, & Kucan, 2002). However, what counts as a high-quality encounter—that is, a successful learning episode—may vary for different words at different points along the learning trajectory (Frishkoff, Collins-Thompson, Perfetti, & Callan, 2008). At this time little is known about the shapes of these trajectories or about how they vary in different learning contexts.…”

Word Maturity: A New Metric for Word Knowledge

“…Although estimating the average frequency or school level of the vocabulary known by a reader is a well-developed technology (e.g., Laufer, Elder, Hill, & Congdon, 2004, Meara & Buxton, 1987Nation, 1983;Read 1988), with some recent exceptions (e.g., Frishkoff, Collins-Thompson, Perfetti, & Callan, 2008), what is measured is essentially only the number, kinds, usage frequency, or grade levels in which they are used. Biemiller's (2008); Beck, McKeown, and Kucan's (2002); and similar prescriptions for appropriate words and kinds of words to teach are steps in this direction.…”