Looking and touching: what extant approaches reveal about the structure of early word knowledge

Abstract

The goal of the current study is to assess the temporal dynamics of vision and action to evaluate the underlying word representations that guide infants’ responses. Sixteen-month-old infants participated in a two-alternative forced-choice word-picture matching task. We conducted a moment-by-moment analysis of looking and reaching behaviors as they occurred in tandem to assess the speed with which a prompted word was processed (visual reaction time) as a function of the type of haptic response: Target, Distractor, or No Touch. Visual reaction times (visual RTs) were significantly slower during No Touches compared to Distractor and Target Touches, which were statistically indistinguishable. The finding that visual RTs were significantly faster during Distractor Touches compared to No Touches suggests that incorrect and absent haptic responses appear to index distinct knowledge states: incorrect responses are associated with partial knowledge whereas absent responses appear to reflect a true failure to map lexical items to their target referents. Further, we found that those children who were faster at processing words were also those children who exhibited better haptic performance. This research provides a methodological clarification on knowledge measured by the visual and haptic modalities and new evidence for a continuum of word knowledge in the second year of life.

Assessing a continuum of lexical–semantic knowledge in the second year of life: A multimodal approach

“…For Aim 2, we conducted a moment-by-moment analysis of looking and reaching behaviors as they occurred in tandem to assess the speed with which a prompted word is processed (visual RT) as a function of the type of haptic response: target touch (touched the picture of the prompted image), distractor touch (touched the picture of the unprompted image), or no touch (failed to touch either image). In line with previous results, we predicted that visual RT would vary as a function of haptic response (Hendrickson et al, 2015). Specifically, we predicted that visual RTs would be fastest for target touches and slowest for no touches, with an intermediate speed of processing for visual RTs associated with distractor touches.…”

“…Therefore, in the case of distractor touches, the visual and haptic response modalities conflict; that is, children are quick to fixate the target image but touch the distractor image. So, although evidence within the same study demonstrates a significant relation among different measures of word comprehension (visual reaction time, haptic response, and parent report), at the item level word knowledge is highly task dependent; children can demonstrate knowledge in one modality (e.g., visual) but not in the other (e.g., haptic) (Hendrickson et al, 2015). …”

“…An alternate view suggests that word knowledge is not dichotomous but instead exists on a continuum from absence of knowledge, to partial knowledge, to robust knowledge (Frishkoff, Perfetti, & Westbury, 2009; Henderson, Weighall, & Gaskell, 2013; Hendrickson, Mitsven, Poulin-Dubois, Zesiger, & Friend, 2015; Ince & Christman, 2002; McClelland & Elman, 1986; McMurray, 2007; Schwanenflugel, Stahl, & McFalls, 1997; Shore & Durso, 1991; Steele, 2012; Stein & Shore, 2012; Suanda, Mugwanya, & Namy, 2014; Whitmore, Shore, & Smith, 2004; Zareva, 2012). In contrast to theories that suggest that word learning is all-or-none (Gallistel, Fairhurst, & Balsam, 2004; Trueswell et al, 2013), incremental learning theories of word comprehension rely on the assumption that word knowledge is incremental and unfolds over time (McMurray, Horst, & Samuelson, 2012; Rogers & McClelland, 2004; Siskind, 1996; Spivey et al, 2010; Yu, 2008; Yu & Smith, 2007; Yurovsky, Fricker, Yu, & Smith, 2014).…”

“…Behavioral evidence of such a phenomenon has recently been observed through dissociations of visual (looking) and haptic (touching) responses during a word comprehension task (Gurteen, Horne, & Erjavec, 2011; Hendrickson et al, 2015). Using a forced-choice paradigm, a moment-by-moment analysis of looking and touching behaviors—measures of word processing and comprehension, respectively—was conducted to assess the speed with which a prompted word was processed (visual reaction time) as a function of haptic response: target touch (touched the picture of the word referent), distractor touch (touched the picture of the unprompted word), or no touch (failed to touch either image).…”

“…The first aim (Aim 1) was to examine developmental changes in the speed and accuracy of word recognition across these three measures in the same cohort of children throughout the second year of life. The second aim (Aim 2) was to replicate and extend the finding of visual and haptic behavioral dissociations and corresponding partial knowledge states across the second year (Hendrickson et al, 2015). Finally, the third aim (Aim 3) was to investigate the role that partial knowledge plays in early word comprehension over this same time period.…”