Getting the gist of events: Recognition of two-participant actions from brief displays.

Abstract

Unlike rapid scene and object recognition from brief displays, little is known about recognition of event categories and event roles from minimal visual information. In three experiments, we displayed naturalistic photographs of a wide range of two-participant event scenes for 37 ms and 73 ms followed by a mask, and found that event categories (the event gist, e.g., ‘kicking’, ‘pushing’, etc.) and event roles (i.e., Agent and Patient) can be recognized rapidly, even with various actor pairs and backgrounds. Norming ratings from a subsequent experiment revealed that certain physical features (e.g., outstretched extremities) that correlate with Agent-hood could have contributed to rapid role recognition. In a final experiment, using identical twin actors, we then varied these features in two sets of stimuli, in which Patients had Agent-like features or not. Subjects recognized the roles of event participants less accurately when Patients possessed Agent-like features, with this difference being eliminated with two-second durations. Thus, given minimal visual input, typical Agent-like physical features are used in role recognition but, with sufficient input from multiple fixations, people categorically determine the relationship between event participants.

Priming sentence planning

“…The implication of this result for theories of formulation is that accessibility effects are modulated by higher-level, relational information: in order to prioritize encoding of the agent, speakers first had to identify the two characters in terms of their event roles. The degree to which identification of agents requires extensive encoding of event gist is debatable because of lower-level perceptual correlates of ''agenthood'' (Hafri et al, 2012); nevertheless, selection of starting points appears to be sensitive to a combination of non-relational and relational variables.…”

“…Hafri et al (2012) recently showed that speakers can extract basic information about event structure in less than 100 ms from perceptual features of individual characters that are typically associated with ''agenthood'' (also see Bock et al, 2003;Dobel et al, 2007;Potter, 1976). Given the speed with which speakers can link visual information to event categories, the two experiments in this paper suggest that processing occurring within the first 400 ms of picture onset must be a multi-faceted process.…”

“…Replicating Kuchinsky and Bock (2010), speakers should begin their sentences with first-fixated characters or easy-to-name characters less often in higher-codability than lower-codability events. Early eye movements should also show sensitivity to higher-level event properties (see Bock et al, 2003;Dobel, Gumnior, Bolte, & Zwitserlood, 2007;Hafri, Papafragou, & Trueswell, 2012, for demonstrations of rapid encoding of event gist). Speakers should be less likely to prioritize encoding of one character over the other character in the first 400 ms of picture inspection in higher-codability events than in lower-codability events; instead, they should direct their gaze preferentially to the subject character later in higher-codability events, resulting in slower divergence of fixations to the two characters in higher-than lower-codability events immediately after picture onset.…”

Vision and Language in Cross-Linguistic Research on Sentence Production

“…This suggests that, very rapidly after picture onset, speakers are able to generate a rudimentary structural plan and this plan can then guide subsequent conceptual and linguistic encoding operations. Supporting the conclusion of fast message-level encoding, a number of recent studies have demonstrated that very brief presentations (40-300 ms) of event pictures are in fact sufficient for speakers to identify event categories, as well as the role and identity of characters in the event (Dobel et al 2007;Hafri et al 2013). Identification of such information is presumably sufficient to allow for the generation of a conceptual and linguistic structural frame, which can direct the eye to efficiently sample information from the scene as the structure calls for it.…”

Processes of incremental message planning during conversation

“…However, these studies typically do not pinpoint the level of representation (message or sentence level) at which this incremental preparation occurs. Since speakers can encode rudimentary conceptual information in a simple visual display within a few hundred milliseconds (e.g., Dobel et al, 2007;Hafri et al, 2013), a complete message may have been set before speech onset, with only linguistic encoding proceeding incrementally thereafter.…”