Perspective determines the production and interpretation of pointing gestures

Herbort, Oliver; Krause, Lisa-Marie; Kunde, Wilfried

doi:10.3758/s13423-020-01823-7

Cited by 7 publications

(4 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…That is, leftward biases were highest for points to the right and relatively low for leftward points. One possible explanation is that the perception of the respective azimuth is difficult to perceive, and this ambiguity fosters central tendencies (Herbort et al, 2021). Depending on the target region, the central tendency is differently impacting error sizes of interpretations, while the absolute interpretation error depends on the viewpoint and the general bias direction on only using the right arm for pointing gestures.…”

Section: Resultsmentioning

confidence: 99%

“…First, it is not consistent with all the data reported so far (Cooney et al, 2018). Second, it does not account for the finding that pointing gestures may be interpreted differently when seen from different viewpoints as indicated for systematic errors (Herbort et al, 2021; Herbort & Kunde, 2016; Mayer et al, 2020). However, neither a model of the systematic errors of pointing perception nor a detailed explanation of what these distortions are based on could be provided so far that integrates previous findings…”

mentioning

confidence: 97%

See 1 more Smart Citation

The observer’s perspective determines which cues are used when interpreting pointing gestures.

Krause

Herbort

2021

Journal of Experimental Psychology: Human Perception and Perfor

Self Cite

View full text Add to dashboard Cite

Though ubiquitous in human communication, pointing gestures are often misunderstood. This study addressed how the observer's perspective affects pointing perception. More specifically, we tested the hypothesis that two different visual cues-namely (a) the vector defined by the pointer's arm or finger and (b) the pointer's index finger position in the observer's visual field-determine pointing perception and that their relative influence depends on the observer's perspective. In three experiments, participants judged the location at which a virtual or real pointer was pointing from different viewpoints. The experiments show that the observer perspective has a considerable effect on pointing perception. The more the observer's gaze direction is aligned with the pointing arm, the more observers rely on the position of the pointing finger in their visual field and the less they rely on its direction. Public Significance StatementPointing gestures are an essential part of human communication, but observers often have difficulties to interpret the target correctly. In this study, we show that two different strategies for the interpretation are used, depending on the observer's perspective on the pointing gesture and how this affects the perception. When standing close to the pointer, people rely on the same strategy as pointers do-using the pointer's index finger as an indicator for the target location-but when seeing the pointing arm from a sideward viewpoint, it is mainly extrapolated.

show abstract

Section: Resultsmentioning

confidence: 99%

mentioning

confidence: 97%

The observer’s perspective determines which cues are used when interpreting pointing gestures.

Krause

Herbort

2021

Journal of Experimental Psychology: Human Perception and Perfor

Self Cite

View full text Add to dashboard Cite

show abstract

“…In Section 3.2, we discussed (conceptual) perspective cues as a form of positive context where similarities between co-referents are emphasized in the generation process, leading to descriptions that are coherent in terms of the conceptual perspectives for all targets in the referent set. Importantly, conceptual perspectives relate to alternative ways of categorizing objects and entities, as opposed to visual perspective in the sense of different viewpoints in three-dimensional scenarios (Herbort et al, 2021 ). For symbolic REG the choice of conceptual perspectives is investigated in Gatt and van Deemter ( 2006 , 2007 ).…”

Section: Toward a Wider Notion Of Context In Visual Regmentioning

confidence: 99%

Rethinking symbolic and visual context in Referring Expression Generation

Schüz¹,

Gatt²,

Zarrieß³

2023

Front. Artif. Intell.

View full text Add to dashboard Cite

Situational context is crucial for linguistic reference to visible objects, since the same description can refer unambiguously to an object in one context but be ambiguous or misleading in others. This also applies to Referring Expression Generation (REG), where the production of identifying descriptions is always dependent on a given context. Research in REG has long represented visual domains through symbolic information about objects and their properties, to determine identifying sets of target features during content determination. In recent years, research in visual REG has turned to neural modeling and recasted the REG task as an inherently multimodal problem, looking at more natural settings such as generating descriptions for objects in photographs. Characterizing the precise ways in which context influences generation is challenging in both paradigms, as context is notoriously lacking precise definitions and categorization. In multimodal settings, however, these problems are further exacerbated by the increased complexity and low-level representation of perceptual inputs. The main goal of this article is to provide a systematic review of the types and functions of visual context across various approaches to REG so far and to argue for integrating and extending different perspectives on visual context that currently co-exist in research on REG. By analyzing the ways in which symbolic REG integrates context in rule-based approaches, we derive a set of categories of contextual integration, including the distinction between positive and negative semantic forces exerted by context during reference generation. Using this as a framework, we show that so far existing work in visual REG has considered only some of the ways in which visual context can facilitate end-to-end reference generation. Connecting with preceding research in related areas, as possible directions for future research, we highlight some additional ways in which contextual integration can be incorporated into REG and other multimodal generation tasks.

show abstract

“…Hofemann et al [5] for instance use pointing gestures to instruct a robot which part to pick from a table, while pointing gestures were studied in a student-tutor relationship by Sathayanarayana et al [14]. Besides possible applications of detecting pointing gestures, the pointing accuracy was also intensively studied, e.g., in [1,4].…”

Section: Related Workmentioning

confidence: 99%

Pointing, Pairing and Grouping Gesture Recognition in Virtual Reality

Gorobets

Merkle²,

Kunz³

2022

Lecture Notes in Computer Science

View full text Add to dashboard Cite

During a team discussion, participants frequently perform pointing, pairing, or grouping gestures on artifacts on a whiteboard. While the content of the whiteboard is accessible to the blind and visually impaired people, the referring deictic gestures are not. This paper thus introduces an improved algorithm to detect such gestures and to classify them. Since deictic gestures such as pointing, pairing and grouping are performed by sighted users only, we used a VR environment for the development of the gesture recognition algorithm and for the subsequent user studies.

show abstract

Perspective determines the production and interpretation of pointing gestures

Cited by 7 publications

References 16 publications

The observer’s perspective determines which cues are used when interpreting pointing gestures.

The observer’s perspective determines which cues are used when interpreting pointing gestures.

Rethinking symbolic and visual context in Referring Expression Generation

Pointing, Pairing and Grouping Gesture Recognition in Virtual Reality

Contact Info

Product

Resources

About