SpaceRefNet: a neural approach to spatial reference resolution in a real city environment

Abstract

Adding interactive capabilities to pedestrian wayfinding systems in the form of spoken dialogue will make them more natural to humans. Such an interactive wayfinding system needs to continuously understand and interpret pedestrian's utterances referring to the spatial context. Achieving this requires the system to identify exophoric referring expressions in the utterances, and link these expressions to the geographic entities in the vicinity. This exophoric spatial reference resolution problem is difficult, as there are often several dozens of candidate referents. We present a neural network-based approach for identifying pedestrian's references (using a network called RefNet) and resolving them to appropriate geographic objects (using a network called Space-RefNet). Both methods show promising results beating the respective baselines and earlier reported results in the literature.

Decision-Theoretic Question Generation for Situated Reference Resolution: An Empirical Study and Computational Model

“…As a result, these approaches tend to struggle with open-world interaction. Neural approaches have recently been developed for situated, multi-modal dialogue [20,22,24,33,39]. While promising, these approaches typically require large, labeled training sets and are generally limited to the specific domains in which they were trained.…”

How Should Agents Ask Questions For Situated Learning? An Annotated Dialogue Corpus

“…Situated interaction is an area of interest to the Dialogue Systems community (Bohus, 2019), with recent papers investigating aspects of language interaction in situated environments both empirically and computationally Gupta et al, 2019;Kalpakchi and Boye, 2019;Kleingarn et al, 2019). This topic is critical for the development of technologies that interact with humans in real and virtual environments, including automated vehicles, smart home appliances, robots, and others.…”