Computational Models for Spatial Prepositions

Abstract

Developing computational models of spatial prepositions (such as on, in, above, etc.) is crucial for such tasks as human-machine collaboration, story understanding, and 3D model generation from descriptions. However, these prepositions are notoriously vague and ambiguous, with meanings depending on the types, shapes and sizes of entities in the argument positions, the physical and task context, and other factors. As a result truth value judgments for prepositional relations are often uncertain and variable. In this paper we treat the modeling task as calling for assignment of probabilities to such relations as a function of multiple factors, where such probabilities can be viewed as estimates of whether humans would judge the relations to hold in given circumstances. We implemented our models in a 3D blocks world and a room world in a computer graphics setting, and found that true/false judgments based on these models do not differ much more from human judgments that the latter differ from one another. However, what really matters pragmatically is not the accuracy of truth value judgments but whether, for instance, the computer models suffice for identifying objects described in terms of prepositional relations, (e.g., the box to the left of the table, where there are multiple boxes). For such tasks, our models achieved accuracies above 90% for most relations.

A Linguistic Analysis of Visually Grounded Dialogues Based on Spatial Expressions

“…In computer vision, the VisualGenome dataset (Krishna et al, 2017) provides rich annotation of spatial scene graphs constructed from raw images, but not from raw dialogues. Ramisa et al (2015); Platonov and Schubert (2018) also worked on modelling spa-tial prepositions in single sentences. To the best of our knowledge, our work is the first to apply, model and analyze spatial expressions in visually grounded dialogues at full scale.…”

“…According to Landau (2017), there are 2 classes of relations in spatial language: functional class whose core meanings engage force-dynamic relationship (such as on, in) and geometric class whose core meanings engage geometry (such as left, above). Since functional relations are less common in this dataset and more difficult to define due to their vagueness and context dependence (Platonov and Schubert, 2018), we focus on the following 5 categories of geometric relations and attribute comparisons, including a total of 24 canonical relations which can be defined explicitly.…”

Fundamental laws and principles in geoinformation science

“…There are theories concerning the relations of the partial and total order of spatial objects as described by prepositions such as in front of, behind, above, and below (e.g. [8] ). Geoinformation science is also an applied science, which means that many of its theoretical foundations are originally developed in geography, ecology, statistics, demography, operations research, sociology, mathematics, computer science or in other fields.…”

A Cognitively Motivated Approach to Spatial Information Extraction

“…Spranger et al, 2016) as well as spatial role labeling (Kordjamshidi et al, 2011). At times, works focus on one aspect of spatial semantic analysis only, such as prepositions (Platonov and Schubert, 2018) and spatial frames and their relations (Ulinski et al, 2019).…”

“…However, most of the solutions focus on specific spatial phenomena suffering from a lack of generalizability (e.g. Platonov and Schubert, 2018;Ulinski et al, 2019), or are generalizable machine learning approaches (e.g. Kordjamshidi et al, 2011), but lack explainability of provided decisions.…”