Spatial Natural Language Generation for Location Description in Photo Captions

Abstract

Abstract. We present a spatial natural language generation system to create captions that describe the geographical context of geo-referenced photos. An analysis of existing photo captions was used to design templates representing typical caption language patterns, while the results of human subject experiments were used to create field-based spatial models of the applicability of some commonly used spatial prepositions. The language templates are instantiated with geo-data retrieved from the vicinity of the photo locations. A human subject evaluation was used to validate and to improve the spatial language generation procedure, examples of the results of which are presented in the paper.

Mining nearness relations from an n-grams Web corpus in geographical space

“…Other possible exploitations of our approach include selection of more likely referents from candidate toponyms pairs in producing realistic natural language phrases, using a population-based model (c.f. Hall, Jones, & Smart, 2015). It is, however, important to note that all of these proposals would require implementation and, more importantly, evaluation, to demonstrate their efficacy.…”

Using Spatial Image Schemata in the characterization of spatial relations

“…Spatial descriptions are known as locative or locational expressions. For Hall et al (2015) spatial descriptions are composed of entities which report the spatial position of objects or persons in relation to another reference element. This research presents the results of a survey performed with users in order to identify the spatial relations used in spatial descriptions of unknown environments.…”

“…In the process of communicating through spoken or written Natural Language (NL), places or events are described using expressions such as "the drugstore is in front of the gas station". This type of description is the predominant way of communication among human beings and it is known as locative or locational expressions (Herskovits 1985;Hall et al 2015;Kim et al 2017). Herskovits (1985) describes locative or locational expression as any spatial expression composed of a sentence with a spatial term and its object.…”

“…In the area of geoinformation, spatial relations play a relevant role in spatial data modelling, spatial query and analysis. Regarding the activities of spatial cognition, spatial relations assist in the spatial reasoning and in the understanding of the maps (Hall and Jones 2008;Zhang et al 2013;Hall et al 2015). Realizing the relevance of spatial relations, researches on spatial relationships mainly aim to understand how people use them in their daily lives by performing spatial descriptions.…”

The Use of Spatial Terms “Near”, “Very Near, “Next To”, “Side by Side and “Nearby” in the Descriptions of Spatial Configurations

“…When describing the strategies needed to reach a specific location or report the spatial configuration of an environment, the individual performs the spatial description task. The purpose of these descriptions is to identify objects or events in a specific environment or to provide information from a specific, unknown region to the user receiving the information (Hall et al 2015;Stock 2010;Herskovits 1985). Such descriptions are commonplace in people's daily life and occur naturally, since in dealing with abstract notions human beings choose to reflect using spatial concepts -as a consequence of their ability to reason spatially.…”