A map of words: Retrieving the spatial layout of underground stations from natural language

Abstract

Recent evidence has indicated that spatial representations, such as large-scale geographical maps, can be retrieved from natural language alone through cognitively plausible distributional-semantic models based on non-spatial associative-learning mechanisms. Here, we demonstrate that analogous spatial maps can be extracted from purely linguistic data even at the medium-scale level. Our results indeed show that it is possible to retrieve the underground maps of five European cities from linguistic data, suggesting in turn that the ability to reconstruct spatial maps from language does not strictly depend on the scale being mapped. Furthermore, we show that different spatial representations (i.e., with information encoded either in terms of relative spatial distances or absolute locations defined by coordinate axes) can be retrieved from natural language. These findings contribute to a growing body of research that challenges the traditional view of cognitive maps as exclusively relying on specialized spatial computations and highlight the importance of non-spatial associative-learning mechanisms within the linguistic environment in the setting of spatial representations.

Proceedings of the 3rd Workshop on Multi-lingual Representation Learning (MRL)

“…gender, racial, ethnic, occupational) identification and mitigation approaches in PLMs including, reducing token sensitivity during text generation (Liang et al, 2021), investigating model sensitivity (Immer et al, 2022), prompting using natural sentences (Alnegheimish et al, 2022) and probing via embedding lookup (Ahn and Oh, 2021). On the other hand, representing space and time utilizing maps and language is a long-standing domain of research (Louwerse and Benesh, 2012; Gatti et al, 2022;Anceresi et al, 2023) 2022) refer to these model weights as expert units. Now, we can prioritize these identified expert units during text generation by artificially simulating the presence of the concept word "doctor" in the input.…”

Geographic and Geopolitical Biases of Language Models

“…gender, racial, ethnic, occupational) identification and mitigation approaches in PLMs including, reducing token sensitivity during text generation (Liang et al, 2021), investigating model sensitivity (Immer et al, 2022), prompting using natural sentences (Alnegheimish et al, 2022) and probing via embedding lookup (Ahn and Oh, 2021). On the other hand, representing space and time utilizing maps and language is a long-standing domain of research (Louwerse and Benesh, 2012;Gatti et al, 2022;Anceresi et al, 2023). More recently, numerous studies are experimenting with geoadaptation of PLMs (Hofmann et al, 2023), what behavior these PLMs exhibit while probing with geographic-context, cultural-commonsense as well as temporal reasoning (Yin et al, 2022;Ghosh et al, 2021;Thapliyal et al, 2022;Hlavnova and Ruder, 2023;Shwartz, 2022;Tan et al, 2023) or how large PLMs learn the representation of space and time (Gurnee and Tegmark, 2023).…”

Decomposing geographical judgments into spatial, temporal and linguistic components

“…These models serve as a valuable proxy to quantify the role of linguistic experience in shaping knowledge, as they capture meanings from statistical patterns of word distributions in natural language (Lenci, 2008), without directly accessing nor computing any spatial relationships. Interestingly, there is evidence that these models can capture the layout of medium spatial environments (Anceresi et al, 2023) as well as humans' behavior in the geographical domains (Gatti et al, 2022) and beyond, like in the case of the human body (Gatti et al, 2023).…”