Social centralization and semantic collapse: Hyperbolic embeddings of networks and text

Li, Linzhuo; Wu, Lingfei; Evans, James

doi:10.1016/j.poetic.2019.101428

Cited by 13 publications

(13 citation statements)

References 67 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…where 𝒗 and 𝒖 are the "in-vector" and "out-vector", respectively, 𝑍 𝑖 = 𝑗 ′ ∈A exp(𝒖 𝑗 ′ • 𝒗 𝑖 ) is a normalization constant, and A is the set of all locations. We follow the standard practice and only use the in-vector, 𝒗, which is known to be superior to the out-vector in link prediction benchmarks [20][21][22][23][24][25]28 .…”

Section: Embeddingmentioning

confidence: 99%

“…Here, we study the skip-gram negative sampling (SGNS), or word2vec neural-network architecture (see Methods). This neural embedding model, originally designed for learning models of language 19 , has been making breakthroughs by revealing novel insights into texts [20][21][22][23][24][25] , networks [26][27][28] and trajectories [29][30][31][32][33][34] . It works under the notion that a good representation should facilitate prediction, learning a mapping between words that can predict a target word based on its context (surrounding words).…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Unsupervised Embedding of Trajectories Captures the Latent Structure of Mobility

Murray¹,

Yoon²,

Kojaku³

et al. 2021

Preprint

View full text Add to dashboard Cite

Human mobility drives major societal phenomena including epidemics, economies, and innovation. Historically, mobility was constrained by geographic distance, however, in the globalizing world, language, culture, and history are increasingly important. Here, we show a mathematical equivalence between word2vec model and the gravity model of mobility and demonstrate that, by using three human trajectory datasets, word2vec encodes nuanced relationships between locations into a systematic and meaningful vector-space, providing a functional distance between locations, as well as a representation for studying the many dimensions of mobility. Focusing on the case of scientiﬁc mobility, we show that embeddings implicitly learn cultural, linguistic, and hierarchical relationships at multiple levels of granularity. Connecting neural embeddings to the gravity model opens up new avenues for the study of mobility.

show abstract

Section: Embeddingmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Unsupervised Embedding of Trajectories Captures the Latent Structure of Mobility

Murray¹,

Yoon²,

Kojaku³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…The meaning of a word is a fundamental object of study in computational linguistics, where researchers have in particular focused on the disambiguation and identification of lexical semantic change (e.g., Schlechtweg, Hätty, et al 2019;Tahmasebi, Borin, and Jatowt 2019) and word senses (e.g., Amrami and Goldberg 2019;Başkaya et al 2013;Daille et al 2016). Similarly, changes in meaning structures have been examined in the field of computational sociology (e.g., Kozlowski, Taddy, and Evans 2020;Linzhuo, Lingfei, and James 2020;Padgett et al 2020;Rule, Cointet, and Bearman 2015;Schoots et al 2020), although these latter studies are not based on deep language models and therefore do not take context or polysemy into account.…”

Section: Related Work and Contributionmentioning

confidence: 99%

“…Projecting these relationships into Euclidean spaces may imply a loss of information if semantic ties do not follow the triangle inequality. Similarly, hyperbolic spaces, while appropriate for tree-like syntactic relations, are also not be sufficient to represent semantic relationships, where triads of tokens need not be closed in terms of similarity (Chami et al 2020;Linzhuo, Lingfei, and James 2020;Nickel and Kiela 2018).…”

Section: A1 Challenges In the Application Of Deep Language Modelsmentioning

confidence: 99%

Text analysis and deep learning: A network approach

Marquart¹

2021

Preprint

View full text Add to dashboard Cite

Much information available to applied researchers is contained within written language or spoken text. Deep language models such as BERT have achieved unprecedented success in many applications of computational linguistics. However, much less is known about how these models can be used to analyze existing text. We propose a novel method that combines transformer models with network analysis to form a self-referential representation of language use within a corpus of interest. Our approach produces linguistic relations strongly consistent with the underlying model as well as mathematically well-defined operations on them, while reducing the amount of discretionary choices of representation and distance measures. It represents, to the best of our knowledge, the first unsupervised method to extract semantic networks directly from deep language models. We illustrate our approach in a semantic analysis of the term "founder". Using the entire corpus of Harvard Business Review from 1980 to 2020, we find that ties in our network track the semantics of discourse over time, and across contexts, identifying and relating clusters of semantic and syntactic relations. Finally, we discuss how this method can also complement and inform analyses of the behavior of deep learning models.

show abstract

“…A vector-space embedding of locations (airports, accommodations, and organizations) is learned by using trajectories as input to the standard with skip-gram negative sampling, or word2vec neural-network architecture (see Methods). This neural embedding model, originally designed for learning language models [19], has been making breakthroughs by revealing novel insights into texts [20][21][22][23][24][25] and networks [26,27]. The model is also computationally efficient, robust to noise, and can encode relations between entities as geometric relationships in the vector space [22,25,28,29].…”

Section: Introductionmentioning

confidence: 99%

Unsupervised embedding of trajectories captures the latent structure of mobility

Murray¹,

Yoon²,

Kojaku³

et al. 2020

Preprint

View full text Add to dashboard Cite

Human mobility and migration drive major societal phenomena such as the growth and evolution of cities, epidemics, economies, and innovation. Historically, human mobility has been strongly constrained by physical separation-geographic distance. However, geographic distance is becoming less relevant in the increasingly-globalized world in which physical barriers are shrinking while linguistic, cultural, and historical relationships are becoming more important. As understanding mobility is becoming critical for contemporary society, finding frameworks that can capture this complexity is of paramount importance. Here, using three distinct human trajectory datasets, we demonstrate that a neural embedding model can encode nuanced relationships between locations into a vector-space, providing an effective measure of distance that reflects the multi-faceted structure of human mobility. Focusing on the case of scientific mobility, we show that embeddings of scientific organizations uncover cultural and linguistic relations, and even academic prestige, at multiple levels of granularity. Furthermore, the embedding vectors reveal universal relationships between organizational characteristics and their place in the global landscape of scientific mobility. The ability to learn scalable, dense, and meaningful representations of mobility directly from the data can open up a new avenue of studying mobility across domains.

show abstract

Social centralization and semantic collapse: Hyperbolic embeddings of networks and text

Cited by 13 publications

References 67 publications

Unsupervised Embedding of Trajectories Captures the Latent Structure of Mobility

Unsupervised Embedding of Trajectories Captures the Latent Structure of Mobility

Text analysis and deep learning: A network approach

Unsupervised embedding of trajectories captures the latent structure of mobility

Contact Info

Product

Resources

About