Sparse Word Graphs: A Scalable Algorithm for Capturing Word Correlations in Topic Models

Habich, Dirk; Volk, Peter Benjamin; Lehner, Wolfgang; Dittmann, Ralf W.; Utzny, Clemens

doi:10.1109/icdmw.2007.39

Cited by 7 publications

(5 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…With respect to topic models estimated with bigrams [MacKay and Peto, 1995, Wallach, 2006, Wang et al, 2007, Yan et al, 2013, Nallapati et al, 2007, one of the advantages of LDA2Net is that its results do not depend on any assumption on the distribution family and data generation process of bigrams, being LDA2Net based on the observed frequencies of bigrams in documents, which are combined (a-posteriori) with LDA output matrices, as shown in Figure 1.…”

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

LDA2Net: Digging under the surface of COVID-19 topics in scientific literature

Minello,

Santagiustina,

Warglien

2021

Preprint

View full text Add to dashboard Cite

During the COVID-19 pandemic, the scientific literature related to SARS-COV-2 has been growing dramatically, both in terms of the number of publications and of its impact on people's life. This literature encompasses a varied set of sensible topics, ranging from vaccination, to protective equipment efficacy, to lockdown policy evaluation. Up to now, hundreds of thousands of papers have been uploaded on online repositories and published in scientific journals. As a result, the development of digital methods that allow an in-depth exploration of this growing literature has become a relevant issue, both to identify the topical trends of COVID-related research and to zoom-in its sub-themes. This work proposes a novel methodology, called LDA2Net, which combines topic modelling and network analysis to investigate topics under their surface. Specifically, LDA2Net exploits the frequencies of pairs of consecutive words to reconstruct the network structure of topics discussed in the Cord-19 corpus. The results suggest that the effectiveness of topic models can be magnified by enriching them with word network representations, and by using the latter to display, analyse, and explore COVID-related topics at different levels of granularity.

show abstract

Section: Discussionmentioning

confidence: 99%

“…Attempts to include and model syntagmatic information (i.e. information concerning sequential relations between words) in topic models have already been investigated in MacKay and Peto [1995], Wallach [2006], Wang et al [2007], Yan et al [2013], Nallapati et al [2007].…”

Section: Introductionmentioning

confidence: 99%

LDA2Net: Digging under the surface of COVID-19 topics in scientific literature

Minello,

Santagiustina,

Warglien

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Attempts to include and model syntagmatic information (i.e. information concerning sequential relations between words) in topic models have already been investigated in [11][12][13][14][15] as well. Moreover, using a Dirichlet distribution implicitly assumes that few items overwhelmingly contribute to each topic.…”

Section: Related Workmentioning

confidence: 99%

“…Concerning topic models estimated with bigrams [11][12][13][14][15], one of the advantages of LDA2Net is that its results do not depend on any assumption on the distribution family and data generation process of bigrams, being LDA2Net based on the observed frequencies of bigrams in documents, which are combined (a-posteriori) with LDA output matrices, as shown in Fig 1.…”

Section: Related Workmentioning

confidence: 99%

LDA2Net Digging under the surface of COVID-19 scientific literature topics via a network-based approach

Minello,

Santagiustina,

Warglien

2024

PLoS ONE

View full text Add to dashboard Cite

During the COVID-19 pandemic, the scientific literature related to SARS-COV-2 has been growing dramatically. These literary items encompass a varied set of topics, ranging from vaccination to protective equipment efficacy as well as lockdown policy evaluations. As a result, the development of automatic methods that allow an in-depth exploration of this growing literature has become a relevant issue, both to identify the topical trends of COVID-related research and to zoom-in on its sub-themes. This work proposes a novel methodology, called LDA2Net, which combines topic modelling and network analysis, to investigate topics under their surface. More specifically, LDA2Net exploits the frequencies of consecutive words pairs (i.e. bigram) to build those network structures underlying the hidden topics extracted from large volumes of text by Latent Dirichlet Allocation (LDA). Results are promising and suggest that the topic model efficacy is magnified by the network-based representation. In particular, such enrichment is noticeable when it comes to displaying and exploring the topics at different levels of granularity.

show abstract

“…In any language, there exists a semantic structure among words, which leads to the correlation of words in constituting the meaning in an expression, and more specifically, the correlation of their roles in a natural function β. This structure has been studied as the semantic correlation of words [17,29,16] in a machine learning context. Given a correlation structure on model coefficients, assuming model sparsity and imposing an ℓ1-norm penalty are no longer appropriate.…”

Section: Model Compression: Correlationmentioning

confidence: 99%

Learning Compressible Models

Zhang¹,

Schneider²,

Dubrawski³

2010

Proceedings of the 2010 SIAM International Conference on Data Mining

View full text Add to dashboard Cite

In this paper, we study the combination of compression and ℓ1-norm regularization in a machine learning context: learning compressible models. By including a compression operation into the ℓ1 regularization, the assumption on model sparsity is relaxed to compressibility: model coefficients are compressed before being penalized, and sparsity is achieved in a compressed domain rather than the original space. We focus on the design of different compression operations, by which we can encode various compressibility assumptions and inductive biases, e.g., piecewise local smoothness, compacted energy in the frequency domain, and semantic correlation. We show that use of a compression operation provides an opportunity to leverage auxiliary information from various sources, e.g., domain knowledge, coding theories, unlabeled data. We conduct extensive experiments on brain-computer interfacing, handwritten character recognition and text classification. Empirical results show clear improvements in prediction performance by including compression in ℓ1 regularization. We also analyze the learned model coefficients under appropriate compressibility assumptions, which further demonstrate the advantages of learning compressible models instead of sparse models.

show abstract

Sparse Word Graphs: A Scalable Algorithm for Capturing Word Correlations in Topic Models

Abstract: Statistical topic models such as the Latent Dirichlet Allocation (LDA)

Cited by 7 publications

References 13 publications

LDA2Net: Digging under the surface of COVID-19 topics in scientific literature

LDA2Net: Digging under the surface of COVID-19 topics in scientific literature

LDA2Net Digging under the surface of COVID-19 scientific literature topics via a network-based approach

Learning Compressible Models

Contact Info

Product

Resources

About