A Spectral Algorithm for Latent Dirichlet Allocation

Anandkumar, Anima; Liu, Yikai; Hsu, Daniel; Foster, Dean P.; Kakade, Sham M.

doi:10.1007/s00453-014-9909-1

Cited by 121 publications

(242 citation statements)

References 47 publications

Supporting

Mentioning

239

Contrasting

Order By: Relevance

“…The first two are proved in Anandkumar et al [6], and the third can be proved by a similar proof of Theorem 4.1 in Anandkumar et al [7]. We omit the details here.…”

Section: Algorithm 3: Tensor Orthogonal Decomposition (Tod)mentioning

confidence: 71%

“…This gives a higher value to a phrase with scores of (3,3,3) than to one with scores of (1,3,5), though the average score is identical. Finally, IdealScore K is calculated using the scores of the top K phrases out of all judged phrases.…”

Section: User Study and Quantitative Resultsmentioning

confidence: 99%

“…By using this dataset, which roughly repesents a subtopic of the full DBLP 5 The one exception is venues, as there are only 20 venues in the DBLP dataset, so we set K = 3 in this case.…”

Section: Methods For Comparisonmentioning

confidence: 99%

“…Recently, an alternative inference of topic models has been proposed based on the method of moments [5], and improved in [6]. Compared with ML inference, it has the following two advantages: i) the distance between inferred corpus-level parameters and the true parameters has a theoretical upper bound that is inversely related to sample size; ii) the convergence is guaranteed with a bounded number of iterations.…”

Section: Traditional Topic Modelingmentioning

confidence: 99%

“…Its efficiency is improved in [9]. This method requires stronger assumptions than [5] and the error bound is weaker.…”

Section: Traditional Topic Modelingmentioning

confidence: 99%

See 4 more Smart Citations

Mining latent entity structures from massive unstructured and interconnected data

Wang

2014

Proceedings of the 2014 ACM SIGMOD International Conference on Management of Data

View full text Add to dashboard Cite

The "big data" era is characterized by an explosion of information in the form of digital data collections, ranging from scientific knowledge, to social media, news, and everyone's daily life. Valuable knowledge about multi-typed entities is often hidden in the unstructured or loosely structured but interconnected data. Mining latent structured information around entities uncovers semantic structures from massive unstructured data and hence enables many high-impact applications, including taxonomy or knowledge base construction, multi-dimensional data analysis and information or social network analysis.A mining framework is proposed, to solve and integrate a chain of tasks: hierarchical topic discovery, topical phrase mining, entity role analysis and entity relation mining. It reveals two main forms of structures: topical and relational structures. The topical structure summarizes the topics associated with entities with various granularity, such as the research areas in computer science. The framework enables recursive construction of phrase-represented and entity-enriched topic hierarchy from text-attached information networks. It makes breakthrough in terms of quality and computational efficiency. The relational structure recovers the hidden relationship among entities, such as advisor-advisee. A probabilistic graphical modeling approach is proposed. The method can utilize heterogeneous attributes and links to capture all kinds of semantic signals, including constraints and dependencies, to recover the hierarchical relationship with the best known accuracy.ii

show abstract

“…The first two are proved in Anandkumar et al [6], and the third can be proved by a similar proof of Theorem 4.1 in Anandkumar et al [7]. We omit the details here.…”

Section: Algorithm 3: Tensor Orthogonal Decomposition (Tod)mentioning

confidence: 71%

Section: User Study and Quantitative Resultsmentioning

confidence: 99%

“…By using this dataset, which roughly repesents a subtopic of the full DBLP 5 The one exception is venues, as there are only 20 venues in the DBLP dataset, so we set K = 3 in this case.…”

Section: Methods For Comparisonmentioning

confidence: 99%

Section: Traditional Topic Modelingmentioning

confidence: 99%

“…Its efficiency is improved in [9]. This method requires stronger assumptions than [5] and the error bound is weaker.…”

Section: Traditional Topic Modelingmentioning

confidence: 99%

See 3 more Smart Citations

Mining latent entity structures from massive unstructured and interconnected data

Wang

2014

Proceedings of the 2014 ACM SIGMOD International Conference on Management of Data

View full text Add to dashboard Cite

show abstract

Document representation methods for clustering bilingual documents

Zhang

2016

Proc. Assoc. Info. Sci. Tech.

View full text Add to dashboard Cite

Globalization places people in a multilingual environment. There is a growing number of users to access and share information in several languages for public or private purpose. In order to deliver relevant information in different languages, efficient multilingual documents management is worthy of study. Generally, classification and clustering are two typical methods for documents management. However, lack of training data and high efforts for corpus annotation will increase the cost for classifying multilingual documents which needs to bridge language gaps as well. Clustering is more suitable to implement in such practical applications. There are two main factors involved in documents clustering, document representation method and clustering algorithm. In this paper, we focus on document representation method and demonstrate that the choice of representation methods has impacts on quality of clustering results. In our experiment, we use parallel corpora (English-Chinese documents on topic of technology information) and comparable corpora (English and Chinese documents on topics of mobile technology and wind energy) as dataset. We compare four different types of document representation methods: Vector Space Model, Latent Semantic Indexing, Latent Dirichlet Allocation and Doc2Vec. Experimental results show that, accuracy of Vector Space Model were not competitive with other methods in all clustering tasks. Latent Semantic Indexing is overly sensitive to corpora itself, for it behaved differently when clustering two different topics of

show abstract

$$\mathtt{SpectralLeader}$$: Online Spectral Learning for Single Topic Models

Kveton

Wen

et al. 2019

Machine Learning and Knowledge Discovery in Databases

View full text Add to dashboard Cite

We study the problem of learning a latent variable model from a stream of data. Latent variable models are popular in practice because they can explain observed data in terms of unobserved concepts. These models have been traditionally studied in the offline setting. In the online setting, on the other hand, the online EM is arguably the most popular algorithm for learning latent variable models. Although the online EM is computationally efficient, it typically converges to a local optimum. In this work, we develop a new online learning algorithm for latent variable models, which we call SpectralLeader. SpectralLeader always converges to the global optimum, and we derive a sublinear upper bound on its n-step regret in the bag-of-words model. In both synthetic and real-world experiments, we show that SpectralLeader performs similarly to or better than the online EM with tuned hyper-parameters.

show abstract

A Spectral Algorithm for Latent Dirichlet Allocation

Cited by 121 publications

References 47 publications

Mining latent entity structures from massive unstructured and interconnected data

Mining latent entity structures from massive unstructured and interconnected data

Document representation methods for clustering bilingual documents

$$\mathtt{SpectralLeader}$$: Online Spectral Learning for Single Topic Models

Contact Info

Product

Resources

About