From paragraph to graph: Latent semantic analysis for information visualization

Abstract

Most techniques for relating textual information rely on intellectually created links such as author-chosen keywords and titles, authority indexing terms, or bibliographic citations. Similarity of the semantic content of whole documents, rather than just titles, abstracts, or overlap of keywords, offers an attractive alternative. Latent semantic analysis provides an effective dimension reduction method for the purpose that reflects synonymy and the sense of arbitrary word combinations. However, latent semantic analysis correlations with human text-to-text similarity judgments are often empirically highest at Ϸ300 dimensions. Thus, two-or threedimensional visualizations are severely limited in what they can show, and the first and͞or second automatically discovered principal component, or any three such for that matter, rarely capture all of the relations that might be of interest. It is our conjecture that linguistic meaning is intrinsically and irreducibly very high dimensional. Thus, some method to explore a high dimensional similarity space is needed. But the 2.7 ؋ 10 7 projections and infinite rotations of, for example, a 300-dimensional pattern are impossible to examine. We suggest, however, that the use of a high dimensional dynamic viewer with an effective projection pursuit routine and user control, coupled with the exquisite abilities of the human visual system to extract information about objects and from moving patterns, can often succeed in discovering multiple revealing views that are missed by current computational algorithms. We show some examples of the use of latent semantic analysis to support such visualizations and offer views on future needs. M ost techniques for relating textual information rely on intellectually created links such as author-chosen keywords and titles, authority indexing terms, or bibliographic citations (1). Similarity of the semantic content of whole documents, rather than just titles, abstracts, or an overlap of keywords, offers an attractive alternative. Latent semantic analysis (LSA) provides an effective dimension reduction method for the purpose that reflects synonymy and the sense of arbitrary word combinations (2, 3). Latent Semantic AnalysisLSA is one of a growing number of corpus-based techniques that employ statistical machine learning in text analysis. Other techniques include the generative models of Griffiths and Steyvers (4) and Erosheva et al. (5), and the string-edit-based method of S. Dennis (6) and several new computational realizations of LSA. Unfortunately, to date none of the other methods scales to text databases of the size often desired for visualization of domain knowledge. The linear singular value decomposition (SVD) technique described here has been applied to collections of as many as a half billion documents containing 750,000 unique word types, all of which are used in measuring the similarity of two documents. LSA presumes that the overall semantic content of a passage, such as a paragraph, abstract, or full coherent document, can be useful...

CUR matrix decompositions for improved data analysis

“…Internet term-document data are a common application of SVD-based techniques, often via latent semantic analysis (24,25). The Open Directory Project (ODP) (26) is a multilingual open content directory of World Wide Web links.…”

Storylines: Visual exploration and analysis in latent semantic spaces

“…The frequency of latent dimensions identified by LSI follows the Zipf-distribution [4]. Dimensions with smaller contributions can be safely omitted.…”

“…Landauer [4] describes a linear SVD technique and applies it to a collection of a half billion documents containing 750 000 unique word types. LSA presumes that the overall semantic content of a document can be approximated as a sum of the meaning of its words.…”

Latent Semantic Analysis: five methodological recommendations

“…Many quantitative literature overview studies use LSA for document clustering purposes (Landauer et al 2004, Ord et al 2005, while some studies (Sidorova et al 2008) use factor analysis and others (Larsen et al 2008) use both clustering and factor analysis. In this section we discuss differences and similarities between clustering and factor analysis extensions to LSA.…”

“…(1) quantitative literature reviews (as done in Landauer et al 2004, Ord et al 2005, Larsen et al 2008, Sidorova et al 2008, or Hovorka et al 2009 (Salton 1975), where a corpus of d documents using a vocabulary of t terms is used to compile a t×d matrix A, containing the number of times each term appears in each document (term frequencies). Some trivial terms such as "the", "of", etc.…”