Chunks of thought: Finding salient semantic structures in texts

Abstract

As the availability of large, digital text corpora increases, so does the need for automatic methods to analyze them and to extract significant information from them. A number of algorithms have been developed for these applications, with topic modeling-based algorithms such as latent Dirichlet allocation (LDA) enjoying much recent popularity. In this paper, we focus on a specific but important problem in text analysis: Identifying coherent lexical combinations that represent "chunks of thought" within the larger discourse. We term these salient semantic chunks (SSCs), and present two complimentary approaches for their extraction. Both these approaches derive from a cognitive rather than purely statistical perspective on the generation of texts. We apply the two algorithms to a corpus of abstracts from IJCNN 2009, and show that both algorithms find meaningful chunks that elucidate the semantic structure of the corpus in complementary ways.

ANSWER: An unsupervised attractor network method for detecting salient words in text corpora

“…The ANSWER model draws upon this work for its theoretical underpinnings. It is based on a neurodynamical model of thinking called IDEA (itinerant dynamics with emergent attractors) that we have described previously [45], [46], [47], [48], and our earlier work on computational models of ideation and priming [49], [50], [51], [52].…”

“…Elsewhere, we have suggested that these might represent latent ideas that could be the basis of subsequent innovation [53]. In ANSWER, the sampling of ideas from the ASN occurs using the attractor network approach derived from our previously described IDEA model [45], [46], [47], [48]. In that model, the ASN is constructed from the text corpus as a recurrent neural network with neural units representing words and weights encoding associations between them.…”

“…3) A heuristic algorithm described in [48] was used to re move further non-salient words. A relative prominence value, R( Vi), was calculated for each word, Vi, using two quantities: fELP( v i)' its frequency (in occurrences per million words) in the 40,481-word English Lexicon Project (ELP) corpus (elexicon.wustl.edu); and fe( Vi)'…”