A stochastic model of word occurrences in hierarchically structured written texts

Abstract

In previous studies, we have treated real written texts as time series data and have tried to investigate dynamic correlations of word occurrences by utilizing autocorrelation functions (ACFs) and also by simulation of pseudo-text synthesis. The results showed that words that appear in written texts can be classified into two groups: a group of words showing dynamic correlations (Type-I words), and a group of words showing no dynamic correlations (Type-II words). In this study, we investigate the characteristics of these two types of words in terms of their waiting time distributions (WTDs) of word occurrences. The results for Type-II words show that the stochastic processes that govern generating Type-II words are superpositions of Poisson point processes with various rate constants. We further propose a model of WTDs for Type-I words in which the hierarchical structure of written texts is considered. The WTDs of Type-I words in real written texts agree well with the predictions of the proposed model, indicating that the hierarchical structure of written texts is important for generating long-range dynamic correlations of words.

Modeling Long-Range Dynamic Correlations of Words in Written Texts with Hawkes Processes

“…In our previous study [ 16 ], a model of stochastic process that yields Type-I words has been proposed. The characteristic of the model is that a waiting time distribution (WTD) of word occurrences has a fractal structure, which is naturally introduced from the hierarchical structure of written texts, i.e., volumes, chapters, sections, subsections, paragraphs, and sentences.…”

“…The characteristic of the model is that a waiting time distribution (WTD) of word occurrences has a fractal structure, which is naturally introduced from the hierarchical structure of written texts, i.e., volumes, chapters, sections, subsections, paragraphs, and sentences. More specifically, the fractal nature of WTDs has been clarified through the following procedures [ 16 ]. First, we construct an intensity function of word occurrence along a text, , which describes the occurrence probability of a considered word at time .…”

“…In general, waiting time distributions (WTDs) are also used along with ACFs in time series analysis because they both contain equivalent information in principle. However, ACFs is more suitable in this study, because ACFs offer more precise descriptions of dynamic correlations for the case of word occurrence signals [ 16 ].…”

“…Seven texts employed in this study, which are famous academic books chosen so as to represent wide range of written texts, are listed in Table 1 with their short names and some information. The procedure for preparing texts is the same as in the previous paper [ 16 ]. In this study, we determined Type-I words to be analyzed through following steps.…”

“…For example, although word-level long-range correlations have been clearly detected especially for words which play important roles in describing the main theme of texts [ 7 , 14 , 15 ], the stochastic process which brings the dynamic correlations to word occurrences is unknown. More specifically, there exist models of stochastic process yielding long-range dynamic correlations of important words [ 15 , 16 ], but no clear conclusion has yet been obtained. Since occurrences of a word in a written text can be treated as a point process as will be described in the next section, this problem is equivalent to identifying a suitable point process which can reproduce the occurrence patterns of the considered word.…”