Origin of Dynamic Correlations of Words in Written Texts

Abstract

In a previous study, we introduced dynamical aspects of written texts by regarding serial sentence number from the first to last sentence of a given text as discretized time. Using this definition of a textual timeline, we defined an autocorrelation function (ACF) for word occurrences and demonstrated its utility both for representing dynamic word correlations and for measuring word importance within the text. In this study, we seek a stochastic process governing occurrences of a given word having strong dynamic correlations. This is valuable because words exhibiting strong dynamic correlations play a central role in developing or organizing textual contexts. While seeking this stochastic process, we find that additive binary Markov chain theory is useful for describing strong dynamic word correlations, in the sense that it can reproduce characteristics of autocovariance functions (an unnormalized version of ACFs) observed in actual written texts. Using this theory, we propose a model for time-varying probability that describes the probability of word occurrence in each sentence in a text. The proposed model considers hierarchical document structures such as chapters, sections, subsections, paragraphs, and sentences. Because such a hierarchical structure is common to most documents, our model for occurrence probability of words has a wide range of universality for interpreting dynamic word correlations in actual written texts. The main contributions of this study are, therefore, finding usability of the additive binary Markov chain theory to analyze dynamic correlations in written texts and offering a new model of word occurrence probability in which common hierarchical structure of documents is taken into account.

Simulation of pseudo-text synthesis for generating words with long-range dynamic correlations

“…However, the former is still not adequate especially at regions of waiting times longer than 10 sentences. On the other hand, as indicated by diamonds in the figure, simulated signals of word occurrences by the use of an additive binary Markov chain model [16] completely reproduce characteristics of real waiting time distributions. Note that the memory effect of word occurrences is taken into account in the additive binary Markov chain model through memory functions which persist several tens of sentences.…”

“…At this stage, despite the importance of Type-I words, the stochastic process yielding them has not been clarified except a formal modeling by the use of an additive binary Markov chain [16]. This is mainly because known models to yield ACFs of KWW type are mostly developed in the field of condensed matter physics [17][18][19], and therefore, they are based on different situations from ours.…”

“…The left column shows plots on linear scales, while the plots in right column are on logarithmic scales. Solid circles indicate real waiting time distributions observed in written texts, and diamonds indicate simulated distributions by the use of an additive binary Markov chain model[16]. Red lines are the best fitted curves of the q-exponential distributions, and blue lines are those of the exponential distributions.…”

A stochastic model of word occurrences in hierarchically structured written texts

“…The occurrence patterns of a considered word in a given written text can be interpreted as time series data along the text. Since analyses based on this dynamical interpretation provide abundant information compared to static pictures in which the written text is treated as an aggregation of statistically distributed words with certain probability distributions, methods of time series analysis have been adapted to written texts for various purposes including analysis of literary styles [1], investigations of word rhythms [2][3][4][5][6], and measurements of word importance [7,8].…”

“…Occurrence of words in a context-spe-cific and bursty manner, and long-range dynamic correlations are seen in the plots in the left and right columns, respectively. Red curves in the plots in the right column show best fitted results by use of the stretched exponential functions [7][8][9] optimized parameters of which are displayed in each of the plots constant values of almost zero. The words employed in the figure are typical Type-I words picked from 'An Inquiry into the Nature and Causes of the Wealth of Nations' , which is an famous book in the field of political economy written by Adam Smith.…”