Giant Components in Texts Generated by a Stream

Abstract

Given a text stream, we associate a stream of edges in a graph G and study its large clusters by analysing the giant components of random subgraphs, obtained by sampling some edges with different distributions. For a stream of Tweets, we show that the large giant components of uniform sampled edges of the Twitter graph reflect the large clusters of G. For a stream of text, the uniform sampling is inefficient but the weighted sampling where the weight is proportional to the Word2vec similarity provides good results. Nodes of high degree of the giant components define the central words and central sentences of the text.