Utilizing global and path information with language modelling for hierarchical text classification

Abstract

Hierarchical text classification of a Web taxonomy is challenging because it is a very large-scale problem with hundreds of thousand categories and associated documents. Furthermore, the conceptual levels and training data availabilities of categories vary widely. The narrow-down approach is the state-of-the-art that utilizes a search engine for generating candidates from the taxonomy and builds a classifier for the final category selection. In this paper, we take the same approach but address the issue of using global information in a language modelling framework to improve effectiveness. We propose three methods of using non-local information for the task: a passive way of utilizing global information for smoothing, an aggressive way where a top-level classifier is built and integrated with a local model, and a method of using label terms associated with the path from a category to the root, which is based on our systematic observation that they are underrepresented in the documents. For evaluation, we constructed a document collection from Web pages in the Open Directory Project (ODP). A series of experiments and their results show the superiority of our methods and reveal the role of global information in hierarchical text classification.

External methods to address limitations of using global information on the narrow-down approach for hierarchical text classification

“…Our key contribution is the derivation of new approaches for the category selection stage. For the robust understanding of our novel methods, we first give a quick summary of language models and introduce an existing category selection approach that utilizes global information aggressively [16] because the aggressive method outperforms the passive one. We then describe our proposed two category selection methods in Sections 3.3 and 3.4, respectively.…”

“…The difference is that the former is to update centroids immediately after classification while the latter is after classifying all the training documents. We borrow the online update method used in Oh and Myaeng [16] and the best performing parameter setting [18] is chosen.…”

“…Other narrow-down approaches [14, 15] incorporate global information available at the top of the hierarchy and combine it with the local information associated with the candidates for increased classification effectiveness. Oh et al [16] proposed passive and aggressive methods utilizing global information based on language modelling framework. In addition, the label language model is developed to weight label terms in local model by observing that the label terms do not occur frequently as expected.…”

“…Based on the observation that the narrow-down approach relies on local information associated with documents of candidates, a series of methods for utilizing global information associated with top-level categories under the root for category selection are proposed [14–16]. The key idea of using global information is that guidance from the top level helps in finding an answer category.…”

“…The key idea of using global information is that guidance from the top level helps in finding an answer category. Two different types of methods using global information based on a statistical language modelling framework [16] have been developed: aggressive and passive methods. In both methods, local and global language models for a category and top-level category are generated and combined to produce a score for a candidate.…”