LDA boost classification: boosting by topics

Abstract

AdaBoost is an efficacious classification algorithm especially in text categorization (TC) tasks. The methodology of setting up a classifier committee and voting on the documents for classification can achieve high categorization precision. However, traditional Vector Space Model can easily lead to the curse of dimensionality and feature sparsity problems; so it affects classification performance seriously. This article proposed a novel classification algorithm called LDABoost based on boosting ideology which uses Latent Dirichlet Allocation (LDA) to modeling the feature space. Instead of using words or phrase, LDABoost use latent topics as the features. In this way, the feature dimension is significantly reduced. Improved Naïve Bayes (NB) is designed as the weaker classifier which keeps the efficiency advantage of classic NB algorithm and has higher precision. Moreover, a two-stage iterative weighted method called Cute Integration in this article is proposed for improving the accuracy by integrating weak classifiers into strong classifier in a more rational way. Mutual Information is used as metrics of weights allocation. The voting information and the categorization decision made by basis classifiers are fully utilized for generating the strong classifier. Experimental results reveals LDABoost making categorization in a low-dimensional space, it has higher accuracy than traditional AdaBoost algorithms and many other classic classification algorithms. Moreover, its runtime consumption is lower than different versions of AdaBoost, TC algorithms based on support vector machine and Neural Networks.

LDA-AdaBoost.MH: Accelerated AdaBoost.MH based on latent Dirichlet allocation for text categorization

“…Mallet is a library of Java code for machine learning applied to text applications developed by Andrew McCallum. The number of topics for each dataset is obtained according to equation (16). For LDA estimation of the training set, the inputs are (number of iterations = 2000, α= 50/K, β= 0.1), and for the LDA prediction, the number of the given resampling iterations is 1000.…”

“…A related study was conducted by Lei et al [16] to use LDA as a feature representation method for TC based on the binary version of AdaBoost algorithm and using Naive Bayes as a weak learner for AdaBoost. However, Naive Bayes works based on the feature frequencies, while representing the documents as latent topics means that each document is represented as a small number of weighted and unique topics.…”

Classifying Web Pages by Aimed Nation Using Machine Learning

“…The results show that this improved random forests outperformed the popular text classification methods such as Naïve bayes, SVM, KNN, RF in terms of classification performance, it gave an f-score up to 91%. Lei, Qiao, Qimin & Qitao (2012) perform topic text categorization using LDAboost ensemble method learning. The experimental result showed that LDAboost increase the performance from 73.3% to 90%.…”

“…A second classifier is then created after it, to focus on the instances in the training data that the first classifier got wrong. The process continues to add classifiers until a limit is reached in the number of models or accuracy (Lei, Qiao, Qimin & Qitao, 2012).…”

Topic Models Ensembles for AD-HOC Information Retrieval

“…The proposed method was based on boosting algorithms for multilabel multiclass text categorization, outperforming text classifiers based on TF-IDF [12] and naive Bayes. The use of LDA-based features in boosting algorithms was introduced by La et al [30]. The method, named LDABoost, uses latent topics extracted from one LDA model as text features.…”