Two scalable algorithms for associative text classification

Yoon, Yongwook; Lee, Gary G.

doi:10.1016/j.ipm.2012.09.003

Cited by 17 publications

(8 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Furthermore, since associative relationships reflect the close dependencies among predictive variables, the extraction of highly confident rules produces rule-based models more interpretable than models generated by using other techniques [10,11]. Thus, in the last years, ACs have been successfully exploited in a number of real world applications such as phishing detection in websites [12], XML document classification [13], text analysis [14], and disease classification [14,15].…”

Section: Introductionmentioning

confidence: 99%

A MapReduce solution for associative classification of big data

Bechini

Marcelloni

Segatori

2016

Information Sciences

103

View full text Add to dashboard Cite

Associative classifiers have proven to be very effective in classification problems. Unfortunately, the algorithms used for learning these classifiers are not able to adequately manage big data because of time complexity and memory constraints. To overcome such drawbacks, we propose a distributed association rule-based classification scheme shaped according to the MapReduce programming model. The scheme mines classification association rules (CARs) using a properly enhanced, distributed version of the well-known FP-Growth algorithm. Once CARs have been mined, the proposed scheme performs a distributed rule pruning. The set of survived CARs is used to classify unlabeled patterns. The memory usage and time complexity for each phase of the learning process are discussed, and the scheme is evaluated on seven real-world big datasets on the Hadoop framework, characterizing its scalability and achievable speedup on small computer clusters. The proposed solution for associative classifiers turns to be suitable to practically address big datasets even with modest hardware support. Comparisons with two state-of-the-art distributed learning algorithms are also discussed in terms of accuracy, model complexity, and computation time

show abstract

Section: Introductionmentioning

confidence: 99%

A MapReduce solution for associative classification of big data

Bechini

Marcelloni

Segatori

2016

Information Sciences

103

View full text Add to dashboard Cite

show abstract

“…A classifier can be used to infer that a new record belongs to a certain class. Thus far, classification technology has been used in many applications, including customer relationship management, medical diagnosis, and fraud prevention (Jyoti et al 2011 ; Ngai et al 2009 ; Yoon and Lee 2013 ). Researchers have developed many classification techniques, which can be categorized as rule-based or non-rule-based approaches.…”

Section: Introductionmentioning

confidence: 99%

Building an associative classifier with multiple minimum supports

Tsai

et al. 2016

SpringerPlus

View full text Add to dashboard Cite

Classification is one of the most important technologies used in data mining. Researchers have recently proposed several classification techniques based on the concept of association rules (also known as CBA-based methods). Experimental evaluations on these studies show that in average the CBA-based approaches can yield higher accuracy than some of conventional classification methods. However, conventional CBA-based methods adopt a single threshold of minimum support for all items, resulting in the rare item problem. In other words, the classification rules will only contain frequent items if minimum support (minsup) is set as high or any combinations of items are discovered as frequent if minsup is set as low. To solve this problem, this paper proposes a novel CBA-based method called MMSCBA, which considers the concept of multiple minimum supports (MMSs). Based on MMSs, different classification rules appear in the corresponding minsups. Several experiments were conducted with six real-world datasets selected from the UCI Machine Learning Repository. The results show that MMSCBA achieves higher accuracy than conventional CBA methods, especially when the dataset contains rare items.

show abstract

“…Association rule mining has become a very popular method for generating highly accurate classification models, called associative classifiers (ACs). Such models have been extensively studied in the literature [1], [2], [3] and have been recently exploited in a large number of real world applications, such as detection phishing activities in websites [4], and text analysis [5].…”

Section: Introductionmentioning

confidence: 99%

A MapReduce-based fuzzy associative classifier for big data

Ducange

Marcelloni

Segatori

2015

2015 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE)

View full text Add to dashboard Cite

In this paper, we propose an efficient distributed fuzzy associative classification model based on the MapReduce paradigm. The learning algorithm first mines a set of fuzzy association classification rules by employing a distributed version of a fuzzy extension of the well-known FP-Growth algorithm. Then, it prunes this set by using three purposely adapted types of pruning. We implemented the distributed fuzzy associative classifier using the Hadoop framework. We show the scalability of our approach by carrying out a number of experiments on a real-world big dataset. In particular, we evaluate the achievable speedup on a small computer cluster, highlighting that the proposed approach allows handling big datasets even with modest hardware support

show abstract

Two scalable algorithms for associative text classification

Cited by 17 publications

References 7 publications

A MapReduce solution for associative classification of big data

A MapReduce solution for associative classification of big data

Building an associative classifier with multiple minimum supports

A MapReduce-based fuzzy associative classifier for big data

Contact Info

Product

Resources

About