Enhanced SMOTE algorithm for classification of imbalanced big-data using Random Forest

Bhagat, Reshma C.; Patil, Sachin

doi:10.1109/iadcc.2015.7154739

Cited by 74 publications

(29 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Finally, a preliminary study regarding multi-class imbalanced classification was introduced in [48]. This methodology consisted of two steps.…”

Section: Multi-class Imbalancementioning

confidence: 99%

“…Specifically, "Data pre-processing studies" section contains the description for those techniques Data pre-processing [41][42][43][44][45][46][47][48] Cost-sensitive learning [41,[49][50][51] Applications on imbalanced Big Data [52][53][54] related to data pre-processing. "Cost-sensitive learning studies" section those approaches that carry out an algorithmic modification by means of a cost-sensitive learning.…”

Section: Addressing Imbalanced Classification In Big Data Problems: Cmentioning

confidence: 99%

See 1 more Smart Citation

An insight into imbalanced Big Data classification: outcomes and challenges

Fernández

Río

Chawla

et al. 2017

Complex Intell. Syst.

188

View full text Add to dashboard Cite

Big Data applications are emerging during the last years, and researchers from many disciplines are aware of the high advantages related to the knowledge extraction from this type of problem. However, traditional learning approaches cannot be directly applied due to scalability issues. To overcome this issue, the MapReduce framework has arisen as a "de facto" solution. Basically, it carries out a "divide-andconquer" distributed procedure in a fault-tolerant way to adapt for commodity hardware. Being still a recent discipline, few research has been conducted on imbalanced classification for Big Data. The reasons behind this are mainly the difficulties in adapting standard techniques to the MapReduce programming style. Additionally, inner problems of imbalanced data, namely lack of data and small disjuncts, are accentuated during the data partitioning to fit the MapReduce programming style. This paper is designed under three main pillars. First, to present the first outcomes for imbalanced classification in Big Data problems, introducing the current

show abstract

“…Finally, a preliminary study regarding multi-class imbalanced classification was introduced in [48]. This methodology consisted of two steps.…”

Section: Multi-class Imbalancementioning

confidence: 99%

Section: Addressing Imbalanced Classification In Big Data Problems: Cmentioning

confidence: 99%

An insight into imbalanced Big Data classification: outcomes and challenges

Fernández

Río

Chawla

et al. 2017

Complex Intell. Syst.

188

View full text Add to dashboard Cite

show abstract

“…Specifically, we used the LIBSVM tool [6] to build the SVM Model, and we used the Spark ML package 5 to build the random forest model. As data sampling has been shown in [3,7] to improve the random forest classification performance, for the training of our random forest model, we oversampled each class by duplicating the examples to match the class with the greatest number of examples. This reduces the imbalance due to the distribution of the employers within different industries.…”

Section: Learning Algorithmsmentioning

confidence: 99%

Automatically Detecting Errors in Employer Industry Classification Using Job Postings

Chern¹,

Liu²,

Chao³

et al. 2018

Data Sci. Eng.

View full text Add to dashboard Cite

In the recruitment domain, knowing the employer industry of jobs is important to get an insight about the demand in each industry. The existing system at CareerBuilder uses an employer name normalization system and an employer knowledge base (KB) to infer the employer industry of a job. However, errors may occur during the computation of the job employer and in the construction of the employer KB with the industry attributes. Since the KB is huge, it is not possible to manually detect the errors. Therefore, in this paper we use machine learning techniques to automatically detect the errors. With the observation that the main jobs posted by an employer often relate to the employer industry, e.g., truck driver jobs often correspond to employers in the transportation industry, we develop a system that classifies the industry of an employer using job posting data. We aggregate job postings from an employer and derive features from employer names, employer descriptions, job titles, and job descriptions to predict the industry of the employer. Two models are used for classification:(1) support vector machine and (2) random forest. Our experiments show that random forest is more effective than SVM in identifying the errors in the existing industry classification system, which achieves precision 0.69, recall 0.78, and f-score 0.73. It especially better handles mixed feature vectors when normalization errors occur. We also observe that generally our models perform better in detecting errors for industries that have higher error rates.

show abstract

“…In our CIA system we used to consider an analysis on various strategies. Binarization technique aviles various categorization techniques [22]. Learning techniques ensures optimized future extraction.…”

Section: IImentioning

confidence: 99%

A novel method to classify organizational data using CIA tree approach

Renu

2015

2015 Online International Conference on Green Engineering and Technologies (IC-GET)

View full text Add to dashboard Cite

Data classification plays an important role in an organizational information and communication technology. An efficient data classification technique reduces the complexity of data management.Here we focusing the main security threats such as Confidentiality(C), Integrity (I) and Availability (A).A tree approach is used for data classification.CIA tree is a binary tree which each level represents confidentiality, Integrity and Availability. The three tree approaches are: -Binary tree approach, weighted tree approach and Complex network approach. The binary tree approach directs user into a specified path, where the file gets a path value. In weighted tree approach user get four choices based on root value. Left or right sub tree is activated at a time based on the root value. User can select a value depends on the security of the file. In Complex network approach the full network is activate every time and user can select a value from eight different values. The security, accuracy, processing speed etc of each tree varies differently Index Terms-CIA tree, weighted approach, binary tree approach, complex network approach

show abstract

Enhanced SMOTE algorithm for classification of imbalanced big-data using Random Forest

Cited by 74 publications

References 12 publications

An insight into imbalanced Big Data classification: outcomes and challenges

An insight into imbalanced Big Data classification: outcomes and challenges

Automatically Detecting Errors in Employer Industry Classification Using Job Postings

A novel method to classify organizational data using CIA tree approach

Contact Info

Product

Resources

About