A Comparative Study of Text Preprocessing Techniques for Natural Language Call Routing

Sergienko, Roman; Shan, Muhammad; Schmitt, Alexander

doi:10.1007/978-981-10-2585-3_2

Cited by 7 publications

(3 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Preprocessing (Sergienko et al, 2017) is performed on incoming email files to transform strings of characters into a representation suitable for the classification algorithm (Figure 1, Tables I and II). Figure 1 clearly depicts the structure of a spam classifier in which an email file with header and body goes for preprocessing.…”

Section: Preprocessingmentioning

confidence: 99%

A study of boosted evolutionary classifiers for detecting spam

Trivedi

Dey

2019

GKMC

View full text Add to dashboard Cite

Purpose Email is a rapid and cheapest medium of sharing information, whereas unsolicited email (spam) is constant trouble in the email communication. The rapid growth of the spam creates a necessity to build a reliable and robust spam classifier. This paper aims to presents a study of evolutionary classifiers (genetic algorithm [GA] and genetic programming [GP]) without/with the help of an ensemble of classifiers method. In this research, the classifiers ensemble has been developed with adaptive boosting technique. Design/methodology/approach Text mining methods are applied for classifying spam emails and legitimate emails. Two data sets (Enron and SpamAssassin) are taken to test the concerned classifiers. Initially, pre-processing is performed to extract the features/words from email files. Informative feature subset is selected from greedy stepwise feature subset search method. With the help of informative features, a comparative study is performed initially within the evolutionary classifiers and then with other popular machine learning classifiers (Bayesian, naive Bayes and support vector machine). Findings This study reveals the fact that evolutionary algorithms are promising in classification and prediction applications where genetic programing with adaptive boosting is turned out not only an accurate classifier but also a sensitive classifier. Results show that initially GA performs better than GP but after an ensemble of classifiers (a large number of iterations), GP overshoots GA with significantly higher accuracy. Amongst all classifiers, boosted GP turns out to be not only good regarding classification accuracy but also low false positive (FP) rates, which is considered to be the important criteria in email spam classification. Also, greedy stepwise feature search is found to be an effective method for feature selection in this application domain. Research limitations/implications The research implication of this research consists of the reduction in cost incurred because of spam/unsolicited bulk email. Email is a fundamental necessity to share information within a number of units of the organizations to be competitive with the business rivals. In addition, it is continually a hurdle for internet service providers to provide the best emailing services to their customers. Although, the organizations and the internet service providers are continuously adopting novel spam filtering approaches to reduce the number of unwanted emails, the desired effect could not be significantly seen because of the cost of installation, customizable ability and the threat of misclassification of important emails. This research deals with all the issues and challenges faced by internet service providers and organizations. Practical implications In this research, the proposed models have not only provided excellent performance accuracy, sensitivity with low FP rate, customizable capability but also worked on reducing the cost of spam. The same models may be used for other applications of text mining also such as sentiment analysis, blog mining, news mining or other text mining research. Originality/value A comparison between GP and GAs has been shown with/without ensemble in spam classification application domain.

show abstract

Section: Preprocessingmentioning

confidence: 99%

A study of boosted evolutionary classifiers for detecting spam

Trivedi

Dey

2019

GKMC

View full text Add to dashboard Cite

show abstract

“…In text categorization and sentiment analysis, Support Vector Machine is often considered as the best classifier providing the greatest performances for those tasks [25]. It"s among the class of classifiers based on kernel substitution [26].In this work, the version Sequential Minimal Optimization (SMO) developed in [27] is used.…”

Section: A Experimental Setup 1) Preprocessingmentioning

confidence: 99%

WordNet based Implicit Aspect Sentiment Analysis for Crime Identification from Twitter

Hannach¹,

Benkhalifa²

2018

ijacsa

View full text Add to dashboard Cite

Crime analysis has become an interesting field that deals with serious public safety issues recognized around the world. Today, investigating Twitter Sentiment Analysis (SA) is a continuing concern within this field. Aspect based SA, the process by which information can be extracted, analyzed and classified, is applied to tweet datasets for sentiment polarity classification to predict crimes. This paper addresses the aspect identification task involving implicit aspect implied by adjectives and verbs for crime tweets. The proposed hybrid model is based on WordNet semantic relations and Term-Weighting scheme, to enhance training data for (1) Crime Implicit Aspect sentences detection (IASD) and (2) Crime Implicit Aspect Identification (IAI). The performance is evaluated using three classifiers Multinomial Naïve Bayes, Support Vector Machine and Random Forest on three Twitter crime datasets. The obtained results demonstrate the effectiveness of WN synonym and definition relations and prove the importance of verbs in training data enhancement for crime IASD and IAI.

show abstract

“…Tokenization process, basically practical is by recognizing the token and their check. Tokenization is a strategy of unmistakable verification of token/subjects inside data compositions and it serves to decreased interest with an immense degree [13]. In a present-day time of data/information, when data/information is broadening complex on reliably from its beginning stage, in a kind of compositions, site pages, etc., so the noteworthiness of effective and profitable tokenization count wraps up perceptibly essential for an IR system [14].…”

mentioning

confidence: 99%

An Effective Preprocessing Algorithm for Information Retrieval System

Sunita*¹,

Vijay²

2019

IJRTE

View full text Add to dashboard Cite

The innovation of web produced a huge of information, evaluates by empowering Internet users to post their assessments, remarks, and audits on the web. Preprocessing helps to understand a user query in the Information Retrieval (IR) system. IR acts as the container to representation, seeking and access information that relates to a user search string. The information is present in natural language by using some words; it’s not structured format, and sometimes that word often ambiguous. One of the major challenges determines in current web search vocabulary mismatch problem during the preprocessing. In an IR system determine a drawback in web search; the search query string is that the relationships between the query expressions and the expanded terms are limited. The query expressions relate to search term fetching information from the IR. The expanded terms by adding those terms that is most similar to the words of the search string. In this manuscript, we mainly focus on behind user’s search string on the web. We identify the best features within this context for term selection in supervised learning based model. In this proposed system the main focus of preprocessing techniques like Tokenization, Stemming, spell check, find dissimilar words and discover the keywords from the user query because provide better results for the user

show abstract

A Comparative Study of Text Preprocessing Techniques for Natural Language Call Routing

Cited by 7 publications

References 17 publications

A study of boosted evolutionary classifiers for detecting spam

A study of boosted evolutionary classifiers for detecting spam

WordNet based Implicit Aspect Sentiment Analysis for Crime Identification from Twitter

An Effective Preprocessing Algorithm for Information Retrieval System

Contact Info

Product

Resources

About