Influence of Pre-Processing Strategies on the Performance of ML Classifiers Exploiting TF-IDF and BOW Features

Pimpalkar, Amit; Raj, R. Jeberson Retna

doi:10.14201/adcaij2020924968

Cited by 43 publications

(10 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…TF-IDF not only vectorizes the text data but also quantifies the features among the whole corpus [ 45 ]. TF-IDF is obtained by taking a product of two metrics including the number of times a word has appeared in a document and the IDF (inverse document frequency) of a word in a collection of documents [ 46 ]. It assigns weights with respect to the importance of a term in a given corpus.…”

Section: Methodsmentioning

confidence: 99%

Analyzing preventive precautions to limit spread of COVID-19

Ahmad

Rustam²,

Saad³

et al. 2022

PLoS ONE

View full text Add to dashboard Cite

With the global spread of COVID-19, the governments advised the public for adopting safety precautions to limit its spread. The virus spreads from people, contaminated places, and nozzle droplets that necessitate strict precautionary measures. Consequently, different safety precautions have been implemented to fight COVID-19 such as wearing a facemask, restriction of social gatherings, keeping 6 feet distance, etc. Despite the warnings, highlighted need for such measures, and the increasing severity of the pandemic situation, the expected number of people adopting these precautions is low. This study aims at assessing and understanding the public perception of COVID-19 safety precautions, especially the use of facemask. A unified framework of sentiment lexicon with the proposed ensemble EB-DT is devised to analyze sentiments regarding safety precautions. Extensive experiments are performed with a large dataset collected from Twitter. In addition, the factors leading to a negative perception of safety precautions are analyzed by performing topic analysis using the Latent Dirichlet allocation algorithm. The experimental results reveal that 12% of the tweets correspond to negative sentiments towards facemask precaution mainly by its discomfort. Analysis of change in peoples’ sentiment over time indicates a gradual increase in the positive sentiments regarding COVID-19 restrictions.

show abstract

Section: Methodsmentioning

confidence: 99%

Analyzing preventive precautions to limit spread of COVID-19

Ahmad

Rustam²,

Saad³

et al. 2022

PLoS ONE

View full text Add to dashboard Cite

show abstract

“…This method is called One-Hot, that is, one-hot coding [ 18 ]. You can also adopt the strategy of TF-IDF (term frequency, inverse document frequency) [ 19 , 20 ], in which TF (Term frequency) is called word frequency, and its importance is shown in

…”

Section: Relevant Technical Theory Of Intelligent Grading Of English ...mentioning

confidence: 99%

Machine Learning-Based Intelligent Scoring of College English Teaching in the Field of Natural Language Processing

Wang

2022

Computational Intelligence and Neuroscience

View full text Add to dashboard Cite

The current education evaluation is limited not only to the mode of simplification, indexing, and datafication, but also to the scientific nature of college teaching evaluation. This work firstly conducts a theoretical analysis of natural language processing technology, analyzes the related technologies of intelligent scoring, designs a systematic process for intelligent scoring of college English teaching, and finally conducts theoretical research on the Naive Bayesian algorithm in machine learning. In addition, the error of intelligent scoring of English teaching in colleges and universities and the accuracy of scoring and classification are analyzed and researched. The results show that the error between manual scoring and machine scoring is basically about 2 points and the minimum error of intelligent scoring in college English teaching under machine scoring can reach 0 points. There is a certain bias in manual scoring, and scoring on the machine can reduce the generation of this error. The Naive Bayes algorithm has the highest classification accuracy on the college intelligent scoring dataset, which is 76.43%. The weighted Naive Bayes algorithm has been improved in the classification accuracy of college English teaching intelligent scoring, with an average accuracy rate of 74.87%. To sum up, the weighted Naive Bayes algorithm has better performance in the classification accuracy of college English intelligent scoring. This work has a significant effect on the scoring of the college intelligent teaching scoring system under natural language processing and the classification of college teaching intelligence scoring under the Naive Bayes algorithm, which can improve the efficiency of college teaching scoring.

show abstract

“…The text is converted into a bag of words in BOW (11) . With size m x n, the feature matrix is created.…”

Section: Bow Featuresmentioning

confidence: 99%

Structuring of Unstructured Data from Heterogeneous Sources

Shilpa¹,

Shambhavi²

2022

IJST

View full text Add to dashboard Cite

Objectives: To develop a new data gathering processing under Big Data Perspectives. To convert unstructured text data into structured format by not missing out any text data available. Methods: The unstructured data is preprocessed using modified stemming and tokenization. From the stemming output, the proposed Term Frequency-Inverse Document Frequency (TF-IDF) and N-gram features are derived. Unstructured data is considered from multiple sources like twitter, consumer complaints and news blog. Findings: The proposed model with extant TF-IDF features has exposed relatively high Mean Average Error (MAE) value which is 1.4325 when compared to the proposed model without optimization to be 0.5197. Novelty: The novelty of the research work is of the stemming process where dictionary checking process is added and the improved feature extraction, interclass dispersion coefficient is computed in TF-IDF features.

show abstract

Influence of Pre-Processing Strategies on the Performance of ML Classifiers Exploiting TF-IDF and BOW Features

Cited by 43 publications

References 35 publications

Analyzing preventive precautions to limit spread of COVID-19

Analyzing preventive precautions to limit spread of COVID-19

Machine Learning-Based Intelligent Scoring of College English Teaching in the Field of Natural Language Processing

Structuring of Unstructured Data from Heterogeneous Sources

Contact Info

Product

Resources

About