Sentiment Analysis on Twitter Data using Apache Spark Framework

Elzayady, Hossam; Badran, Khaled; Salama, Gouda I.

doi:10.1109/icces.2018.8639195

Cited by 23 publications

(10 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In our proposed technique, API model is designed which scrapes the tweets using the Twitter API. The dataset obtained is stored using Apache Spark ( Elzayady, Badran & Salama, 2018 ). Scrapped data is preprocessed to remove unnecessary content, for example links, usernames, and special characters.…”

Section: Literature Reviewmentioning

confidence: 99%

A machine learning-based approach for sentiment analysis on distance learning from Arabic Tweets

Almalki

2022

PeerJ Computer Science

View full text Add to dashboard Cite

Social media platforms such as Twitter, YouTube, Instagram and Facebook are leading sources of large datasets nowadays. Twitter’s data is one of the most reliable due to its privacy policy. Tweets have been used for sentiment analysis and to identify meaningful information within the dataset. Our study focused on the distance learning domain in Saudi Arabia by analyzing Arabic tweets about distance learning. This work proposes a model for analyzing people’s feedback using a Twitter dataset in the distance learning domain. The proposed model is based on the Apache Spark product to manage the large dataset. The proposed model uses the Twitter API to get the tweets as raw data. These tweets were stored in the Apache Spark server. A regex-based technique for preprocessing removed retweets, links, hashtags, English words and numbers, usernames, and emojis from the dataset. After that, a Logistic-based Regression model was trained on the pre-processed data. This Logistic Regression model, from the field of machine learning, was used to predict the sentiment inside the tweets. Finally, a Flask application was built for sentiment analysis of the Arabic tweets. The proposed model gives better results when compared to various applied techniques. The proposed model is evaluated on test data to calculate Accuracy, F1 Score, Precision, and Recall, obtaining scores of 91%, 90%, 90%, and 89%, respectively.

show abstract

Section: Literature Reviewmentioning

confidence: 99%

A machine learning-based approach for sentiment analysis on distance learning from Arabic Tweets

Almalki

2022

PeerJ Computer Science

View full text Add to dashboard Cite

show abstract

“…Experiment is implemented using two Arabic datasets as in (V.A), to be prepared for eliminating any needless data and each row of datasets was converted into a vector of unigrams, bigram and trigram using (TF-IDF) technique [45,47]. 75 % of data is chosen randomly for training sets and the rest is allocated for testing.…”

Section: Experiments 1: Three Traditional Machine Learning Algorithmsmentioning

confidence: 99%

Arabic Opinion Mining Using Combined CNN - LSTM Models

Elzayady¹,

Badran²,

Salama³

2020

IJISA

Self Cite

View full text Add to dashboard Cite

In the last few years, Sentiment Analysis regarding customers' reviews in order to comprehend the opinion polarity on social media has received considerable attention. However, the improvement of deep learning for sentiment analysis relating to customer reviews in Arabic language has received less attention. In fact, many users post and jot down their reviews in Arabic daily, so we ought to shed more light on Arabic sentiment analysis. Most likely all previous work depends on conventional classification techniques, such as KNN, Naï ve Bayes (NB), etc. But in this work, we implement two deep learning models: Long Short Term Memory (LSTM) and Convolution Neural Networks (CNN), in addition to three traditional techniques: Naï ve Bayes, K-Nearest Neighbor (KNN), Decision trees for sentiment analysis and compared the experimental results. Also, we offer a combined model from CNN and Recurrent Neural Network (RNN) architecture where this model collects local features through CNN as the input for RNN for Arabic sentiment analysis of short texts. An appropriate data preparation has been conducted for each utilized dataset. Our Conducted experiments for each dataset against traditional machine learning classifier; KNN, NB, and decision trees and regular deep learning models; CNN and LSTM, has resulted in impressive performance using our proposed combined (CNN-LSTM) model with an average accuracy of 85,83%, 86,88% for HTL and LABR datasets respectively.

show abstract

“…Along these lines, big data came into image in the real time business examination of processing data. Presently, individuals are communicating their opinions through online blogs, conversation forms and furthermore some online applications like Facebook, Twitter, and so on [2,3]. In the most recent decade, there has been an enormous development in the utilization of micro blogging stages, like Twitter [4] which is overpowered by astonishing statistics [5].…”

Section: Introductionmentioning

confidence: 99%

An adaptive clustering and classification algorithm for Twitter data streaming in Apache Spark

Hasan¹,

Alhayali

Zaki³

et al. 2019

TELKOMNIKA

View full text Add to dashboard Cite

On-going big data from social networks sites alike Twitter or Facebook has been an entrancing hotspot for investigation by researchers in current decades as a result of various aspects including up-to-date-ness, accessibility and popularity; however anyway there may be a trade off in accuracy. Moreover, clustering of twitter data has caught the attention of researchers. As such, an algorithm which can cluster data within a lesser computational time, especially for data streaming is needed. The presented adaptive clustering and classification algorithm is used for data streaming in Apache spark to overcome the existing problems is processed in two phases. In the first phase, the input pre-processed twitter data is viably clustered utilizing an Improved Fuzzy C-means clustering and the proposed clustering is additionally improved by an Adaptive Particle swarm optimization (PSO) algorithm. Further the clustered data streaming is assessed utilizing spark engine. In the second phase, the input pre-processed Higgs data is classified utilizing the modified support vector machine (MSVM) classifier with grid search optimization. At long last the optimized information is assessed in spark engine and the assessed esteem is utilized to discover an accomplished confusion matrix. The proposed work is utilizing Twitter dataset and Higgs dataset for the data streaming in Apache Spark. The computational examinations exhibit the superiority of presented approach comparing with the existing methods in terms of precision, recall, F-score, convergence, ROC curve and accuracy.

show abstract

Sentiment Analysis on Twitter Data using Apache Spark Framework

Cited by 23 publications

References 15 publications

A machine learning-based approach for sentiment analysis on distance learning from Arabic Tweets

A machine learning-based approach for sentiment analysis on distance learning from Arabic Tweets

Arabic Opinion Mining Using Combined CNN - LSTM Models

An adaptive clustering and classification algorithm for Twitter data streaming in Apache Spark

Contact Info

Product

Resources

About