The Role of Pre-processing in Twitter Sentiment Analysis

Bao, Yanwei; Quan, Changqin; Wang, Limin; Ren, Fuji

doi:10.1007/978-3-319-09339-0_62

Cited by 81 publications

(49 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Every word contains in a tweet is important in decision making, so pre-processing of these tweets is an important task because these messages are full of slang, misspellings, and words from other languages [60][61][62][63]. In order to tackle the problems with the noise in texts, normalization of tweets is performed by applying text preprocessing steps like tokenization, stop words removal, stemming, lemmatization, feature weighting, dimensionality reduction, frequency based methods proposed by Bao et .al [62].…”

Section: Data Preprocessingmentioning

confidence: 99%

See 1 more Smart Citation

Towards Prediction of Election Outcomes Using Social Media

Jain¹,

Kumar²

2017

IJISA

View full text Add to dashboard Cite

Abstract-Exploiting social media data by extracting key information from it is one of the great challenges in data mining and knowledge discovery. Every election campaign has an online presence of voters which uses these social media platform to express their sentiments and opinions towards political parties, leaders and important topics. This paper presents a novel data collection technique for prediction of election outcomes and a topic modeling method for extracting topics. Data collection technique used RSS (Rich Site Summary) feeds of news articles and trending keywords from Twitter simultaneously and constructed an intelligent prediction model based primarily on the volume of tweets and sentiment of users. This paper effort to improve electoral predictions using social media data based dynamic keyword methodology.Different techniques for electoral prediction based on social media data has been investigated based on existing literature and isolate the factors which improve our methodology. Meaningful inferences such as the popularity of leaders and parties during different intervals, trending issues, and important factors are extracted from the data set. The election outcomes are compared with traditional methods used by survey agencies for exit polls and validation of results showed that social media data can predict with better accuracy. The research has identified that data collection technique and timing play an important role in yielding better accuracy in predicting outcomes and extracting meaningful inferences.

show abstract

Section: Data Preprocessingmentioning

confidence: 99%

“…In order to tackle the problems with the noise in texts, normalization of tweets is performed by applying text preprocessing steps like tokenization, stop words removal, stemming, lemmatization, feature weighting, dimensionality reduction, frequency based methods proposed by Bao et .al [62].…”

Section: Data Preprocessingmentioning

confidence: 99%

Towards Prediction of Election Outcomes Using Social Media

Jain¹,

Kumar²

2017

IJISA

View full text Add to dashboard Cite

show abstract

“…Therefore, pre-processing is typically conducted to convert the text into textual features that could be fit into the SA methods. Once the pre-processed text features are extracted, they are ready to be fit in the next phase of SAFeature Selection [10] [11]. Pre-processing is usually based on NLP techniques such as tokenization (splitting the sentences into words), de-noising (remove special characters, capture symbols for emotions), normalization (remove duplicate characters, identify root words etc.…”

Section: A Preprocessingmentioning

confidence: 99%

“…For stop words, they constructed list of domain specific stop words which are not standard stop words but carry no information for the specific domain. Bao et al [11] evaluated the effects of text pre-processing in twitter sentiment analysis. They first considered username, hashtags, emotions, digital symbols, single letters, punctuations and other non-alphabetic symbols for de-noising.…”

Section: A Preprocessingmentioning

confidence: 99%

“…Table 1 below categorizes the surveyed Arabic SA studies into these classes, and fig.1 shows the result. [7], [9], [10], [11], [13], [19], [20], [27], [28], [29], [33], [41], [42], [48], [49], [52], [65], [69] Arabic sentiment analysis studies used different method based on the used approach, some of these methods considered as a dominance. We have collected the various methods used in the different Arabic SA studies for supervised, semi-supervised, unsupervised and hybrid experiments and presented the result in Table 2.…”

Section: B Arabic Sentiment Analysismentioning

confidence: 99%

See 1 more Smart Citation

Arabic Sentiment Analysis: A Survey

Assiri¹,

Emam²,

Al-Dossari³

2015

ijacsa

View full text Add to dashboard Cite

Abstract-Most social media commentary in the Arabic language space is made using unstructured non-grammatical slang Arabic language, presenting complex challenges for sentiment analysis and opinion extraction of online commentary and micro blogging data in this important domain. This paper provides a comprehensive analysis of the important research works in the field of Arabic sentiment analysis. An in-depth qualitative analysis of the various features of the research works is carried out and a summary of objective findings is presented. We used smoothness analysis to evaluate the percentage error in the performance scores reported in the studies from their linearly-projected values (smoothness) which is an estimate of the influence of the different approaches used by the authors on the performance scores obtained. To solve a bounding issue with the data as it was reported, we modified existing logarithmic smoothing technique and applied it to pre-process the performance scores before the analysis. Our results from the analysis have been reported and interpreted for the various performance parameters: accuracy, precision, recall and F-score.

show abstract