The Effect of Stemming and Removal of Stopwords on the Accuracy of Sentiment Analysis on Indonesian-language Texts

Abstract

Preprocessing is an essential task for sentiment analysis since textual information carries a lot of noisy and unstructured data. Both stemming and stopword removal are pretty popular preprocessing techniques for text classification. However, the prior research gives different results concerning the influence of both methods toward accuracy on sentiment classification. Therefore, this paper conducts further investigations about the effect of stemming and stopword removal on Indonesian language sentiment analysis. Furthermore, we propose four preprocessing conditions which are with using both stemming and stopword removal, without using stemming, without using stopword removal, and without using both. Support Vector Machine was used for the classification algorithm and TF-IDF as a weighting scheme. The result was evaluated using confusion matrix and k-fold cross-validation methods. The experiments result show that all accuracy did not improve and tends to decrease when performing stemming or stopword removal scenarios. This work concludes that the application of stemming and stopword removal technique does not significantly affect the accuracy of sentiment analysis in Indonesian text documents.

Dataset of stopwords extracted from Uzbek texts

“…Most of the recent literature presents methods that are based on statistical approaches. The sources mainly use the TF-IDF method to analyze the text, such as [2] , [3] , [4] , [5] , [6] , [7] . In a wide group of languages, the stopwords may be simply exposed by their relatively higher occurrence frequencies.…”

Lockdowns, lethality, and laissez-faire politics. Public discourses on political authorities in high-trust countries during the COVID-19 pandemic

“…Nevertheless, in order to provide some level of protection of the Twitter users whose tweets were included in this study, usernames have been removed from the data made available in the supporting information for this study. The Twitter API limits the retrieval of tweets to tweets from the past week; in order to circumvent this limitation, we used the software GetOldTweets by Jefferson Henrique [88], which has gained some prominence among social media researchers in recent years, as it permits the collection of up to 100% of all available tweets under the search terms entered [85,[89][90][91][92]. As this method of data collection does not include any metadata, no personal data was collected within the context of this study.…”

Deteksi Kesamaan Teks Jawaban pada Sistem Test Essay Online dengan Pendekatan Neural Network

“…Hasil dari proses tokenisasi yang berupa kumpulan kata dalam array, diproses lebih lanjut dengan menghilangkan kata umum yang sering muncul (stop word), namun tidak perlu diperhitungkan untuk mendapatkan konteks utama dalam sebuah teks. Contoh yang stop word pada Bahasa Indonesia adalah 'yang', 'di', 'untuk', dan 'dari' [14].…”