TurkiS: A Turkish Sentiment Analyzer Using Domain-specific Automatic Labelled Dataset

Abstract

A preliminary task of sentiment analysis aims to detect polarities of a text either positive or negative. To increase the overall performance of the polarity detection for supervised learning methods, it requires properly labeled training texts. Also, the quality of labeled texts is critical for correct polarity detections. In this study, we provide a training and test dataset generator for Turkish sentiment analysis in which supervised learning methods can be trained without any human labor. To achieve these requirements, we extract comments and polarity scores from a popular e-commerce website for electronic devices in Turkey. Also, we employ a lexicon-based polarity detection to filter automatically labeled comments which exploit a translated version of a well-known sentiment lexicon into Turkish. We compare well-known supervised learning methods trained by both unfiltered and filtered versions of this dataset. The experimental setup is conducted by the generated evaluation dataset for the e-commerce domain. The test set contains 30% randomly selected from the generated dataset. Experimental results show that all supervised learning methods including SVM with linear kernel model, Multinomial Naive Bayes, and Logistic Regression perform 8.2 percent better in the filtered version of the dataset than the unfiltered version. Moreover, the logistic regression gets the highest score when it uses count vectorizer as a feature extraction mechanism.