A language-model-based approach for subjectivity detection

Abstract

The rapid growth of opinionated text on the Web increases the demand for efficient methods for detecting subjective texts. In this paper, a subjectivity detection method is proposed which utilizes a language-model-based structure to define a subjectivity score for each document where the topic relevance of documents does not affect the subjectivity scores. In order to overcome the limited content in short documents, we further propose an expansion method to better estimate the language models. Since the lack of linguistic resources in resource-lean languages like Persian makes subjectivity detection difficult in these languages, the method is proposed in two versions: a semi-supervised version for resource-lean languages and a supervised version. Experimental evaluations on five datasets in two languages, English and Persian, demonstrate that the method performs well in distinguishing subjective documents from objective ones in both languages.

Proceedings of the Tenth Workshop on Computational Approaches to Subjectivity, Sentiment and Social Media Analysis

“…Language modeling is an approach which has been widely used in Information retrieval recently. In this paper, we employed the language-model based subjectivity detection method proposed in (Karimi and Shakery, 2017). In (Karimi and Shakery, 2017), each test document is assigned a subjectivity score based on its similarity to the language models of subjective and objective train datasets.…”

“…In this paper, we employed the language-model based subjectivity detection method proposed in (Karimi and Shakery, 2017). In (Karimi and Shakery, 2017), each test document is assigned a subjectivity score based on its similarity to the language models of subjective and objective train datasets. This score is computed from the difference of the similarity between the test document language model and the language model of subjective train dataset (subjective model), sim subj (d), and the similarity between the test document language model and the language model of objective train dataset (objective model), sim obj (d).…”

“…• In previous papers, this method has been used for detecting positive documents from negative ones (Hu et al, 2007) and also for detecting subjective documents from objective ones (Karimi and Shakery, 2017).…”

Mutual information and sensitivity analysis for feature selection in customer targeting: A comparative study

“…Feature selection using MI has been a subject of research in numerous problems. Moreover, studies are usually devoted to testing new feature selection approaches to well-known datasets, not focusing explicitly on the advantages to the business associated with the problems being addressed [38,39]. However, no studies were found on feature selection using MI specifically focusing on CT, only a few papers published related to customer churning [40,41].…”

A Statistical Approach for Knowledge Discovery: Bootstrapped Analysis of Language Models for Knowledge base Population from Unstructured Text

“…To this aim, this study proposes a language model-based method to nd the representative textual patterns of every relation as n-grams to be used for extracting new information. Statistical language models have been widely used for various natural language processing and text retrieval tasks, including opinion mining [10], ad-hoc retrieval [11], sentence retrieval [12], and word prediction [13]. In this paper, we bene t from this approach for information extraction from unstructured texts.…”