A Proposal for New Evaluation Metrics and Result Visualization Technique for Sentiment Analysis Tasks

Valverde-Albacete, Francisco J.; Carrillo-de-Albornoz, Jorge; Peláez-Moreno, Carmen

doi:10.1007/978-3-642-40802-1_5

Cited by 21 publications

(10 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The CBET can be used to visualize the performance of supervised classifiers in a straightforward manner as announced in the Introduction: Consider the confusion matrix

of a classifier chain on a supervised classification task given the random variable of true class labels

and that of predicted labels

as depicted in Figure 1 a, which now play the role of

and

. From this confusion matrix, we can estimate the joint distribution

between the random variables, so that the entropy triangle for

produces valuable information about the actual classifier used to solve the task [ 6 , 13 ] and even the theoretical limits of the task; for instance, whether it can be solved in a trustworthy manner by classification technology and with what effectiveness.…”

Section: Methodsmentioning

confidence: 99%

Assessing Information Transmission in Data Transformations with the Channel Multivariate Entropy Triangle

Valverde-Albacete

Peláez-Moreno

2018

Entropy

Self Cite

View full text Add to dashboard Cite

Data transformation, e.g., feature transformation and selection, is an integral part of any machine learning procedure. In this paper, we introduce an information-theoretic model and tools to assess the quality of data transformations in machine learning tasks. In an unsupervised fashion, we analyze the transformation of a discrete, multivariate source of information X into a discrete, multivariate sink of information Y related by a distribution P XY . The first contribution is a decomposition of the maximal potential entropy of (X, Y), which we call a balance equation, into its (a) non-transferable, (b) transferable, but not transferred, and (c) transferred parts. Such balance equations can be represented in (de Finetti) entropy diagrams, our second set of contributions. The most important of these, the aggregate channel multivariate entropy triangle, is a visual exploratory tool to assess the effectiveness of multivariate data transformations in transferring information from input to output variables. We also show how these decomposition and balance equations also apply to the entropies of X and Y, respectively, and generate entropy triangles for them. As an example, we present the application of these tools to the assessment of information transfer efficiency for Principal Component Analysis and Independent Component Analysis as unsupervised feature transformation and selection procedures in supervised classification tasks.

show abstract

“…The CBET can be used to visualize the performance of supervised classifiers in a straightforward manner as announced in the Introduction: Consider the confusion matrix

of a classifier chain on a supervised classification task given the random variable of true class labels

and that of predicted labels

as depicted in Figure 1 a, which now play the role of

and

. From this confusion matrix, we can estimate the joint distribution

between the random variables, so that the entropy triangle for

Section: Methodsmentioning

confidence: 99%

Assessing Information Transmission in Data Transformations with the Channel Multivariate Entropy Triangle

Valverde-Albacete

Peláez-Moreno

2018

Entropy

Self Cite

View full text Add to dashboard Cite

show abstract

“…the aggregation and integration of information in multiple languages, media, and coming from different domains, such as: semantic annotation and question answering in the biomedical domain; selecting success criteria in an academic library catalogue; finding similar content in different scenarios on the Web; interactive information retrieval and formative evaluation for medical professionals; microblog summarization and disambiguation; multimodal music tagging; multi-faceted IR in multimodal domains; ranking in faceted search [33,56,109,110,127,152,183,184,235,241,244,245,249]; results of a search system but also for improving interaction with and exploration of experimental outcomes such as exploiting visual analytics for failure analysis; comparing the relative performances of IR systems; and visualization for sentiment analysis [19,68,143,263];…”

Section: The Conferencementioning

confidence: 99%

CLEF 15th Birthday

Ferro

2014

SIGIR Forum

View full text Add to dashboard Cite

2014 marks the 15 th birthday for CLEF, an evaluation campaign activity which has applied the Cranfield evaluation paradigm to the testing of multilingual and multimodal information access systems in Europe. This paper provides a summary of the motivations which led to the establishment of CLEF, and a description of how it has evolved over the years, the major achievements, and what we see as the next challenges.1 http://www.clef-initiative.eu/ 2 http://trec.nist.gov/ 3

show abstract

“…Figure 1. Number of social media user compared to total population in Indonesia in million according to Hootsuite and We Are Social [24][25][26][27][28]. All data is retrieved per January of each year.…”

Section: Introductionmentioning

confidence: 99%

Identifying Personality Traits for Indonesian User from Twitter Dataset

Jeremy¹,

Prasetyo²,

Suhartono³

2019

IJFIS

View full text Add to dashboard Cite

Social media allows the user to convey their actual self and share their life experiences through numerous ways. This behavior in turn reflects the user's personality. In this paper, we experiment to automatically predict user's personality based on Big Five Personality Trait on Twitter. Our focus is towards Indonesian user. Not only word n-gram, Twitter metadata is also used in a certain combination to determine the feature that will be used to predict the personality. Our research also attempts to find optimum setting based on the number of n-gram, classifier, and twitter metadata. Our experiment yields 0.7482 at most on F-Measure. We conclude that among all scenario, twitter metadata is the least impactful feature, while word n-gram impacts the most.

show abstract

A Proposal for New Evaluation Metrics and Result Visualization Technique for Sentiment Analysis Tasks

Cited by 21 publications

References 11 publications

Assessing Information Transmission in Data Transformations with the Channel Multivariate Entropy Triangle

Assessing Information Transmission in Data Transformations with the Channel Multivariate Entropy Triangle

CLEF 15th Birthday

Identifying Personality Traits for Indonesian User from Twitter Dataset

Contact Info

Product

Resources

About