Adaptive Distribution of Vocabulary Frequencies: A Novel Estimation Suitable for Social Media Corpus

Igawa, Rodrigo Augusto; Kido, Guilherme Sakaji; Seixas, José Luis; Barbon, Sylvio

doi:10.1109/bracis.2014.58

Cited by 4 publications

(6 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Figure 3 shows the operation of Louvain method. Although the exact computational complexity of Louvain method is not known, this method sometimes behaves as O (nlog(n)), where most effort is in the first phase of the algorithm [10].…”

Section: Louvain Methodsmentioning

confidence: 99%

“…The mathematical model ADVF proposed by Igawa et. al [10] aims to evaluate the noise level of a dataset from social media corpus. Combined with other approaches, can improve pre-processing techniques toward noisy terms elimination.…”

Section: Adaptive Distribution Of Vocabulary Frequenciesmentioning

confidence: 99%

“…Noise is an undesirable textual feature and to eliminate or reduce it is necessary specific measures. One example is the Adaptive Distribution of Vocabulary Frequencies (ADVF) [10], capable of highlighting terms detected as textual noise.…”

Section: Introductionmentioning

confidence: 99%

“…In other words, we interpreted a community in a graph such as a group of terms around a topic and based on the modularity it is possible to detect the existence of different topics in the same dataset. To detect the noisy terms we applied ADVF [10], and to treat the causes of artificial topics, like spam, we suggest an architecture analysis of the graph, mainly the density value.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Topic Modeling based on Louvain method in Online Social Networks

Kido¹,

Igawa²,

Barbon³

2016

Anais Do Simpósio Brasileiro De Sistemas De Informação (SBSI)

Self Cite

View full text Add to dashboard Cite

Online Social Networks (OSNs) are the most used media nowadays, such as Twitter. The OSNs provide valuable information to marketing and competitiveness based on users posts and opinions stored inside huge volume of data from several themes, topics and subjects. In order to mining the topics discussed on an OSN we present a novel application of Louvain method for Topic Modeling based on communities detection in graphs by modularity. The proposed approach succeeded in finding topics in five different datasets composed of textual content from Twitter and Youtube. Another important contribution achieved was about the presence of texts posted by spammers. In this case, a particular behavior observed by graph architecture (density and degree) allows the classification of a topic as natural or artificial, this last created by the spammers on OSNs.

show abstract

Section: Louvain Methodsmentioning

confidence: 99%

Section: Adaptive Distribution Of Vocabulary Frequenciesmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Topic Modeling based on Louvain method in Online Social Networks

Kido¹,

Igawa²,

Barbon³

2016

Anais Do Simpósio Brasileiro De Sistemas De Informação (SBSI)

Self Cite

View full text Add to dashboard Cite

show abstract

“…Regarding DSS creation, the first step (1) is pre-processing of the raw dataset, where are performed the text cleaning [Almeida et al 2018], Part-Of-Speech (POS) tagging [Collins 2002] and stopword removal [Igawa et al 2014]. During this step, useless spaces and special characters are converted and then tokenized.…”

Section: Text Processingmentioning

confidence: 99%

A Multi-label Classification System to Distinguish among Fake, Satirical, Objective and Legitimate News in Brazilian Portuguese

Morais

Abonizio

Tavares

et al. 2020

iSys

View full text Add to dashboard Cite

Currently, there has been a significant increase in the diffusion of fake news worldwide, especially the political class, where the possible misinformation that can be propagated, appearing at the elections debates around the world. However, news with a recreational purpose, such as satirical news, is often confused with objective fake news. In this work, we decided to address the differences between objectivity and legitimacy of news documents, where each article is treated as belonging to two conceptual classes: objective/satirical and legitimate/fake. Therefore, we propose a DSS (Decision Support System) based on a Text Mining (TM) pipeline with a set of novel textual features using multi-label methods for classifying news articles on these two domains. For this, a set of multi-label methods was evaluated with a combination of different base classifiers and then compared with a multi-class approach. Also, a set of real-life news data was collected from several Brazilian news portals for these experiments. Results obtained reported our DSS as adequate (0.80 f1-score) when addressing the scenario of misleading news, challenging the multi-label perspective, where the multi-class methods (0.01 f1-score) overcome by the proposed method. Moreover, it was analyzed how each stylometric features group used in the experiments influences the result aiming to discover if a particular group is more relevant than others. As a result, it was noted that the complexity group of features could be more relevant than others.

show abstract

Deciding among Fake, Satirical, Objective and Legitimate news

Morais

Abonizio

Tavares

et al. 2019

Proceedings of the XV Brazilian Symposium on Information Systems

Self Cite

View full text Add to dashboard Cite

Currently, the widespread of fake news has raised on the political class and society members in general, increasing concerns about the potential of misinformation that can be propagated, appearing on the center of the debate about election results around the world. On the other hand, satirical news has an entertaining purpose and are mistakenly put on the same boat of objective fake news. In this work, we address the differences between objectivity and legitimacy of news documents, treating each article as having two conceptual classes: objective/satirical and legitimate/fake. Thus, we propose a Decision Support System (DSS) based on a text mining pipeline and a set of novel textual features that uses multi-label methods for classifying news articles on those two domains. For validating the approach, a set of multi-label methods was evaluated with a combination of different base classifiers and then compared to a multi-class approach. Results reported our DSS as proper (0.80 F1-score) in addressing the scenario of misleading news from challenging perspective of multi-label modeling, outperforming the multi-class methods (0.71 F1-score) over a real-life news dataset collected from several portals of news.

show abstract

Adaptive Distribution of Vocabulary Frequencies: A Novel Estimation Suitable for Social Media Corpus

Cited by 4 publications

References 20 publications

Topic Modeling based on Louvain method in Online Social Networks

Topic Modeling based on Louvain method in Online Social Networks

A Multi-label Classification System to Distinguish among Fake, Satirical, Objective and Legitimate News in Brazilian Portuguese

Deciding among Fake, Satirical, Objective and Legitimate news

Contact Info

Product

Resources

About