Exploring the impact of short-text complexity and structure on its quality in social media

Abstract

PurposeThe purpose of this paper is to explore to which extent the quality of social media short text without extensions can be investigated and what are the predictors, if any, of such short text that lead to trust its content.Design/methodology/approachThe paper applies a trust model to classify data collections based on metadata into four classes: Very Trusted, Trusted, Untrusted and Very Untrusted. These data are collected from the online communities, Genius and Stack Overflow. In order to evaluate short texts in terms of its trust levels, the authors have conducted two investigations: (1) A natural language processing (NLP) approach to extract relevant features (i.e. Part-of-Speech and various readability indexes). The authors report relatively good performance of the NLP study. (2) A machine learning technique in more precise, a random forest (RF) classifierusing bag-of-words model (BoW).FindingsThe investigation of the RF classifier using BoW shows promising intermediate results (on average 62% accuracy of both online communities) in short-text quality identification that leads to trust.Practical implicationsAs social media becomes an increasingly new and attractive source of information, which is mostly provided in the form of short texts, businesses (e.g. in search engines for smart data) can filter content without having to apply complex approaches and continue to deal with information that is considered more trustworthy.Originality/valueShort-text classifications with regard to a criterion (e.g. quality, readability) are usually extended by an external source or its metadata. This enhancement either changes the original text if it is an additional text from an external source, or it requires text metadata that is not always available. To this end, the originality of this study faces the challenge of investigating the quality of short text (i.e. social media text) without having to extend or modify it using external sources. This modification alters the text and distorts the results of the investigation.

TopicsRanksDC: Distance-Based Topic Ranking Applied on Two-Class Data

“…We have applied an under-sampling approach that reduces the number of samples of the majority class to the minority class, thus reducing the bias in the size distribution of the data subsets. For more details see [13]. We have set the ratio of the reduction to be up to 2 fold.…”

“…The number of words/features after performing the pre-processing stage is 714 for trusted vs untrusted data set, 848 words for the very-trusted vs very-untrusted data set and 1440 words for the human-aids vs mouse-cancer data set. For more detail see [13].…”

Comparative Analysis of Word Embeddings in Assessing Semantic Similarity of Complex Sentences

“…Readability indices are used by researchers to measure the complexity of text data mostly in text simplification tasks [30], [17] [31], [32], [33], [34]. In order to prove that the sentences chosen for building this dataset are more complex than the existing benchmark datasets, a comparative readability analysis is conducted between the two existing benchmark datasets and the proposed dataset.…”

“…Two of the most popular datasets are the STS benchmark dataset [15] and the SICK dataset [16] on which the BERT models have achieved near-perfect results [12]. Analyzing the readability of the sentences in these datasets, we find that the sentences in these datasets have a low readability index which is a measure of complexity of sentences [17]. However, various real world applications of semantic similarity involve more complex sentences to be analysed [18].…”

TextNetTopics: Text Classification Based Word Grouping as Topics and Topics’ Scoring

“…In other words, the textual data is represented as a bag of topics ( Zhou et al, 2009 ; Yousef et al, 2020a ) rather than a bag of words. However, in the short-text corpus, an advanced approach must be developed ( Al Qundus et al, 2020 ).…”