textPrep: A Text Preprocessing Toolkit for Topic Modeling on Social Media Data

Churchill, Rob; Singh, Lisa

doi:10.5220/0010559000600070

Cited by 6 publications

(5 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Some of the challenges with social data have been looked at in [9], [14], [17], [22]. There also has been a few attempts to provide a quality framework for social media data as in [5], [20]. To the best of our knowledge, this is the first body of work that focuses on data quality challenges of social media analytics for FnB companies and provides a framework to combat it.…”

Section: Challenges Addressedmentioning

confidence: 99%

“…Thus, ensuring the accuracy, and trustworthiness of data is currently a major problem for business. Despite the importance of assessing and governing data quality for social media analytics, there aren't many studies that specifically address this issue [5], [20]. Moreover, the ideal way to assess the impact of poor data quality is to employ some downstream tasks, but in order to do that, we typically need a test set made up of test samples and their ground truth labels.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

FOODIE: A Data-centric Sifting Framework for Social Media Analytics

Agarwal,

Imran,

Perera

2023

International Conferences on Software Engineering and Knowledge Engineering

View full text Add to dashboard Cite

There has been a great deal of research conducted in the past on utilizing social media analytics to derive consumer insights and understand their behaviors. However, when such studies are applied to real-world data in an industrial use-case, the results are often found to be incorrect and erroneous. This is a major barrier for companies that provide social media analytics-based solutions to customer-centric industries such as Food and Beverage (FnB). One of the key causes of this barrier is the failure to appropriately process and curate raw social data prior to analytics. In this study, we discuss the challenges we encountered when dealing with social data throughout our industrial experience and propose a standard solution -FOODIE. This is a framework specifically designed for the FnB industry to process social conversation data accurately and in a standard manner prior to perform various downstream tasks on it. The three stages of this paradigm are preparation, sifting, and evaluation. Through this framework, we have reduced the data to error ratio from 8.76% to 0.01% which is quite significant given the volume of the data. While this framework is designed for the FnB space, it can be customized to suit the needs of various other industries.

show abstract

Section: Challenges Addressedmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

FOODIE: A Data-centric Sifting Framework for Social Media Analytics

Agarwal,

Imran,

Perera

2023

International Conferences on Software Engineering and Knowledge Engineering

View full text Add to dashboard Cite

show abstract

“…The quality of emerging topics relies on the strategies used to preprocess text. However, research suggests that no one configuration of preprocessing rules is optimal across datasets and model types [185][186][187]. Therefore, we will explore various approaches to preprocessing the clinical notes, focusing on different ways to select the terms used to train the topic models (e.g., based on term frequency, term frequency-inverse document frequency weights, and named entity recognition).…”

Section: Emr-based Nlp and ML For The Characterization Of Longitudina...mentioning

confidence: 99%

Characterization and prediction of individual functional outcome trajectories in schizophrenia spectrum disorders (PREDICTS study): Study protocol

Agarwal,

Dissanayake,

Agid

et al. 2023

PLoS ONE

View full text Add to dashboard Cite

Schizophrenia spectrum disorders (SSDs) are associated with significant functional impairments, disability, and low rates of personal recovery, along with tremendous economic costs linked primarily to lost productivity and premature mortality. Efforts to delineate the contributors to disability in SSDs have highlighted prominent roles for a diverse range of symptoms, physical health conditions, substance use disorders, neurobiological changes, and social factors. These findings have provided valuable advances in knowledge and helped define broad patterns of illness and outcomes across SSDs. Unsurprisingly, there have also been conflicting findings for many of these determinants that reflect the heterogeneous population of individuals with SSDs and the challenges of conceptualizing and treating SSDs as a unitary categorical construct. Presently it is not possible to identify the functional course on an individual level that would enable a personalized approach to treatment to alter the individual’s functional trajectory and mitigate the ensuing disability they would otherwise experience. To address this ongoing challenge, this study aims to conduct a longitudinal multimodal investigation of a large cohort of individuals with SSDs in order to establish discrete trajectories of personal recovery, disability, and community functioning, as well as the antecedents and predictors of these trajectories. This investigation will also provide the foundation for the co-design and testing of personalized interventions that alter these functional trajectories and improve outcomes for people with SSDs.

show abstract

“…This method is informative; however, the authors evaluated only one preprocessing method just over newspaper text, and the social media data was not investigated in their study. Churchill and Singh [47] suggested a standardized preprocessing approach for utilizing on-topic modelling over social media data. They showed the influence and usefulness of the proposed approach on topic modelling with various social media data.…”

Section: ) Preprocessingmentioning

confidence: 99%

Enhancing Big Social Media Data Quality for Use in Short-Text Topic Modeling

et al. 2022

View full text Add to dashboard Cite

With the emergence of microblogging platforms and social media applications, large amounts of user-generated data in the form of comments, reviews, and brief text messages are produced every day. Microblog data is typically of poor quality; hence improving the quality of the data is a significant scientific and practical challenge. In spite of the relevance of the problem, there has been not much work so far, especially in regard to microblog data quality for Short-Text Topic Modelling (STTM) purposes. This paper addresses this problem and proposes an approach called the social media data cleansing model (SMDCM) to improve data quality for STTM. We evaluate SMDCM using six topic modelling methods, namely the Latent Dirichlet Allocation (LDA), Word-Network Topic Model (WNTM), Pseudo-document-based Topic Modelling (PTM), Biterm Topic Model (BTM), Global and Local word embedding-based Topic Modeling (GLTM), and Fuzzy Topic modelling (FTM). We used the Real-world Cyberbullying Twitter (RW-CB-Twitter) and the Cyberbullying Mendeley (CB-MNDLY) datasets in the evaluation. The results proved the efficiency of the GLTM and WNTM over the other STTM models when applying the SMDCM techniques, which achieved optimum topic coherence and high accuracy values on RW-CB-Twitter and CB-MNDLY datasets.

show abstract

textPrep: A Text Preprocessing Toolkit for Topic Modeling on Social Media Data

Cited by 6 publications

References 0 publications

FOODIE: A Data-centric Sifting Framework for Social Media Analytics

FOODIE: A Data-centric Sifting Framework for Social Media Analytics

Characterization and prediction of individual functional outcome trajectories in schizophrenia spectrum disorders (PREDICTS study): Study protocol

Enhancing Big Social Media Data Quality for Use in Short-Text Topic Modeling

Contact Info

Product

Resources

About