Comparing the Quality and Speed of Sentence Classification with Modern Language Models

Fiok, Krzysztof; Karwowski, Waldemar; Gutiérrez, Edgar; Reza-Davahli, Mohammad

doi:10.3390/app10103386

Cited by 15 publications

(10 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…This method benefits from the recent advantages in the field of ML, DL and NLP and operates on unstructured tweet text only. Based on performance of recent methods in this field [ 31 , 32 ], we decided to train a robustly optimized BERT pretraining approach model (RoBERTa “large” version) [ 33 ] and gradient boosting classifier (XGBoost) [ 34 ] on a sample of our Twitter data and further automatically infer the classes of all remaining tweets.…”

Section: Methodsmentioning

confidence: 99%

A Study of the Effects of the COVID-19 Pandemic on the Experience of Back Pain Reported on Twitter® in the United States: A Natural Language Processing Approach

Fiok

Karwowski

Gutiérrez

et al. 2021

IJERPH

Self Cite

View full text Add to dashboard Cite

The COVID-19 pandemic has changed our lifestyles, habits, and daily routine. Some of the impacts of COVID-19 have been widely reported already. However, many effects of the COVID-19 pandemic are still to be discovered. The main objective of this study was to assess the changes in the frequency of reported physical back pain complaints reported during the COVID-19 pandemic. In contrast to other published studies, we target the general population using Twitter as a data source. Specifically, we aim to investigate differences in the number of back pain complaints between the pre-pandemic and during the pandemic. A total of 53,234 and 78,559 tweets were analyzed for November 2019 and November 2020, respectively. Because Twitter users do not always complain explicitly when they tweet about the experience of back pain, we have designed an intelligent filter based on natural language processing (NLP) to automatically classify the examined tweets into the back pain complaining class and other tweets. Analysis of filtered tweets indicated an 84% increase in the back pain complaints reported in November 2020 compared to November 2019. These results might indicate significant changes in lifestyle during the COVID-19 pandemic, including restrictions in daily body movements and reduced exposure to routine physical exercise.

show abstract

Section: Methodsmentioning

confidence: 99%

A Study of the Effects of the COVID-19 Pandemic on the Experience of Back Pain Reported on Twitter® in the United States: A Natural Language Processing Approach

Fiok

Karwowski

Gutiérrez

et al. 2021

IJERPH

Self Cite

View full text Add to dashboard Cite

show abstract

“…Deep learning methods are effective for prediction because they automatically extract appropriate features from datasets [ 34 ]. RNN, a deep learning method, can store extensive historical information and use it to accurately predict the next steps in time-series problems [ 35 ].…”

Section: Mathematical Modelsmentioning

confidence: 99%

Optimizing COVID-19 vaccine distribution across the United States using deterministic and stochastic recurrent neural networks

2021

Self Cite

View full text Add to dashboard Cite

Optimizing COVID-19 vaccine distribution can help plan around the limited production and distribution of vaccination, particularly in early stages. One of the main criteria for equitable vaccine distribution is predicting the geographic distribution of active virus at the time of vaccination. This research developed sequence-learning models to predict the behavior of the COVID-19 pandemic across the US, based on previously reported information. For this objective, we used two time-series datasets of confirmed COVID-19 cases and COVID-19 effective reproduction numbers from January 22, 2020 to November 26, 2020 for all states in the US. The datasets have 310 time-steps (days) and 50 features (US states). To avoid training the models for all states, we categorized US states on the basis of their similarity to previously reported COVID-19 behavior. For this purpose, we used an unsupervised self-organizing map to categorize all states of the US into four groups on the basis of the similarity of their effective reproduction numbers. After selecting a leading state (the state with earliest outbreaks) in each group, we developed deterministic and stochastic Long Short Term Memory (LSTM) and Mixture Density Network (MDN) models. We trained the models with data from each leading state to make predictions, then compared the models with a baseline linear regression model. We also remove seasonality and trends from a dataset of non-stationary COVID-19 cases to determine the effects on prediction. We showed that the deterministic LSTM model trained on the COVID-19 effective reproduction numbers outperforms other prediction methods.

show abstract

“…Proper configuration of ML and DL models requires experiments and studies. In order to configure model parameters in this study, we based them on past research [63] and our experience in the field.…”

Section: Configuration Of Models Which Used Lstmsmentioning

confidence: 99%

Automated Classification of Evidence of Respect in the Communication through Twitter

et al. 2021

Self Cite

View full text Add to dashboard Cite

Volcanoes of hate and disrespect erupt in societies often not without fatal consequences. To address this negative phenomenon scientists struggled to understand and analyze its roots and language expressions described as hate speech. As a result, it is now possible to automatically detect and counter hate speech in textual data spreading rapidly, for example, in social media. However, recently another approach to tackling the roots of disrespect was proposed, it is based on the concept of promoting positive behavior instead of only penalizing hate and disrespect. In our study, we followed this approach and discovered that it is hard to find any textual data sets or studies discussing automatic detection regarding respectful behaviors and their textual expressions. Therefore, we decided to contribute probably one of the first human-annotated data sets which allows for supervised training of text analysis methods for automatic detection of respectful messages. By choosing a data set of tweets which already possessed sentiment annotations we were also able to discuss the correlation of sentiment and respect. Finally, we provide a comparison of recent machine and deep learning text analysis methods and their performance which allowed us to demonstrate that automatic detection of respectful messages in social media is feasible.

show abstract

Comparing the Quality and Speed of Sentence Classification with Modern Language Models

Cited by 15 publications

References 19 publications

A Study of the Effects of the COVID-19 Pandemic on the Experience of Back Pain Reported on Twitter® in the United States: A Natural Language Processing Approach

A Study of the Effects of the COVID-19 Pandemic on the Experience of Back Pain Reported on Twitter® in the United States: A Natural Language Processing Approach

Optimizing COVID-19 vaccine distribution across the United States using deterministic and stochastic recurrent neural networks

Automated Classification of Evidence of Respect in the Communication through Twitter

Contact Info

Product

Resources

About