Simple Baseline Machine Learning Text Classifiers for Small Datasets

Riekert, Martin; Riekert, Matthias; Klein, Achim

doi:10.1007/s42979-021-00480-4

Cited by 8 publications

(3 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Conventionally, the application of ML techniques, including NLP to text-based data, relied on large data sets for the algorithm to elicit meaningful themes. More recently, though, ML-based analysis of qualitative data yielded interesting results even on smaller data sets ( 60 ). This considered, the current study aimed to merge the potential of analysing data thematically and using NLP.…”

Section: Methodsmentioning

confidence: 99%

Asking questions that are “close to the bone”: integrating thematic analysis and natural language processing to explore the experiences of people with traumatic brain injuries engaging with patient-reported outcome measures

Di Basilio,

King,

Lloyd

et al. 2024

Front. Digit. Health

View full text Add to dashboard Cite

IntroductionPatient-reported outcomes measures (PROMs) are valuable tools for assessing health-related quality of life and treatment effectiveness in individuals with traumatic brain injuries (TBIs). Understanding the experiences of individuals with TBIs in completing PROMs is crucial for improving their utility and relevance in clinical practice.MethodsSixteen semi-structured interviews were conducted with a sample of individuals with TBIs. The interviews were transcribed verbatim and analysed using Thematic Analysis (TA) and Natural Language Processing (NLP) techniques to identify themes and emotional connotations related to the experiences of completing PROMs.ResultsThe TA of the data revealed six key themes regarding the experiences of individuals with TBIs in completing PROMs. Participants expressed varying levels of understanding and engagement with PROMs, with factors such as cognitive impairments and communication difficulties influencing their experiences. Additionally, insightful suggestions emerged on the barriers to the completion of PROMs, the factors facilitating it, and the suggestions for improving their contents and delivery methods. The sentiment analyses performed using NLP techniques allowed for the retrieval of the general sentimental and emotional “tones” in the participants’ narratives of their experiences with PROMs, which were mainly characterised by low positive sentiment connotations. Although mostly neutral, participants’ narratives also revealed the presence of emotions such as fear and, to a lesser extent, anger. The combination of a semantic and sentiment analysis of the experiences of people with TBIs rendered valuable information on the views and emotional responses to different aspects of the PROMs.DiscussionThe findings highlighted the complexities involved in administering PROMs to individuals with TBIs and underscored the need for tailored approaches to accommodate their unique challenges. Integrating TA-based and NLP techniques can offer valuable insights into the experiences of individuals with TBIs and enhance the interpretation of qualitative data in this population.

show abstract

Section: Methodsmentioning

confidence: 99%

Asking questions that are “close to the bone”: integrating thematic analysis and natural language processing to explore the experiences of people with traumatic brain injuries engaging with patient-reported outcome measures

Di Basilio,

King,

Lloyd

et al. 2024

Front. Digit. Health

View full text Add to dashboard Cite

show abstract

“…For all general domain tasks, we use results from BERT-ITPT-FiT [7], which optimises BERT for text classification, on four common benchmarks. For SVM results, we report the score of the best performing variant from a large scale comparison [8].…”

Section: General Domainmentioning

confidence: 99%

The Unreasonable Effectiveness of the Baseline: Discussing SVMs in Legal Text Classification

Clavié¹,

Alphonsus²

2021

Frontiers in Artificial Intelligence and Applications

View full text Add to dashboard Cite

We aim to highlight an interesting trend to contribute to the ongoing debate around advances within legal Natural Language Processing. Recently, the focus for most legal text classification tasks has shifted towards large pre-trained deep learning models such as BERT. In this paper, we show that a more traditional approach based on Support Vector Machine classifiers reaches competitive performance with deep learning models. We also highlight that error reduction obtained by using specialised BERT-based models over baselines is noticeably smaller in the legal domain when compared to general language tasks. We discuss some hypotheses for these results to support future discussions.

show abstract

“…However, machine learning techniques conventionally rely on large data sets to enable the algorithm to elicit meaningful themes. More recent research efforts have turned to the performance of machine learning approaches with smaller data sets and have shown promising results with as few as 300 examples in supervised machine learning (Riekert, Riekert and Klein, 2021). However, the majority of adopt an exploratory approach which is handled via unsupervised ML algorithms.…”

Section: Introductionmentioning

confidence: 99%

Are we There yet? Thematic Analysis, NLP, and Machine Learning for Research

Fitkov-Norris,

Kocheva

2023

ecrm

View full text Add to dashboard Cite

Thematic analysis is a well-established technique for qualitative analysis which is covered in traditional research methods training. The objective of thematic analysis is to elicit themes and significant topics from discursive data such as free style discussions and semi structured or unstructured interviews or comments. The approach is laborious and time consuming and requires a significant input from researchers for identifying and coding the themes although software tools such as NVivo, T-Lab and IRaMuTeQ can aid with results presentation. Recent developments in Machine Learning (ML) and Natural Language Processing (NLP) have boosted interest in text analytics and its applications to social science research. For example, automatic topic identification using ML NLP offers valuable insights in social media analytics. However, machine learning techniques conventionally rely on large data sets to enable the algorithm to elicit themes. More recent research efforts have turned to the performance of machine learning approaches with smaller data sets. This study aims to compare and contrast the effectiveness of Machine Learning NLP vs human generated themes using the text analytics tools NVivo, T-Lab, IRaMuTeQ, as well as the low-code ML tool KNIME for automatically eliciting themes from academic literature review in the contexts of service operations management research and semi-structured customer interviews. Results indicate that the ML NLP approach has the potential to automatically detect research themes even with small data sets, although the results vary across the different tools and are dependent on the capabilities of the built-in text analytic algorithms. In particular, T-Lab offered the best mapping of machine learning derived topics to researcher themes, and KNIME proved the most robust software, able to derive meaningful topics even with very small sample sizes. The implications for training research students are also significant as they suggest that the inclusion of ML NLP tools and algorithms in the training curriculum of social scientists may be beneficial.

show abstract

Simple Baseline Machine Learning Text Classifiers for Small Datasets

Cited by 8 publications

References 35 publications

Asking questions that are “close to the bone”: integrating thematic analysis and natural language processing to explore the experiences of people with traumatic brain injuries engaging with patient-reported outcome measures

Asking questions that are “close to the bone”: integrating thematic analysis and natural language processing to explore the experiences of people with traumatic brain injuries engaging with patient-reported outcome measures

The Unreasonable Effectiveness of the Baseline: Discussing SVMs in Legal Text Classification

Are we There yet? Thematic Analysis, NLP, and Machine Learning for Research

Contact Info

Product

Resources

About