Ukrainian News Corpus as Text Classification Benchmark

Panchenko, Dmytro; Maksymenko, Daniil; Turuta, Olena; Luzan, Mykyta; Tytarenko, Stepan; Turuta, Oleksii

doi:10.1007/978-3-031-14841-5_37

Cited by 4 publications

(5 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…There are many different corpora for training text classification models that have become an important resource for research and applications in the field of natural language processing [17,18]. These corpora represent different types of text data from different sources and cover a wide range of topics.…”

Section: Methodsmentioning

confidence: 99%

Justifying the selection of a neural network linguistic classifier

Barkovska,

Voropaieva,

Ruskikh

2023

ITSSI

View full text Add to dashboard Cite

The subject matter of this article revolves around the exploration of neural network architectures to enhance the accuracy of text classification, particularly within the realm of natural language processing. The significance of text classification has grown notably in recent years due to its pivotal role in various applications like sentiment analysis, content filtering, and information categorization. Given the escalating demand for precision and efficiency in text classification methods, the evaluation and comparison of diverse neural network models become imperative to determine optimal strategies. The goal of this study is to address the challenges and opportunities inherent in text classification while shedding light on the comparative performance of two well-established neural network architectures: Long Short-Term Memory (LSTM) and Convolutional Neural Network (CNN). To achieve the goal, the following tasks were solved: a comprehensive analysis of these neural network models was performed, considering several key aspects. These aspects include classification accuracy, training and prediction time, model size, data distribution, and overall ease of use. By systematically assessing these attributes, this study aims to provide valuable information about the strengths and weaknesses of each model and enable researchers and practitioners to make informed decisions when selecting a neural network classifier for text classification tasks. The following methods used are a comprehensive analysis of neural network models, assessment of classification accuracy, training and prediction time, model size, and data distribution. The following results were obtained: The LSTM model demonstrated superior classification accuracy across all three training sample sizes when compared to CNN. This highlights LSTM's ability to effectively adapt to diverse data types and consistently maintain high accuracy, even with substantial data volumes. Furthermore, the study revealed that computing power significantly influences model performance, emphasizing the need to consider available resources when selecting a model. Conclusions. Based on the study's findings, the Long Short-Term Memory (LSTM) model emerged as the preferred choice for text data classification. Its adeptness in handling sequential data, recognizing long-term dependencies, and consistently delivering high accuracy positions it as a robust solution for text analysis across various domains. The decision is supported by the model's swift training and prediction speed and its compact size, making it a suitable candidate for practical implementation.

show abstract

Section: Methodsmentioning

confidence: 99%

Justifying the selection of a neural network linguistic classifier

Barkovska,

Voropaieva,

Ruskikh

2023

ITSSI

View full text Add to dashboard Cite

show abstract

“…The corpus has been scrapped from seven Ukrainian news websites: BBC News Ukraine, NV (New Voice Ukraine), Ukrainian Pravda, Economic Pravda, European Pravda, Life Pravda, and Unian. Ukrainian computer scientists [23] have developed the described corpus. The researchers give an exhaustive outlook on the data preprocessing steps and the number of texts in the train/test split (57789/ 24765, respectively).…”

Section: Methodsmentioning

confidence: 99%

“…The Kaggle platform offers two training splits from the existing sample: large (57460) and small (9299). In their paper, the academics demonstrate their models' performance scores on the two training splits discussed [23]. We are left with the training splits because we cannot use the initial train and test split as it is unavailable to the public.…”

Section: Methodsmentioning

confidence: 99%

“…The obtained results are evaluated with macroaveraged F1-score, the same criterion as in the paper. The baseline model leveraged in the study by [23] was Naïve Bayes with SVM; we have added the StyloMetrix with Voting Classifier as our baseline. The Voting Classifier is composed of RandomForest, AdaBoost, and Logistic Regression.…”

Section: Methodsmentioning

confidence: 99%

“…Ukrainian remains one of the low-resource languages with few practical applications in machine learning and deep learning. Many studies on the Ukrainian language are conducted in terms of multilingual settings, such as training the multilingual large language models [14], [18], transformers [23], [6], or abstractive summarization [10]. We offer a corpus analysis tool for the Ukrainian languagethe StyloMetrix.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

The Grammar and Syntax Based Corpus Analysis Tool For The Ukrainian Language

Stetsenko,

Okulska

2023

Annals of Computer Science and Information Systems

View full text Add to dashboard Cite

This paper provides an overview of a corpus analysis tool -the StyloMetrix for the Ukrainian language. The StyloMetrix incorporates 104 metrics that cover grammatical, stylistic, and syntactic patterns.The idea of constructing the statistical evaluation of syntactic and grammar features is straightforward and familiar for the languages like English, Spanish, German, and others; it is yet to be developed for low-resource languages like Ukrainian. We describe the StyloMetrix pipeline and provide some experiments with this tool for the text classification task. We also describe our package's main limitations and the metrics' evaluation procedure.

show abstract