BERT for Russian news clustering

Khaustov, S. V.; Gorlova, N. E.; Kalmykov, A. V; Kabaev, A. S.; Moscow, Russia

doi:10.28995/2075-7182-2021-20-385-390

Cited by 2 publications

(2 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We can classify these approaches into two categories. The first category includes traditional machine learning methods [2]. The second category includes deep learning techniques.…”

Section: State Of the Artmentioning

confidence: 99%

See 1 more Smart Citation

Prediction of Author’s Profile basing on Fine-Tuning BERT model

Bsir,

Khoufi,

Zrigui

2024

IJCAI

View full text Add to dashboard Cite

The task of author profiling consists in specifying the infer-demographic features of the social networks' users by studying their published content or the interactions between them. In the literature, many research works were conducted to enhance the accuracy of the techniques used in this process. In fact, the existing methods can be divided into two types: simple linear models and complex deep neural network models. Among them, the transformer-based model exhibited the highest efficiency in NLP analysis in several languages (English, German, French, Turk, Arabic, etc.). Despite their good performance, these approaches do not cover author profiling analysis and, thus, should be further enhanced. So, we propose in this paper a new deep learning strategy by training a customized transformer-model to learn the optimal features of our dataset. In this direction, we fine-tune the model by using the transfer learning approach to improve the results with random initialization. We have achieved about 79% of accuracy by modifying model to apply the retraining process using PAN 2018 authorship dataset.Povzetek: Članek predstavlja novo metodo za napovedovanje avtorjevega profila, ki temelji na modelu Fine-Tuning BERT.

show abstract

“…We can classify these approaches into two categories. The first category includes traditional machine learning methods [2]. The second category includes deep learning techniques.…”

Section: State Of the Artmentioning

confidence: 99%

“…Authors demonstrated the successful models were classifica-tionbased BERT models. However, it turns out clustering embeddings can be almost as effective when trained with correct pooling and loss function [2].…”

Section: State Of the Artmentioning

confidence: 99%

Prediction of Author’s Profile basing on Fine-Tuning BERT model

Bsir,

Khoufi,

Zrigui

2024

IJCAI

View full text Add to dashboard Cite

show abstract

Digital core: neural network recognition of textual geological and geophysical information

Katanov,

Aristov,

Yagafarov

et al. 2023

Izvestiâ vysših učebnyh zavedenij. Neftʹ i gaz

View full text Add to dashboard Cite

The algorithm of analog-to-digital conversion of primary geological and geophysical information (on the example of identification of rock lithotypes based on the text description of the physical core) is presented.As part of the work, a combination of three types of scientific research - prospecting, interdisciplinary and applied, in the formation of the initial base of qualitative data is implemented.Common algorithms for textual information classification and mechanism of initial data preprocessing using tokenization are described.The concept of text pattern recognition is implemented using artificial intelligence methods.For creation of the neural network model of textual geological and geophysical information recognition the Python programming language is used in combination with the convolutional neural network technologies for text classification (TextCNN), bi-directional long-shortterm memory networks (BiLSTM) and bi-directional coder representation networks (BERT).The stack of these technologies and the Python programming language, after developing and testing the basic version of the neural network model of qualitative information recognition, provided an acceptable level of performance of the algorithm of digital transformation of text data.The best result (the current version of neural network model is 1.0; more than 3000 examples for training and testing) was achieved when using the algorithm of text data recognition based on BERT with an accuracy on the validation network (Validation Accuracy) ~0.830173 (25th epoch), with Validation Loss ~0.244719, with Training Loss ~0.000984 and probability of recognition of the studied rock lithotypes more than 95 %.The mechanisms of code modification for further improvement of textual prediction accuracy based on the created neural network were determined.

show abstract

BERT for Russian news clustering

Cited by 2 publications

References 0 publications

Prediction of Author’s Profile basing on Fine-Tuning BERT model

Prediction of Author’s Profile basing on Fine-Tuning BERT model

Digital core: neural network recognition of textual geological and geophysical information

Contact Info

Product

Resources

About