Analysis of the Full-Size Russian Corpus of Internet Drug Reviews with Complex NER Labeling Using Deep Learning Neural Networks and Language Models

Sboev, Alexander; Sboeva, Sanna; Moloshnikov, Ivan; Gryaznov, Artem; Rybka, Roman; Naumov, Alexander; Selivanov, Anton; Rylkov, Gleb; Ilyin, V. A.

doi:10.3390/app12010491

Cited by 7 publications

(12 citation statements)

References 46 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Current study is founded on the dataset Russian Drug Review Corpus (RDRS) [69]. The dataset contains user reviews on medications from the Otzovik.ru site.…”

Section: Methodsmentioning

confidence: 99%

Accuracy Analysis of the End-to-End Extraction of Related Named Entities from Russian Drug Review Texts by Modern Approaches Validated on English Biomedical Corpora

et al. 2023

View full text Add to dashboard Cite

An extraction of significant information from Internet sources is an important task of pharmacovigilance due to the need for post-clinical drugs monitoring. This research considers the task of end-to-end recognition of pharmaceutically significant named entities and their relations in texts in natural language. The meaning of ”end-to-end“ is that both of the tasks are performed within a single process on the ”raw“ text without annotation. The study is based on the current version of the Russian Drug Review Corpus—a dataset of 3800 review texts from the Russian segment of the Internet. Currently, this is the only corpus in the Russian language appropriate for research of the mentioned type. We estimated the accuracy of the recognition of the pharmaceutically significant entities and their relations in two approaches based on neural-network language models. The first core approach is to sequentially solve tasks of named-entities recognition and relation extraction (the sequential approach). The second one solves both tasks simultaneously with a single neural network (the joint approach). The study includes a comparison of both approaches, along with the hyperparameters selection to maximize resulting accuracy. It is shown that both approaches solve the target task at the same level of accuracy: 52–53% macro-averaged F1−score, which is the current level of accuracy for “end-to-end” tasks on the Russian language. Additionally, the paper presents the results for English open datasets ADE and DDI based on the joint approach, and hyperparameter selection for the modern domain-specific language models. The result is that the achieved accuracies of 84.2% (ADE) and 73.3% (DDI) are comparable or better than other published results for the datasets.

show abstract

“…Current study is founded on the dataset Russian Drug Review Corpus (RDRS) [69]. The dataset contains user reviews on medications from the Otzovik.ru site.…”

Section: Methodsmentioning

confidence: 99%

Accuracy Analysis of the End-to-End Extraction of Related Named Entities from Russian Drug Review Texts by Modern Approaches Validated on English Biomedical Corpora

et al. 2023

View full text Add to dashboard Cite

show abstract

“…In this paper, we propose an End-to-End method to the tasks of NER and RE based on a cascade approach, combining the best solutions based on the pre-trained language models (see Section "Materials and Methods"). To fine-tune the models that are parts of the proposed solution, the Russian Drug Review Corpus (RDRS) has been used [15] (see Section "Dataset"). The set of the named entitiy types used for the study includes: Adverse Drug Reaction (ADR), Drug name, Disease name, Indication, and Source of the medication information, that are the basic set for pharmacovigilance purposes.…”

Section: Anton Selivanovmentioning

confidence: 99%

“…The solution to this problem is based on the approach we proposed in [15], that implements multilabel classification of text tokens according to the BIO scheme. This named entity markup scheme assumes that the first token of the entity receives the tag "B-«entity class name»" (beginning of the entity), subsequent ones "I-«entity class name»" (in the PoS(DLCP2022)014 RE from Texts Containing Pharmacologically Significant Information... Anton Selivanov entity), and tokens that aren't included in the entity get tag "O" (out of context).…”

Section: Named Entity Recognitionmentioning

confidence: 99%

“…2. XLM-RoBERTa-sag [15] -version of the XLM-RoBERTa-large model trained on a large set of untagged drug reviews in Russian, including: RuDReC texts [19] (about 1.4 million texts), and collected reviews on medicines from the site irecommend.ru (about 250 thousand texts)).…”

Section: Anton Selivanovmentioning

confidence: 99%

“…Dataset Our research based on the corpus RDRS [15]. The corpus contains the texts of Internetreviews on the medications from the site Otzovik.ru.…”

Section: Anton Selivanovmentioning

confidence: 99%

See 2 more Smart Citations

Relation Extraction from Texts Containing Pharmacologically Significant Information on base of Multilingual Language Models

Selivanov¹,

Gryaznov²,

Rybka³

et al. 2022

Proceedings of the 6th International Workshop on Deep Learning in Computational Physics — PoS(DLCP2022)

View full text Add to dashboard Cite

In this paper we estimate the accuracy of the relation extraction from texts containing pharmacologically significant information on base of the expanded version of RDRS corpus, which contains texts of internet reviews on medications in Russian. The accuracy of relation extraction is estimated and compared for two multilingual language models: XLM-RoBERTa-large and XLM-RoBERTa-large-sag. Earlier research proved XLM-RoBERTa-large-sag to be the most efficient language model for the previous version of the RDRS dataset for relation extraction using a ground-truth named entities annotation. In the current work we use two-step relation extraction approach: automated named entity recognition and extraction of relations between predicted entities. The implemented approach has given an opportunity to estimate the accuracy of the proposed solution to the relation extraction problem, as well as to estimate the accuracy at each step of the analysis. As a result, it is shown, that multilingual XLM-RoBERTa-large-sag model achieves relation extraction macro-averaged f1-score equals to 86.4% on the ground-truth named entities, 60.1% on the predicted named entities on the new version of the RDRS corpus contained more than 3800 annotated texts. Consequently, implemented approach based on the XLM-RoBERTa-large-sag language model sets the state-of-the-art for considered type of texts in Russian.

show abstract

Automatic identification of pavement cracks in public roads using an optimized deep convolutional neural network model

Chen²,

2023

Phil. Trans. R. Soc. A.

View full text Add to dashboard Cite

The current study aims to improve the efficiency of automatic identification of pavement distress and improve the status quo of difficult identification and detection of pavement distress. First, the identification method of pavement distress and the types of pavement distress are analysed. Then, the design concept of deep learning in pavement distress recognition is described. Finally, the mask region-based convolutional neural network (Mask R-CNN) model is designed and applied in the recognition of road crack distress. The results show that in the evaluation of the model's comprehensive recognition performance, the highest accuracy is 99%, and the lowest accuracy is 95% after the test and evaluation of the designed model in different datasets. In the evaluation of different crack identification and detection methods, the highest accuracy of transverse crack detection is 98% and the lowest accuracy is 95%. In longitudinal crack detection, the highest accuracy is 98% and the lowest accuracy is 92%. In mesh crack detection, the highest accuracy is 98% and the lowest accuracy is 92%. This work not only provides an in-depth reference for the application of deep CNNs in pavement distress recognition but also promotes the improvement of road traffic conditions, thus contributing to the progression of smart cities in the future. This article is part of the theme issue 'Artificial intelligence in failure analysis of transportation infrastructure and materials'.

show abstract

Analysis of the Full-Size Russian Corpus of Internet Drug Reviews with Complex NER Labeling Using Deep Learning Neural Networks and Language Models

Cited by 7 publications

References 46 publications

Accuracy Analysis of the End-to-End Extraction of Related Named Entities from Russian Drug Review Texts by Modern Approaches Validated on English Biomedical Corpora

Accuracy Analysis of the End-to-End Extraction of Related Named Entities from Russian Drug Review Texts by Modern Approaches Validated on English Biomedical Corpora

Relation Extraction from Texts Containing Pharmacologically Significant Information on base of Multilingual Language Models

Automatic identification of pavement cracks in public roads using an optimized deep convolutional neural network model

Contact Info

Product

Resources

About