RussianLanguage Thesauri: Automated Construction and Application For Natural Language Processing Tasks

Abstract

The paper reviews the existing Russian-language thesauri in digital form and methods of their automatic construction and application. The authors analyzed the main characteristics of open access thesauri for scientific research, evaluated trends of their development, and their effectiveness in solving natural language processing tasks. The statistical and linguistic methods of thesaurus construction that allow to automate the development and reduce labor costs of expert linguists were studied. In particular, the authors considered algorithms for extracting keywords and semantic thesaurus relationships of all types, as well as the quality of thesauri generated with the use of these tools. To illustrate features of various methods for constructing thesaurus relationships, the authors developed a combined method that generates a specialized thesaurus fully automatically taking into account a text corpus in a particular domain and several existing linguistic resources. With the proposed method, experiments were conducted with two Russian-language text corpora from two subject areas: articles about migrants and tweets. The resulting thesauri were assessed by using an integrated assessment developed in the previous authors’ study that allows to analyze various aspects of the thesaurus and the quality of the generation methods. The analysis revealed the main advantages and disadvantages of various approaches to the construction of thesauri and the extraction of semantic relationships of different types, as well as made it possible to determine directions for future study.

Keyphrase generation for the Russian-language scientific texts using mT5

“…Генерация ключевых слов для русскоязычных научных текстов с помощью модели mT5 А. В. Глазкова 1,3 , Д. А. Морозов 2,3 , М. С. Воробьева 1 , А. А. Ступников 1 DOI: 10.18255/1818-1015-2023-4-418-428 1 Тюменский государственный университет, ул. Володарского, д.…”

“…Автоматизация подбора ключевых слов представляет собой актуальную задачу в условиях большого количества информационных ресурсов. В настоящее время большинство методов подбора ключевых слов протестировано на англоязычных текстовых корпусах, в то время как для анализа русскоязычных текстов используется достаточно узкий набор методов выделения ключевых слов [1]. Существует несколько подходов к подбору ключевых слов: извлечение ключевых слов непосредственно из текста, подбор ключевых слов из заранее определенного перечня тематик или рубрик и генерация ключевых слов на основе семантики текста путем его обобщения и перефразирования [2,3].…”