Creation of Tools and Algorithms for Assessing the Relevance of Documents

Yerimbetova, Aigerim; Sagnayeva, Saule Kairollivevna; Murzin, Feodor; Tussupov, Jamalbek

doi:10.1109/rpc.2018.8482202

Cited by 2 publications

(2 citation statements)

References 3 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This involves aligning an input sentence with a corresponding example from a database, selecting segments of this sentence, and subsequently adjusting or paraphrasing them to produce the intended translation [6]. The articles under review present the development of new linguistic and algorithmic solutions for information retrieval technologies within search systems, taking into account syntax and elements of semantics, including for Turkic texts [7]. The document provides a detailed description of the method for summarizing Kazakh texts [8], these studies do not solve the problem we address.…”

Section: Introductionmentioning

confidence: 99%

Text Generation Models for Paraphrase on Kazakh Language

Kassenkhan,

Mukazhanov,

Nuralykyzy

et al. 2024

kazutb

View full text Add to dashboard Cite

This study delves into the relatively unexplored domain of natural language processing for the Kazakh language—a language with limited computational resources. The paper dissects the effectiveness of diffusion models and transformers in generating text, specifically paraphrases, which is a critical aspect of machine learning applications such as chatbots, virtual assistants, and automated translation services. The researchers methodically adapt these advanced models to understand and generate Kazakh text, tackling the unique challenges posed by the language's complex morphology. The paper is comprehensive in its approach, covering everything from the initial adaptation of the models to the Kazakh language context, to the creation of specialized tokenizer tools, to the translation and preparation of datasets for effective training. Through rigorous testing and performance analysis, the study identifies the strengths and weaknesses of each model type. This is critical as it informs the direction of future research and model development, with the goal of enhancing the fluency and accuracy of automated Kazakh text generation. The paper also discusses the broader impact of its findings, suggesting that the methodologies and insights gained could inform similar efforts in other low-resource languages, thereby contributing to the global field of NLP. The research concludes with reflections on the implications of their findings for the ongoing development of machine learning technologies, asserting the potential of these technologies to accommodate the intricacies of any language, given the right approach and resources. This work not only advances the technical capabilities for Kazakh text generation but also serves as a testament to the potential of machine learning to bridge language gaps and foster greater digital inclusivity.

show abstract

Section: Introductionmentioning

confidence: 99%

Text Generation Models for Paraphrase on Kazakh Language

Kassenkhan,

Mukazhanov,

Nuralykyzy

et al. 2024

kazutb

View full text Add to dashboard Cite

show abstract

“…Therefore, studies that are devoted to the problems of information search have always been relevant for the Internet and have scientific relevance [3]. First of all, this is due to the enormous amount of information resources.…”

Section: Introductionmentioning

confidence: 99%

Grammatical categories determination for Turkish and Kazakh languages based on machine learning algorithms and fulfilling dictionaries of link grammar parser

Yerimbetova

Tussupova²,

Sambetbayeva

et al. 2021

EEJET

View full text Add to dashboard Cite

This research is aimed at identifying the parts of speech for the Kazakh and Turkish languages in an information retrieval system. The proposed algorithms are based on machine learning techniques. In this paper, we consider the binary classification of words according to parts of speech. We decided to take the most popular machine learning algorithms. In this paper, the following approaches and well-known machine learning algorithms are studied and considered. We defined 7 dictionaries and tagged 135 million words in Kazakh and 9 dictionaries and 50 million words in the Turkish language. The main problem considered in the paper is to create algorithms for the execution of dictionaries of the so-called Link Grammar Parser (LGP) system, in particular for the Kazakh and Turkish languages, using machine learning techniques. The focus of the research is on the review and comparison of machine learning algorithms and methods that have accomplished results on various natural language processing tasks such as grammatical categories determination. For the operation of the LGP system, a dictionary is created in which a connector for each word is indicated – the type of connection that can be created using this word. The authors considered methods of filling in LGP dictionaries using machine learning. The complexities of natural language processing, however, do not exclude the possibility of identifying narrower tasks that can already be solved algorithmically: for example, determining parts of speech or splitting texts into logical groups. However, some features of natural languages significantly reduce the effectiveness of these solutions. Thus, taking into account all word forms for each word in the Kazakh and Turkish languages increases the complexity of text processing by an order of magnitude

show abstract

Creation of Tools and Algorithms for Assessing the Relevance of Documents

Cited by 2 publications

References 3 publications

Text Generation Models for Paraphrase on Kazakh Language

Text Generation Models for Paraphrase on Kazakh Language

Grammatical categories determination for Turkish and Kazakh languages based on machine learning algorithms and fulfilling dictionaries of link grammar parser

Contact Info

Product

Resources

About