Processing Words Effectiveness Analysis in Solving the Natural Language Texts Authorship Determination Task

Demidovich, Inna; Shynkarenko, Viktor; Kuropiatnyk, Olena; Кіріченко, О. В.

doi:10.1109/csit52700.2021.9648829

Cited by 3 publications

(9 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The process of dictionary formation and its form is described in more detail in the previous works of the authors [18].…”

Section: Using Dictionaries To Create Author Profilementioning

confidence: 99%

Methods and software for significant indicators determination of the natural language texts author profile

Shynkarenko,

Demydovych

2023

View full text Add to dashboard Cite

Methods for the formation and optimization of author profiles are presented. The author profile is an image – a vector in a multidimensional space, which components are author’s texts measurements by a number of methods based on 4-grams, stemming, recurrence analysis and formal stochastic grammar. The author’s profile is a model of his language, including vocabulary, sentence syntax features. A comparative analysis of each of the methods effectiveness is carried out. By means of the genetic algorithm, a reduced profile of the author is formed. Insignificant indicators are excluded, which allows to reduce their number by 20%. The reduced author’s profile contains attributes that are significant for this author and is an effective attribution of a particular author.

show abstract

“…The process of dictionary formation and its form is described in more detail in the previous works of the authors [18].…”

Section: Using Dictionaries To Create Author Profilementioning

confidence: 99%

Methods and software for significant indicators determination of the natural language texts author profile

Shynkarenko,

Demydovych

2023

View full text Add to dashboard Cite

show abstract

“…The two identified errors in the stemming process are (4) https://www.indjst.org/ a) Over-stemming b) Under-stemming While using the stemming algorithm in natural language processing, larger words are chopped off to derive their roots. When two words are formed from the same root that originally have different stems, then it is referred to as Over-stemming.…”

Section: • Errors In Stemmingmentioning

confidence: 99%

“…The Stemming process does not consider the phonetics of a word and its resultant outcome after stemming. (4) 4. The Stemmer does not guarantee the true sense of word resulting after the stemming process.…”

Section: • Drawbacks Of Porter Stemmermentioning

confidence: 99%

“…The error is caused due to deletion of letter "e" for the words in which m=1 [VC pair] and ends with two consonants. (3,4)…”

Section: Errors and Suggested Modifications: Error #1mentioning

confidence: 99%

“…Hence, the algorithm continues to display several potential imprecisions and drawbacks. These drawbacks are discussed further in the later sections (4) .…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Enhancement in Stemmer Design: Natural Language Semantics Perspective

Rihan,

Astikar

2023

IJST

View full text Add to dashboard Cite

Objective: To enhance the performance and accuracy of the stemming process. Method: The Porters stemmer is used conventionally for removing common morphological and inflectional endings (suffixes) from the words in the English language. It uses a set of pre-defined rules that are less complex when compared to other existing stemmers. We have identified several imprecisions encountered during the stemming process and proposed solutions to remove and invalidate the same. Findings: The experiment was performed on a set of 762 words starting with characters "a", "b", and "c". It was found that out of 762 words used for system validation and testing, the results of 355 words were different when stemmed with MPS [Modified Porter Stemmer], and the remaining 407 words resulted in the same stemmed word after using both stemmers. The Modified Porter Stemmer presented in the current paper with Python implementation has given better results for 46% of words. Novelty: This paper highlights the encountered errors while using the algorithm and provides solutions to enhance the performance and accuracy of the stemming process. The designed stemmer is named "Modified Porter Stemmer" [MPS].

show abstract

A Dual Approach to Establishing the Authority of Technical Natural Language Texts and Their Components

Shynkarenko,

Demidovich,

Kuropiatnyk

2023

STP

View full text Add to dashboard Cite

Purpose. The study is aimed at testing the hypothesis that it is possible to determine plagiarism by methods of establishing the authorship of a text without using a text bank and their direct comparison. Methodology. Constructive and productive models of the processes of establishing the authorship of technical texts for two methods have been developed. The first method is based on the formation of a text model in the form of a set of formal substitution rules with probabilistic weights (as in stochastic formal grammars), which reflects the syntactic features and patterns of text formation by the author. The degree of similarity between the text under study and another text is determined by comparing their models. The second method is a classical approach to detecting borrowings (plagiarism) by directly comparing the text under study with an existing text bank, highlighting repeated text fragments, and determining the degree of originality. Experiments were conducted to establish the correlation between the results of these two methods. The experimental base consisted of 509 text sections of theses of students majoring in «Software Engineering». Findings. Experimental studies have made it possible to establish a high correlation between the results of the two methods. Correlation coefficients in the range of 0.75...1.0 and with an average value of 0.88 were obtained provided that borrowings are taken into account for text fragments of at least five words in length. Originality. For the first time, the authors have identified the possibilities and proposed methods for indirect plagiarism detection without using a large text bank. The essence of the model is to formalize the representation of the author's sentence syntax by a set of substitution rules with probabilistic weights. Practical value. Based on the results obtained, the possibilities for detecting borrowings have been expanded and the effectiveness of the corresponding methods has been increased. Recommendations on the parameters of classical methods for detecting borrowings have been obtained, in particular, it is recommended to take into account text fragments of at least five words in length as a rational parameter when using borrowing detection systems. The possibilities of text authorship detection methods tested on fiction texts are extended to technical texts.

show abstract

Processing Words Effectiveness Analysis in Solving the Natural Language Texts Authorship Determination Task

Cited by 3 publications

References 11 publications

Methods and software for significant indicators determination of the natural language texts author profile

Methods and software for significant indicators determination of the natural language texts author profile

Enhancement in Stemmer Design: Natural Language Semantics Perspective

A Dual Approach to Establishing the Authority of Technical Natural Language Texts and Their Components

Contact Info

Product

Resources

About