The Method of Modelling the Mechanism of Random Access Memory of System for Natural Language Processing

Bisikalo, Oleg; Bogach, Ilona; Sholota, Vladyslava V.

doi:10.1109/tcset49122.2020.235477

Cited by 4 publications

(2 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The value (10) characterizes the maximum uncertainty of the matrix X for a defined matrix V. Let us emphasize other useful properties of function (8).…”

Section: ∆ Is Chosen To Ensure Sufficient Variability Of the Resultin...mentioning

confidence: 99%

See 1 more Smart Citation

Small Stochastic Data Compactification Concept Justified in the Entropy Basis

Kovtun,

Zaitseva,

Levashenko

et al. 2023

Entropy

View full text Add to dashboard Cite

Measurement is a typical way of gathering information about an investigated object, generalized by a finite set of characteristic parameters. The result of each iteration of the measurement is an instance of the class of the investigated object in the form of a set of values of characteristic parameters. An ordered set of instances forms a collection whose dimensionality for a real object is a factor that cannot be ignored. Managing the dimensionality of data collections, as well as classification, regression, and clustering, are fundamental problems for machine learning. Compactification is the approximation of the original data collection by an equivalent collection (with a reduced dimension of characteristic parameters) with the control of accompanying information capacity losses. Related to compactification is the data completeness verifying procedure, which is characteristic of the data reliability assessment. If there are stochastic parameters among the initial data collection characteristic parameters, the compactification procedure becomes more complicated. To take this into account, this study proposes a model of a structured collection of stochastic data defined in terms of relative entropy. The compactification of such a data model is formalized by an iterative procedure aimed at maximizing the relative entropy of sequential implementation of direct and reverse projections of data collections, taking into account the estimates of the probability distribution densities of their attributes. The procedure for approximating the relative entropy function of compactification to reduce the computational complexity of the latter is proposed. To qualitatively assess compactification this study undertakes a formal analysis that uses data collection information capacity and the absolute and relative share of information losses due to compaction as its metrics. Taking into account the semantic connection of compactification and completeness, the proposed metric is also relevant for the task of assessing data reliability. Testing the proposed compactification procedure proved both its stability and efficiency in comparison with previously used analogues, such as the principal component analysis method and the random projection method.

show abstract

“…The value (10) characterizes the maximum uncertainty of the matrix X for a defined matrix V. Let us emphasize other useful properties of function (8).…”

Section: ∆ Is Chosen To Ensure Sufficient Variability Of the Resultin...mentioning

confidence: 99%

“…The most widely used method for reducing data dimensionality is singular value decomposition (SVD, [10][11][12]). The matrices obtained as a result of SVD have a very specific interpretation in the machine learning methodology.…”

Section: Outputmentioning

confidence: 99%