Information Theoretic Methods for Variable Selection—A Review

Mielniczuk, Jan

doi:10.3390/e24081079

Cited by 8 publications

(4 citation statements)

References 67 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Along with dealing with the deterministic number of selected factors, there is a research approach based on developing the rules for stopping the procedures used to identify the relevant set. In this regard, we indicate, e.g., article [52], dedicated to information methods for selecting relevant factors. The study of non-discrete stochastic models is also of undoubted interest, see, e.g., [53].…”

Section: Discussionmentioning

confidence: 99%

Forward Selection of Relevant Factors by Means of MDR-EFE Method

Bulinski

2024

Mathematics

View full text Add to dashboard Cite

The suboptimal procedure under consideration, based on the MDR-EFE algorithm, provides sequential selection of relevant (in a sense) factors affecting the studied, in general, non-binary random response. The model is not assumed linear, the joint distribution of the factors vector and response is unknown. A set of relevant factors has specified cardinality. It is proved that under certain conditions the mentioned forward selection procedure gives a random set of factors that asymptotically (with probability tending to one as the number of observations grows to infinity) coincides with the “oracle” one. The latter means that the random set, obtained with this algorithm, approximates the features collection that would be identified, if the joint distribution of the features vector and response were known. For this purpose the statistical estimators of the prediction error functional of the studied response are proposed. They involve a new version of regularization. This permits to guarantee not only the central limit theorem for normalized estimators, but also to find the convergence rate of their first two moments to the corresponding moments of the limiting Gaussian variable.

show abstract

Section: Discussionmentioning

confidence: 99%

Forward Selection of Relevant Factors by Means of MDR-EFE Method

Bulinski

2024

Mathematics

View full text Add to dashboard Cite

show abstract

“…Redundancy is a problem widely studied by researchers, and several information‐theoretic methods have been proposed 19,27–29 in this context. Information theory‐based methods intend to select a subset S of features from the original feature space F, that can optimize the MI

I\left(S;C\right)

while minimizing the redundancy level.…”

Section: Related Workmentioning

confidence: 99%

Feature redundancy removal for text classification using correlated feature subsets

Farek,

Benaidja

2023

Computational Intelligence

View full text Add to dashboard Cite

The curse of high dimensionality in text classification is a worrisome problem that requires efficient and optimal feature selection (FS) methods to improve classification accuracy and reduce learning time. Existing filter‐based FS methods evaluate features independently of other related ones, which can then lead to selecting a large number of redundant features, especially in high‐dimensional datasets, resulting in more learning time and less classification performance, whereas information theory‐based methods aim to maximize feature dependency with the class variable and minimize its redundancy for all selected features, which gradually becomes impractical when increasing the feature space. To overcome the time complexity issue of information theory‐based methods while taking into account the redundancy issue, in this article, we propose a new feature selection method for text classification termed correlation‐based redundancy removal, which aims to minimize the redundancy using subsets of features having close mutual information scores without sequentially seeking already selected features. The idea is that it is not important to assess the redundancy of a dominant feature having high classification information with another irrelevant feature having low classification information and vice‐versa since they are implicitly weakly correlated. Our method, tested on seven datasets using both traditional classifiers (Naive Bayes and support vector machines) and deep learning models (long short‐term memory and convolutional neural networks), demonstrated strong performance by reducing redundancy and improving classification compared to ten competitive metrics.

show abstract

“…The paper suggests using Delayed Mutual Information (Delayed Mutual Information (DMI)) to incorporate the slight time delays among brain areas inside the BN method because Information Theoretic approaches do not make any hypothesis about the dependency between time series [39]. By simulating the delays found in the brain, we hypothesize that we can create a more representational model for a Bayesian Network.…”

Section: Introductionmentioning

confidence: 99%

The Time Lag in Local Field Potential Signals for the Development of its Bayesian Belief Network

Tsukahara,

Junior,

Ruggiero

et al. 2024

Preprint

View full text Add to dashboard Cite

Purpose: : The objective is to suggest time as an important variable to consider in the network model, specifically when discussing causality. Methods: : There is a consideration of the context of functional connectivity because of the time importance of observing the feature inside the neuroscience context. Using a set comprised of three rats Local Field Potentials, a network model was constructed from data using the Bayesian Network method, considering and not taking into account the time delay of communication among brain areas recorded in this study. To this end, there was the appliance of the Delayed Mutual Information method to identify the lag between Local Field Potentials and K2 Score to compare the models. Results: : Bayesian Network depicted the probabilistic relationship among rat´s brain areas. Delayed Mutual Information captured the lag among brain areas, and after its appliance on the Bayesian Network model, posed better results. Conclusion: : The main contribution of this study is to incorporate the minor delays inside the Bayesian Network method, achieved by applying the Delayed Mutual Information method before its application. That supports the neurophysiology dynamics considering an essential feature in the suggested methodology to support the functional connectivity analysis among brain areas.

show abstract

Information Theoretic Methods for Variable Selection—A Review

Cited by 8 publications

References 67 publications

Forward Selection of Relevant Factors by Means of MDR-EFE Method

Forward Selection of Relevant Factors by Means of MDR-EFE Method

Feature redundancy removal for text classification using correlated feature subsets

The Time Lag in Local Field Potential Signals for the Development of its Bayesian Belief Network

Contact Info

Product

Resources

About