On Relevant Features Selection Based on Information Theory

Forward Selection of Relevant Factors by Means of MDR-EFE Method

“…where the functions appearing in (34) were introduced in Section 2. For any x ∈ X and y ∈ Y, the inequalities L U y (x) ≥ 0, L U y+1 (x) < 0 are satisfied if and only if, for arbitrary δ N (x, y; U) > 0 such that δ N (x, y; U) → 0, as N → ∞, and all sufficiently large N ∈ N, the following inequalities are valid: L U y (x) + δ N (x, y; U) > 0, L U y+1 (x) + δ N (x, y; U) < 0 (the analogous statement is true for inequalities corresponding to coordinates y = m and y = −m in Formula ( 19)).…”

“…The authors of [33] show that the various algorithms employed can be interpreted as procedures based on proper approximations of the certain objective function. In [34] the principle attention is paid to simple models describing the phenomenon of epistasis observed in genetics, when individual factors do not affect the response, and some combinations of them lead to essential effects (in statistics one says "synergy interaction" of factors). Besides we also demonstrated that a number of well-known algorithms, for instance, mRMR (Minimum Redundancy Maximum Relevance) using mutual information and/or interaction information with a sequential procedure for selecting relevant factors can lead to the identification of the desired set with probability which is negligibly small.…”