The use of information and information gain in the analysis of attribute dependencies

Moliński, Krzysztof; Dobek, A.; Tomaszyk, Kamila

doi:10.2478/bile-2013-0011

Cited by 3 publications

(4 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Hydrological systems are characterised with a level of uncertainty [ 1 , 2 ], dispersion or compactness [ 3 , 4 ], uniformity or concentration [ 5 ]. For example, higher entropy is associated with higher dispersion [ 3 , 4 ].…”

Section: Introductionmentioning

confidence: 99%

“…From information theory perspective, Shannon entropy is the average uncertainty of a random variable and gives on average the minimum number of bits needed, to characterise the random variable [ 6 ]. In other words, entropy is the expected value of a random variable called information and is based in the event’s probability [ 1 , 2 ]. The expected surprise about the truth can be an interpretation of entropy-measure for uncertainty [ 7 ] or ignorance [ 8 ].…”

Section: Introductionmentioning

confidence: 99%

“…A system generally moves to a less orderly phase from an orderly phase and from less probable phase to more probable phase, in this way entropy is maximized and a probability set defines the entropy magnitude [ 9 ]. Less credible predictions about a system are usually associated with higher entropy [ 2 ].…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Shannon Entropy for Measuring Spatial Complexity Associated with Mean Annual Runoff of Tertiary Catchments of the Middle Vaal Basin in South Africa

Ilunga

2019

Entropy

View full text Add to dashboard Cite

This study evaluates essentially mean annual runoff (MAR) information gain/loss for tertiary catchments (TCs) in the Middle Vaal basin. Data sets from surface water resources (WR) of South Africa 1990 (WR90), 2005 (WR2005) and 2012 (WR2012) referred in this study as hydrological phases, are used in this evaluation. The spatial complexity level or information redundancy associated with MAR of TCs is derived as well as the relative change in entropy of TCs between hydrological phases. Redundancy and relative change in entropy are shown to coincide under specific conditions. Finally, the spatial distributions of MAR iso-information transmission (i.e., gain or loss) and MAR iso-information redundancy are established for the Middle Vaal basin.

show abstract

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Shannon Entropy for Measuring Spatial Complexity Associated with Mean Annual Runoff of Tertiary Catchments of the Middle Vaal Basin in South Africa

Ilunga

2019

Entropy

View full text Add to dashboard Cite

show abstract

“…It should be noted that there are different measures of dependency (Jakulin, 2005;Moliński, et al, 2012), but in this paper we focus on a measure proposed by Joe (1989). According to Joe (1989), the quotient…”

Section: Measure Of Dependencymentioning

confidence: 99%

Entropy as a measure of dependency for categorized data

Skotarczak

Dobek

Moliński

2018

Biometrical Letters

Self Cite

View full text Add to dashboard Cite

SummaryData arranged in a two-way contingency table can be obtained as a result of many experiments in the life sciences. In some cases the categorized trait is in fact conditioned by an unobservable continuous variable, called liability. It may be interesting to know the relationship between the Pearson correlation coefficient of these two continuous variables and the entropy function measuring the corresponding relation for categorized data. After many simulation trials, a linear regression was estimated between the Pearson correlation coefficient and the normalized mutual information (both on a logarithmic scale). It was observed that the regression coefficients obtained do not depend either on the number of observations classified on a categorical scale or on the continuous random distribution used for the latent variable, but they are influenced by the number of columns in the contingency table. In this paper a known measure of dependency for such data, based on the entropy concept, is applied.

show abstract

Use of entropy in the analysis of nominal traits in sheep

2012

Self Cite

View full text Add to dashboard Cite

In the analysis of dependencies between nominal traits entropy and its function, mutual information seems to be a proper descriptive statistic. This is shown by characterizing the relationships between the prolificacy of dams and selected genetic attributes: the genotype of transferrin, the genotype of hemoglobin, and the type of birth, as well as the environmental attribute, i.e., year of birth. The entropy method may improve the exactitude of investigations concerning the influence of different factors on production trait. The index of relative uniformity, introduced in this study, proved to be an adequate tool for the determination of similarity in the examined flocks. The application of mutual information in the determination of values of the dependence measures in the analyzed experiment was justified.

show abstract

The use of information and information gain in the analysis of attribute dependencies

Cited by 3 publications

References 15 publications

Shannon Entropy for Measuring Spatial Complexity Associated with Mean Annual Runoff of Tertiary Catchments of the Middle Vaal Basin in South Africa

Shannon Entropy for Measuring Spatial Complexity Associated with Mean Annual Runoff of Tertiary Catchments of the Middle Vaal Basin in South Africa

Entropy as a measure of dependency for categorized data

Use of entropy in the analysis of nominal traits in sheep

Contact Info

Product

Resources

About