Cumulative Residual Entropy: A New Measure of Information

We introduce the minimal maximally predictive models ( -machines) of processes generated by certain hidden semi-Markov models. Their causal states are either hybrid discrete-continuous or continuous random variables and causal-state transitions are described by partial differential equations. Closed-form expressions are given for statistical complexities, excess entropies, and differential information anatomy rates. We present a complete analysis of the -machines of continuous-time renewal processes and, then, extend this to processes generated by unifilar hidden semi-Markov models and semi-Markov models. Our information-theoretic analysis leads to new expressions for the entropy rate and the rates of related information measures for these very general continuous-time process classes.

show abstract

“…From the steady-state distribution given in Eq. (16) and from the entropy rate expressions in Eqs. (13)- (14) of Sec.…”

Section: Unifilar Hidden Semi-markov Modelsmentioning

confidence: 99%

Informational and Causal Architecture of Continuous-time Renewal Processes

Marzen

Crutchfield

2017

J Stat Phys

View full text Add to dashboard Cite

show abstract

“…Large amount of following studies have successively introduced the concepts of the expansion of Hartley entropy and Shannon entropy [16], relative entropy [17], cumulative residual entropy [18][19][20][21], joint entropy [22,23], conditional entropy [24][25][26], mutual information [27][28][29][30][31][32], cross entropy [33][34][35][36][37][38], fuzzy entropy [15,39], maximum entropy principle [40,41] and minimum cross-entropy principle [42,43], and a series of achievements have been made in these aspects. Zhong makes use of general information functions to unify the methods of describing information metrics with Entropy formulas [4].…”

Section: About the Metrics Of Informationmentioning

confidence: 99%

Objective information theory: A Sextuple model and 9 kinds of metrics

Shen

et al. 2014

2014 Science and Information Conference

View full text Add to dashboard Cite

In the contemporary era, the importance of information is undisputed, but there has never been a common understanding of information, nor a unanimous conclusion to the researches on information metrics. Based on the previous studies, this paper analyzes the important achievements in the researches of the properties and metrics of information as well as their main insufficiencies, and discusses the essence and connotation, the mathematical expressions and other basic problems related to information. On the basis of the understanding of the objectivity of information, it proposes the definitions and a Sextuple model of objective information; discusses the basic properties of information, and brings forward the definitions and mathematical expressions of nine kinds of metrics of information, i.e., extensity, detailedness, sustainability, richness, containability, delay, distribution, validity and matchability. Through these, this paper establishes a basic theory frame of Objective Information Theory to support the analysis and research on information and information system systematically and comprehensively.

show abstract

“…Accordingly, Ebrahimi (1996) introduced the entropy of the residual lifetime X t = [X − t|X > t] as a dynamic measure of uncertainty. Rao et al (2004) defined a new measure of uncertainty based on the survival function of a random variable X, as E(X) = − ∫ +∞ 0F (x) logF(x)dx, and called it cumulative residual entropy(CRE). This measure can be applied in reliability and image alignment.…”

Section: Introductionmentioning

confidence: 99%

Some Characterization Results Based on Dynamic Survival and Failure Entropies

Abbasnejad¹

2011

Communications for Statistical Applications and Methods

View full text Add to dashboard Cite

In this paper, we develop some characterization results in terms of survival entropy of the first order statistic. In addition, we generalize the cumulative entropy recently proposed by Di Crescenzo and Logobardi (2009) to a new measure of information (called the failure entropy) and study some properties of it and its dynamic version. Furthermore, power distribution is characterized based on dynamic failure entropy.

show abstract

Cumulative Residual Entropy: A New Measure of Information

Cited by 513 publications

References 8 publications

Informational and Causal Architecture of Continuous-time Renewal Processes

Informational and Causal Architecture of Continuous-time Renewal Processes

Objective information theory: A Sextuple model and 9 kinds of metrics

Some Characterization Results Based on Dynamic Survival and Failure Entropies

Contact Info

Product

Resources

About