Training deep neural networks with non-uniform frame-level cost function for automatic speech recognition

Sánchez, Aldonso Becerra; Vargas, José Ismael De la Rosa; González, Efrén; Pedroza, A. David; Escalante, Nivia

doi:10.1007/s11042-018-5917-5

Cited by 21 publications

(6 citation statements)

References 63 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It is hoped that readers involved in the many applications of entropy to the assessment of uncertainty may be intrigued to consider the relevance of extropy to their deliberations. One such application to methods of automatic speech recognition has already appeared in [ 18 ]. Several implications for the analysis of order statistics have been discussed in [ 19 , 20 , 21 ].…”

Section: Discussionmentioning

confidence: 99%

The Duality of Entropy/Extropy, and Completion of the Kullback Information Complex

Lad

Sanfilippo

Agrò

2018

Entropy

View full text Add to dashboard Cite

The refinement axiom for entropy has been provocative in providing foundations of information theory, recognised as thoughtworthy in the writings of both Shannon and Jaynes. A resolution to their concerns has been provided recently by the discovery that the entropy measure of a probability distribution has a dual measure, a complementary companion designated as "extropy". We report here the main results that identify this fact, specifying the dual equations and exhibiting some of their structure. The duality extends beyond a simple assessment of entropy, to the formulation of relative entropy and the Kullback symmetric distance between two forecasting distributions. This is defined by the sum of a pair of directed divergences. Examining the defining equation, we notice that this symmetric measure can be generated by two other explicable pairs of functions as well, neither of which is a Bregman divergence. The Kullback information complex is constituted by the symmetric measure of entropy/extropy along with one of each of these three function pairs. It is intimately related to the total logarithmic score of two distinct forecasting distributions for a quantity under consideration, this being a complete proper score. The information complex is isomorphic to the expectations that the two forecasting distributions assess for their achieved scores, each for its own score and for the score achieved by the other. Analysis of the scoring problem exposes a Pareto optimal exchange of the forecasters' scores that both are willing to engage. Both would support its evaluation for assessing the relative quality of the information they provide regarding the observation of an unknown quantity of interest. We present our results without proofs, as these appear in source articles that are referenced. The focus here is on their content, unhindered. The mathematical syntax of probability we employ relies upon the operational subjective constructions of Bruno de Finetti.

show abstract

Section: Discussionmentioning

confidence: 99%

The Duality of Entropy/Extropy, and Completion of the Kullback Information Complex

Lad

Sanfilippo

Agrò

2018

Entropy

View full text Add to dashboard Cite

show abstract

“…One can refer to [5] for the extropy properties of order statistics and record values. The applications of extropy in automatic speech recognition can be found in [6]. Various literature sources have presented a range of extropy measures and their extensions.…”

Section: Introductionmentioning

confidence: 99%

Estimation of Weighted Extropy with Focus on Its Use in Reliability Modeling

Irshad,

Archana,

Maya

et al. 2024

Entropy

View full text Add to dashboard Cite

show abstract

“…The concept of extropy is useful in many fields: for instance, it is applied in automatic speech recognition [4]. In particular, the extropy of a network output with respect to the training set can be obtained in order to compute a kind of transformed cross entropy.…”

Section: Introductionmentioning

confidence: 99%

Interval extropy and weighted interval extropy

2021

View full text Add to dashboard Cite

Recently, Extropy was introduced by Lad, Sanfilippo and Agrò as a complement dual of Shannon Entropy. In this paper, we propose dynamic versions of Extropy for doubly truncated random variables as measures of uncertainty called Interval Extropy and Weighted Interval Extropy. Some characterizations of random variables related to these new measures are given. Several examples are shown. These measures are evaluated under the effect of linear transformations and, finally, some bounds for them are presented.

show abstract

Training deep neural networks with non-uniform frame-level cost function for automatic speech recognition

Cited by 21 publications

References 63 publications

The Duality of Entropy/Extropy, and Completion of the Kullback Information Complex

The Duality of Entropy/Extropy, and Completion of the Kullback Information Complex

Estimation of Weighted Extropy with Focus on Its Use in Reliability Modeling

Interval extropy and weighted interval extropy

Contact Info

Product

Resources

About