On the convergence of Shannon differential entropy, and its connections with density and entropy estimation

Silva, Jorge F.; Parada, Patricio

doi:10.1016/j.jspi.2012.02.023

Cited by 9 publications

(4 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Note that μθ converges point-wise to µ * = (1, 0, • • • ) as θ vanishes 5 . However, by the entropy discontinuity [17], [18], [23], the convergence of the measure to µ * is not sufficient to guarantee that lim θ→0 H(μ θ ) = H(µ * ) = 0.…”

Section: Proofmentioning

confidence: 99%

Universal Weak Variable-Length Source Coding on Countably Infinite Alphabets

Silva

Piantanida

2020

IEEE Trans. Inform. Theory

Self Cite

View full text Add to dashboard Cite

Motivated from the fact that universal source coding on countably infinite alphabets is not feasible, this work introduces the notion of "almost lossless source coding". Analog to the weak variable-length source coding problem studied by Han [3], almost lossless source coding aims at relaxing the lossless block-wise assumption to allow an average per-letter distortion that vanishes asymptotically as the blocklength goes to infinity. In this setup, we show on one hand that Shannon entropy characterizes the minimum achievable rate (similarly to the case of discrete sources) while on the other that almost lossless universal source coding becomes feasible for the family of finite-entropy stationary memoryless sources with countably infinite alphabets. Furthermore, we study a stronger notion of almost lossless universality that demands uniform convergence of the average per-letter distortion to zero, where we establish a necessary and sufficient condition for the so-called family of "envelope distributions" to achieve it. Remarkably, this condition is the same necessary and sufficient condition needed for the existence of a strongly minimax (lossless) universal source code for the family of envelope distributions.Finally, we show that an almost lossless coding scheme offers faster rate of convergence for the (minimax) redundancy compared to the well-known information radius developed for the lossless case at the expense of tolerating a non-zero distortion that vanishes to zero as the block-length grows. This shows that even when lossless universality is feasible, an almost lossless scheme can offer different regimes on the rates of convergence of the (worst case) redundancy versus the (worst case) distortion.The material in this paper was partially published in The IEEE 2016 [1] and IEEE 2017 [2] International Symposium on Information Theory (ISIT). J. F. Silva is with the Information and DecisionUniversal source coding, countably infinite alphabets, weak source coding, envelope distributions, information radius, metric entropy analysis.

show abstract

Section: Proofmentioning

confidence: 99%

Universal Weak Variable-Length Source Coding on Countably Infinite Alphabets

Silva

Piantanida

2020

IEEE Trans. Inform. Theory

Self Cite

View full text Add to dashboard Cite

show abstract

“…It compares the entropy of a parent subspace with those of its children's subspaces to find out the optimum level of resolution using the optimum mother wavelet. The criterion states that if the entropy of a signal at a new level is higher than that of the previous level, the decomposition of the signal is not needed [42].…”

Section: Sending Endmentioning

confidence: 99%

A novel approach for distributed renewable generation and shunt capacitor placing in smart-grid

Özgönenel

Karagöl

Terzi

2014

IEEE PES Innovative Smart Grid Technologies, Europe

View full text Add to dashboard Cite

This paper presents a novel method for the selection of appropriate location and calculation of the size of distributed generation (DG) and sizing of shunt capacitors in radial distribution systems. This methodology is based on wavelet neural network (WNN) which requires less computational burden compared to traditional ones. The proposed optimization technique takes into consideration minimizing the cost of peak power, reducing energy loss and improving voltage stability index. The proposed method of optimization is then compared with the golden section search (GSS), grid search (GS), and Acharya's heuristic method (AHM) on different IEEE radial buses, i.e. 12 bus, 12 bus with DG, 33 bus, 33 bus with DG. The results demonstrates that WNN provides more economical solution by reducing power losses, energy loss, total requiring capacitive compensation and presents a good improvement in nodes voltage.

show abstract

“…On this basis, the use of information entropy is widespread in engineering applications. Different types of information entropy have been defined in accordance with their own usage, such as topological entropy of a given interval map [ 14 ], spatial entropy of pixels [ 15 ], weighted multiscale permutation entropy of nonlinear time series [ 16 ], Shannon differential entropy for distributions [ 17 ], min-and max-entropies [ 18 ], collision entropy [ 19 ], permutation entropy [ 20 ], time entropy [ 21 ], multiscale entropy [ 22 ], wavelet entropy [ 23 ] and so on.…”

Section: Introductionmentioning

confidence: 99%

Operational Safety Assessment of Turbo Generators with Wavelet Rényi Entropy from Sensor-Dependent Vibration Signals

Zhang

Wang

Chen

2015

Sensors

View full text Add to dashboard Cite

With the rapid development of sensor technology, various professional sensors are installed on modern machinery to monitor operational processes and assure operational safety, which play an important role in industry and society. In this work a new operational safety assessment approach with wavelet Rényi entropy utilizing sensor-dependent vibration signals is proposed. On the basis of a professional sensor and the corresponding system, sensor-dependent vibration signals are acquired and analyzed by a second generation wavelet package, which reflects time-varying operational characteristic of individual machinery. Derived from the sensor-dependent signals’ wavelet energy distribution over the observed signal frequency range, wavelet Rényi entropy is defined to compute the operational uncertainty of a turbo generator, which is then associated with its operational safety degree. The proposed method is applied in a 50 MW turbo generator, whereupon it is proved to be reasonable and effective for operation and maintenance.

show abstract

On the convergence of Shannon differential entropy, and its connections with density and entropy estimation

Cited by 9 publications

References 20 publications

Universal Weak Variable-Length Source Coding on Countably Infinite Alphabets

Universal Weak Variable-Length Source Coding on Countably Infinite Alphabets

A novel approach for distributed renewable generation and shunt capacitor placing in smart-grid

Operational Safety Assessment of Turbo Generators with Wavelet Rényi Entropy from Sensor-Dependent Vibration Signals

Contact Info

Product

Resources

About