On the Conditional Smooth Rényi Entropy and its Applications in Guessing and Source Coding

Kuzuoka, Shigeaki

doi:10.1109/tit.2019.2937318

Cited by 16 publications

(49 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The motivation for this work is rooted in the diverse information-theoretic applications of Rényi measures [62]. These include (but are not limited to) asymptotically tight bounds on guessing moments [1], informationtheoretic applications such as guessing subject to distortion [2], joint source-channel coding and guessing with application to sequential decoding [3], guessing with a prior access to a malicious oracle [14], guessing while allowing the guesser to give up and declare an error [50], guessing in secrecy problems [56], [75], guessing with limited memory [64], and guessing under source uncertainty [74]; encoding tasks [12], [13]; Bayesian hypothesis testing [8], [67], [79], and composite hypothesis testing [71], [77]; Rényi generalizations of the rejection sampling problem in [35], motivated by the communication complexity in distributed channel simulation, where these generalizations distinguish between causal and non-causal sampler scenarios [52]; Wyner's common information in distributed source simulation under Rényi divergence measures [87]; various other source coding theorems [15], [23], [24], [36], [49], [50], [68], [76], [78], [79], channel coding theorems [4], [5], [26], [60], [66], [78], [79], [86], including coding theorems in quantum information theory…”

Section: Introductionmentioning

confidence: 99%

Tight Bounds on the Rényi Entropy via Majorization with Applications to Guessing and Compression

Sason

2018

Entropy

View full text Add to dashboard Cite

This paper provides tight bounds on the Rényi entropy of a function of a discrete random variable with a finite number of possible values, where the considered function is not one-to-one. To that end, a tight lower bound on the Rényi entropy of a discrete random variable with a finite support is derived as a function of the size of the support, and the ratio of the maximal to minimal probability masses. This work was inspired by the recently published paper by Cicalese et al.,which is focused on the Shannon entropy, and it strengthens and generalizes the results of that paper to Rényi entropies of arbitrary positive orders. In view of these generalized bounds and the works by Arikan and Campbell, non-asymptotic bounds are derived for guessing moments and lossless data compression of discrete memoryless sources.

show abstract

Section: Introductionmentioning

confidence: 99%

Tight Bounds on the Rényi Entropy via Majorization with Applications to Guessing and Compression

Sason

2018

Entropy

View full text Add to dashboard Cite

show abstract

“…For future work we are very interested in further results based on other (additive) entropies, such as Rényi entropies where other guessing bounds are already investigated [19] past their original use in moment inequalities [25][26][27] and other derived problems such as guessing with limited (or no) memory [28].…”

Section: Discussionmentioning

confidence: 99%

Tight and Scalable Side-Channel Attack Evaluations through Asymptotically Optimal Massey-like Inequalities on Guessing Entropy

Tănăsescu

Choudary

Rioul

et al. 2021

Entropy

View full text Add to dashboard Cite

The bounds presented at CHES 2017 based on Massey’s guessing entropy represent the most scalable side-channel security evaluation method to date. In this paper, we present an improvement of this method, by determining the asymptotically optimal Massey-like inequality and then further refining it for finite support distributions. The impact of these results is highlighted for side-channel attack evaluations, demonstrating the improvements over the CHES 2017 bounds.

show abstract

“…Remark 3. The quantity E[ log ( ) ] that appears on the left-hand side of (52) is closely related to the fundamental limits of the guessing problem [37], [38] allowing errors [39] for a source ; see [34, Section IV].…”

Section: B On the Cutoff Operation For Logarithm Of Integer-valued Random Variablementioning

confidence: 99%

Third-Order Asymptotics of Variable-Length Compression Allowing Errors

Sakai

Yavas

Tan

2021

IEEE Trans. Inform. Theory

View full text Add to dashboard Cite

This study investigates the fundamental limits of variable-length compression in which prefix-free constraints are not imposed (i.e., one-to-one codes are studied) and non-vanishing error probabilities are permitted. Due in part to a crucial relation between the variable-length and fixed-length compression problems, our analysis requires a careful and refined analysis of the fundamental limits of fixed-length compression in the setting where the error probabilities are allowed to approach either zero or one polynomially in the blocklength. To obtain the refinements, we employ tools from moderate deviations and strong large deviations. Finally, we provide the third-order asymptotics for the problem of variable-length compression with non-vanishing error probabilities. We show that unlike several other informationtheoretic problems in which the third-order asymptotics are known, for the problem of interest here, the third-order term depends on the permissible error probability.Index Terms-Variable-length compression, Third-order asymptotics, Average codeword lengths, Moderate deviations, Cramér-type large deviations, Strong large deviations *as → ∞ for fixed 0 ≤ ≤ 1, where prefix ( | ) denotes the minimum of average codeword lengths of binary prefixfree codes for i.i.d. copies of in which the error

show abstract

On the Conditional Smooth Rényi Entropy and its Applications in Guessing and Source Coding

Cited by 16 publications

References 28 publications

Tight Bounds on the Rényi Entropy via Majorization with Applications to Guessing and Compression

Tight Bounds on the Rényi Entropy via Majorization with Applications to Guessing and Compression

Tight and Scalable Side-Channel Attack Evaluations through Asymptotically Optimal Massey-like Inequalities on Guessing Entropy

Third-Order Asymptotics of Variable-Length Compression Allowing Errors

Contact Info

Product

Resources

About