Short lists with short programs in short time

In a lossless compression system with target lengths, a compressor \({\mathscr{C}} \) maps an integer m and a binary string x to an m -bit code p , and if m is sufficiently large, a decompressor \({\mathscr{D}} \) reconstructs x from p . We call a pair ( m , x ) achievable for \(({\mathscr{C}},{\mathscr{D}}) \) if this reconstruction is successful. We introduce the notion of an optimal compressor \({\mathscr{C}}_{\text {opt}} \) by the following universality property: For any compressor-decompressor pair \(({\mathscr{C}}, {\mathscr{D}}) \) , there exists a decompressor \({\mathscr{D}}^{\prime } \) such that if ( m , x ) is achievable for \(({\mathscr{C}},{\mathscr{D}}) \) , then ( m + Δ , x ) is achievable for \(({\mathscr{C}}_\text {opt}, {\mathscr{D}}^{\prime }) \) , where Δ is some small value called the overhead. We show that there exists an optimal compressor that has only polylogarithmic overhead and works in probabilistic polynomial time. Differently said, for any pair \(({\mathscr{C}}, {\mathscr{D}}) \) , no matter how slow \({\mathscr{C}} \) is, or even if \({\mathscr{C}} \) is non-computable, \({\mathscr{C}}_{opt} \) is a fixed compressor that in polynomial time produces codes almost as short as those of \({\mathscr{C}} \) . The cost is that the corresponding decompressor is slower. We also show that each such optimal compressor can be used for distributed compression, in which case it can achieve optimal compression rates as given in the Slepian-Wolf theorem and even for the Kolmogorov complexity variant of this theorem.

show abstract

A Review of Methods for Estimating Algorithmic Complexity: Options, Challenges, and New Directions

Zenil

2020

Entropy

View full text Add to dashboard Cite

Some established and also novel techniques in the field of applications of algorithmic (Kolmogorov) complexity currently co-exist for the first time and are here reviewed, ranging from dominant ones such as statistical lossless compression to newer approaches that advance, complement and also pose new challenges and may exhibit their own limitations. Evidence suggesting that these different methods complement each other for different regimes is presented and despite their many challenges, some of these methods can be better motivated by and better grounded in the principles of algorithmic information theory. It will be explained how different approaches to algorithmic complexity can explore the relaxation of different necessary and sufficient conditions in their pursuit of numerical applicability, with some of these approaches entailing greater risks than others in exchange for greater relevance. We conclude with a discussion of possible directions that may or should be taken into consideration to advance the field and encourage methodological innovation, but more importantly, to contribute to scientific discovery. This paper also serves as a rebuttal of claims made in a previously published minireview by another author, and offers an alternative account.

show abstract

scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.

Contact Info

customersupport@researchsolutions.com

10624 S. Eastern Ave., Ste. A-614

Henderson, NV 89052, USA

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

Blog Terms and Conditions API Terms Privacy Policy Contact Cookie Preferences Do Not Sell or Share My Personal Information

Made with 💙 for researchers

Part of the Research Solutions Family.

Short lists with short programs in short time

Cited by 3 publications

References 24 publications

Searching for shortest and least programs

Searching for shortest and least programs

Universal almost Optimal Compression and Slepian-wolf Coding in Probabilistic Polynomial Time

A Review of Methods for Estimating Algorithmic Complexity: Options, Challenges, and New Directions

Contact Info

Product

Resources

About