Statistical mechanics of lossy data compression using a nonmonotonic perceptron

Hosaka, Tadaaki; Kabashima, Yoshiyuki; Nishimori, Hidetoshi

doi:10.1103/physreve.66.066126

Cited by 34 publications

(67 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The capacity problem for a BIP is a storage problem, although it can alternatively be seen as a compression task [30]. In the simplest version of the problem, a data set…”

Section: Exemplar Problem: the Binary Ising Perceptronmentioning

confidence: 99%

Replication-based inference algorithms for hard computational problems

2013

View full text Add to dashboard Cite

Inference algorithms based on evolving interactions between replicated solutions are introduced and analyzed on a prototypical NP-hard problem: the capacity of the binary Ising perceptron. The efficiency of the algorithm is examined numerically against that of the parallel tempering algorithm, showing improved performance in terms of the results obtained, computing requirements and simplicity of implementation.

show abstract

“…The capacity problem for a BIP is a storage problem, although it can alternatively be seen as a compression task [30]. In the simplest version of the problem, a data set…”

Section: Exemplar Problem: the Binary Ising Perceptronmentioning

confidence: 99%

Replication-based inference algorithms for hard computational problems

2013

View full text Add to dashboard Cite

show abstract

“…The employment of methods derived from statistical mechanics has resulted in significant progress in providing solutions to several problems in information theory, including problems in error correction [1,2,3,4], spreading codes [5,6] and compression codes [7,8,9,10]. Above all, data compression plays an important role as one of the base technologies in many aspects of information transmission.…”

Section: Introductionmentioning

confidence: 99%

Statistical mechanics of lossy compression using multilayer perceptrons

Mimura

Okada

2006

Phys. Rev. E

View full text Add to dashboard Cite

Statistical mechanics is applied to lossy compression using multilayer perceptrons for unbiased Boolean messages. We utilize a tree-like committee machine (committee tree) and tree-like parity machine (parity tree) whose transfer functions are monotonic. For compression using committee tree, a lower bound of achievable distortion becomes small as the number of hidden units K increases. However, it cannot reach the Shannon bound even where K → ∞. For a compression using a parity tree with K ≥ 2 hidden units, the rate distortion function, which is known as the theoretical limit for compression, is derived where the code length becomes infinity.

show abstract

“…Information theory is a major source of problems, and research activity aimed at solving these problems is becoming popular. Identifying information bits with Ising spins, many problems in information theory, such as error correcting/compression codes [1][2][3][4][5][6][7][8][9][10] and cryptosystems [11,12], can be formulated as virtual many-body systems that are subject to disordered interactions. Analysis of the formulated problems using techniques of statistical mechanics has provided various nontrivial results that have not been obtained by conventional methods of information theory [13,14].…”

mentioning

confidence: 99%

Analysis of CDMA systems that are characterized by eigenvalue spectrum

2006

View full text Add to dashboard Cite

Abstract. -An approach to analyze the performance of the code division multiple access (CDMA) scheme, which is a core technology used in modern wireless communication systems, is provided. The approach characterizes the objective system by the eigenvalue spectrum of a cross-correlation matrix composed of signature sequences used in CDMA communication, which enable us to handle a wider class of CDMA systems beyond the basic model reported by Tanaka in Europhys. Lett., 54 (2001) 540. The utility of the scheme is shown by analyzing a system in which the generation of signature sequences is designed for enhancing the orthogonality.Introduction. -Over the last decade, the scope of statistical mechanics has rapidly expanded beyond its original goal of analyzing many-body problems that arise when dealing with material objects. Information theory is a major source of problems, and research activity aimed at solving these problems is becoming popular. Identifying information bits with Ising spins, many problems in information theory, such as error correcting/compression codes [1-10] and cryptosystems [11,12], can be formulated as virtual many-body systems that are subject to disordered interactions. Analysis of the formulated problems using techniques of statistical mechanics has provided various nontrivial results that have not been obtained by conventional methods of information theory [13,14].Code division multiple access (CDMA), which is a core technology used in modern wireless communication, is an example of the successful application of such a statistical mechanical approach. This technology realizes simultaneous communication between multiple users and a single base station by modulating each user's bit signal (symbol) into a sequence of random pattern, termed the signature sequence [15]. CDMA has already been employed in thirdgeneration mobile phone systems and wireless LANs.Tanaka (2001) showed that the replica method of statistical mechanics enables the accurate assessment of the communication performance of a basic CDMA model in which users' sequences are generated independently of each other in a large system limit [16,17]. This research

show abstract

Statistical mechanics of lossy data compression using a nonmonotonic perceptron

Cited by 34 publications

References 25 publications

Replication-based inference algorithms for hard computational problems

Replication-based inference algorithms for hard computational problems

Statistical mechanics of lossy compression using multilayer perceptrons

Analysis of CDMA systems that are characterized by eigenvalue spectrum

Contact Info

Product

Resources

About