A variation of Gallager error-correcting codes is investigated using statistical mechanics. In codes of this type, a given message is encoded into a codeword that comprises Boolean sums of message bits selected by two randomly constructed sparse matrices. The similarity of these codes to Ising spin systems with random interaction makes it possible to assess their typical performance by analytical methods developed in the study of disordered systems. The typical case solutions obtained via the replica method are consistent with those obtained in simulations using belief propagation decoding. We discuss the practical implications of the results obtained and suggest a computationally efficient construction for one of the more practical configurations.

We investigate the performance of parity check codes using the mapping onto Ising spin systems proposed by Sourlas [Nature (London) 339, 693 (1989); Europhys. Lett. 25, 159 (1994)]. We study codes where each parity check comprises products of K bits selected from the original digital message with exactly C checks per message bit. We show, using the replica method, that these codes saturate Shannon's coding bound for K-->infinity when the code rate K/C is finite. We then examine the finite temperature case to assess the use of simulated annealing methods for decoding, study the performance of the finite K case, and extend the analysis to accommodate different types of noisy channels. The connection between statistical physics and belief propagation decoders is discussed and the dynamics of the decoding itself is analyzed. Further insight into new approaches for improving the code performance is given.

The ways in which natural selection can allow the proliferation of cooperative behavior have long been seen as a central problem in evolutionary biology. Most of the literature has focused on interactions between pairs of individuals and on linear public goods games. This emphasis has led to the conclusion that even modest levels of migration would pose a serious problem to the spread of altruism through population viscosity in group structured populations. Here we challenge this conclusion, by analyzing evolution in a framework which allows for complex group interactions and random migration among groups. We conclude that contingent forms of strong altruism that benefits equally all group members, regardless of kinship and without greenbeard effects, can spread when rare under realistic group sizes and levels of migration, due to the assortment of genes resulting only from population viscosity. Our analysis combines group-centric and gene-centric perspectives, allows for arbitrary strength of selection, and leads to extensions of Hamilton’s rule for the spread of altruistic alleles, applicable under broad conditions.

Abstract. -We discuss the connection between information and copula theories by showing that a copula can be employed to decompose the information content of a multivariate distribution into marginal and dependence components, with the latter quantified by the mutual information. We define the information excess as a measure of deviation from a maximum entropy distribution. The idea of marginal invariant dependence measures is also discussed and used to show that empirical linear correlation underestimates the amplitude of the actual correlation in the case of non-Gaussian marginals. The mutual information is shown to provide an upper bound for the asymptotic empirical log-likelihood of a copula. An analytical expression for the information excess of T-copulas is provided, allowing for simple model identification within this family. We illustrate the framework in a financial data set.

We study the online dynamics of learning in fully connected soft committee machines in the student-teacher scenario. The locally optimal modulation function, which determines the learning algorithm, is obtained from a variational argument in such a manner as to maximise the average generalisation error decay per example. Simulations results for the resulting algorithm are presented for a few cases. The symmetric phase plateaux are found to be vastly reduced in comparison to those found when online backpropagation algorithms are used. A discussion of the implementation of these ideas as practical algorithms is given.Key words: neural networks, generalisation, backpropagation, learning algorithms . PACS. #: 87.10.e+10, 05.90.+m, 64.60.Cn Learning how learning occurs in artificial systems has caught the attention of the Statistical Mechanics community in the last decade. This interest was ignited by several reasons, among them, the invention of efficient learning-from-examples methods such as backpropagation, that permit learning in computationally complex machines, to the realisation that ideas from disordered systems, in particular spin glasses, could be applied to the study of attractor as well as feedforward neural networks and to the generalised interest in complex systems with rugged energy landscapes.The main results from the Statistical Mechanics (see e.g. [1-3] ) approach have almost invariantly been obtained in the thermodynamic limit and have benefited from the powerful techniques used to calculate the averages over the disorder introduced by the random nature of the examples.Among several possible approaches to machine learning, online learning [4] has been the subject of an intense research effort due to several factors. In this scheme, examples are used only once, thereby avoiding the need for expensive memory resources, typical of offline methods. This, however, doesn't translate necessarily into poor performance since efficient methods can be devised that have performance comparable to the memory based ones. Furthermore, learning sequentially from single examples has a greater biological flavor than offline processing. While efficiency, computational economy and biological relevance may be the most relevant factors, the theoretical possibility of rather complete analytical studies has also played an important role. If each one of these factors is, by itself, sufficiently important to make online learning an attractive scheme, together they combine to give a most compelling argument for its thorough study.In this letter we present results of the optimisation of online supervised learning in a model consisting of a fully connected multilayer feedforward neural network, in what has become known as the student-teacher scenario. The type of result we present here brings together two separate lines of research that have been recently pursued by several groups.The study of online backpropagation as put forward by Biehl and Schwarze [5] and later developed in [6,7] has permitted the analytical understand...