TOLOMEO, a Novel Machine Learning Algorithm to Measure Information and Order in Correlated Networks and Predict Their State

Abstract

We present ToloMEo (TOpoLogical netwOrk Maximum Entropy Optimization), a program implemented in C and Python that exploits a maximum entropy algorithm to evaluate network topological information. ToloMEo can study any system defined on a connected network where nodes can assume N discrete values by approximating the system probability distribution with a Pottz Hamiltonian on a graph. The software computes entropy through a thermodynamic integration from the mean-field solution to the final distribution. The nature of the algorithm guarantees that the evaluated entropy is variational (i.e., it always provides an upper bound to the exact entropy). The program also performs machine learning, inferring the system’s behavior providing the probability of unknown states of the network. These features make our method very general and applicable to a broad class of problems. Here, we focus on three different cases of study: (i) an agent-based model of a minimal ecosystem defined on a square lattice, where we show how topological entropy captures a crossover between hunting behaviors; (ii) an example of image processing, where starting from discretized pictures of cell populations we extract information about the ordering and interactions between cell types and reconstruct the most likely positions of cells when data are missing; and (iii) an application to recurrent neural networks, in which we measure the information stored in different realizations of the Hopfield model, extending our method to describe dynamical out-of-equilibrium processes.

Computational Approaches to Predict Protein–Protein Interactions in Crowded Cellular Environments

“…The results revealed that positions exhibiting correlated changes in both interacting molecules tend to be near the protein–protein interfaces. This observation opened up the possibility of applying statistical inference techniques based on the maximum entropy principle − to predict pairs of residues that come into contact solely based on their sequence information − and, more specifically, in their evolutionary process.…”

First-Principles Thermodynamics of CsSnI3

“…The properties of interest for this work are related to entropy and pressure as C normalv = T true( ∂ S ∂ T true) V α normalv β T = true( ∂ P ∂ T true) V C normalp = C normalv + V T α normalv true( ∂ P ∂ T true) V where β T is the isothermal compressibility (the inverse of the bulk modulus), and α v is the volumetric expansion coefficient, shown in Figure . Thanks to correlated sampling, ,− we can slightly vary the temperature at a fixed volume without the need for any new DFT calculation, obtaining the free energy and its derivatives (the entropy S and the pressure P ) at temperatures surrounding the simulated one. We estimate the thermodynamic relations in eqs – by employing a finite-difference approach on the correlated sampling simulations (see the SI for more details).…”

“…The properties of interest for this work are related to entropy and pressure as where β T is the isothermal compressibility (the inverse of the bulk modulus), and α v is the volumetric expansion coefficient, shown in Figure 3 . Thanks to correlated sampling, 19 , 33 − 35 we can slightly vary the temperature at a fixed volume without the need for any new DFT calculation, obtaining the free energy and its derivatives (the entropy S and the pressure P ) at temperatures surrounding the simulated one. We estimate the thermodynamic relations in eqs 2 – 4 by employing a finite-difference approach on the correlated sampling simulations (see the SI for more details).…”

Collective behavior and self-organization in neural rosette morphogenesis

“…Colormaps above the plot depict, from left to right, configurations obtained with h = −5, 0, and 5, respectively. (D) Example of an application of a generalized Ising model to the description of the non-equilibrium steady state of a prey-predator model ( Miotto and Monacelli, 2018 ; Miotto and Monacelli, 2021 ). Normalized entropy as a function of the predator motility obtained neglecting interactions (Shannon-Fano) or considering near-neighbor correlations (Max-Ent).…”

“…Varying the external field parameter, h instead modifies the magnetization of the system, as one can see from Figure 5C . Besides its relevance for solid state physics, the Ising model is playing a main role in the area of complex systems as the Hamiltonian (i.e., the energy function) that fully characterize the general Ising or Potts model is the maximum entropy solution for the probability distribution of a system for which densities and correlations are measurable ( Miotto and Monacelli, 2021 ).…”