Automated reverse engineering of nonlinear dynamical systems

Abstract

Complex nonlinear dynamics arise in many fields of science and engineering, but uncovering the underlying differential equations directly from observations poses a challenging task. The ability to symbolically model complex networked systems is key to understanding them, an open problem in many disciplines. Here we introduce for the first time a method that can automatically generate symbolic equations for a nonlinear coupled dynamical system directly from time series data. This method is applicable to any system that can be described using sets of ordinary nonlinear differential equations, and assumes that the (possibly noisy) time series of all variables are observable. Previous automated symbolic modeling approaches of coupled physical systems produced linear models or required a nonlinear model to be provided manually. The advance presented here is made possible by allowing the method to model each (possibly coupled) variable separately, intelligently perturbing and destabilizing the system to extract its less observable characteristics, and automatically simplifying the equations during modeling. We demonstrate this method on four simulated and two real systems spanning mechanics, ecology, and systems biology. Unlike numerical models, symbolic models have explanatory value, suggesting that automated ''reverse engineering'' approaches for model-free symbolic nonlinear system identification may play an increasing role in our ability to understand progressively more complex systems in the future.coevolution ͉ modeling ͉ symbolic identification M any branches of science and engineering represent systems that change over time as sets of differential equations, in which each equation describes the rate of change of a single variable as a function of the state of other variables. The structures of these equations are usually determined by hand from first principles, and in some cases regression methods (1-3) are used to identify the parameters of these equations. Nonparametric methods that assume linearity (4) or produce numerical models (5, 6) fail to reveal the full internal structure of complex systems. A key challenge, however, is to uncover the governing equations automatically merely by perturbing and then observing the system in intelligent ways, just as a scientist would do in the presence of an experimental system. Obstacles to achieving this lay in the lack of efficient methods to search the space of symbolic equations and in assuming that precollected data are supplied to the modeling process.Determining the symbolic structure of the governing dynamics of an unknown system is especially challenging when rare yet informative behavior can go unnoticed unless the system is perturbed in very specific ways. Coarse parameter sweeps or random samplings are unlikely to catch these subtleties, and so passive machine learning methods that rely on offline analysis of precollected data may be ineffective. Instead, active learning (7) processes that are able to generate new perturbations or seek out informative part...

Body schema learning for robotic manipulators from visual self-perception

“…In three alternating phases (modeling, testing, prediction), their system generates new structure hypotheses using stochastic optimization, which are validated by generating actions and by analyzing the following sensory input. In a more general context, structure learning was studied in arbitrary non-linear systems using similar mechanisms by Bongard and Lipson (2007).…”

Network Reconstruction Based on Evolutionary-Game Data via Compressive Sensing

“…Examples of potential applications abound: reconstruction of gene-regulatory networks based on expression data in systems biology [1][2][3][4], extraction of various functional networks in the human brain from activation data in neuroscience [5][6][7][8], and uncovering organizational networks based on discrete data or information in social science and homeland defense. In the past few years, the problem of network reconstruction has received growing attention [9][10][11][12][13][14][15][16]. Most existing works were based, however, on networks of oscillators whose dynamics are mathematically described by coupled, continuous differential equations.…”

“…Convex optimization based on L 1 norm has been used for solving network-construction problems in oscillator networks [9][10][11]. Here, we shall show that the compressive-sensing approach provides a solution to network-construction problems (other than oscillator networks) based on the small amount of data from evolutionary games.…”

“…Although convex optimization, of which compressive sensing is one type, has been used to reconstruct coupled-oscillator networks [9][10][11], we shall show the advantages of compressive sensing, such as its small data requirement, in solving the general inverse problem of network reconstruction based on either continuous [29,30] or discrete data. We propose a mathematical framework to convert the problem of uncovering network topology into that of sparse-signal reconstruction.…”

“…Most existing works were based, however, on networks of oscillators whose dynamics are mathematically described by coupled, continuous differential equations. In particular, either some knowledge about the dynamical evolution of the underlying networked system is needed [9][10][11] or long, oscillatory signals in continuous time are required [12][13][14][15][16]. The advantage of availing oneself of continuous-time data is lost for networks in social, economic, and even biological sciences where node-to-node interactions are governed by evolutionary-game types of dynamics [17][18][19][20][21].…”