Noise and the Evolution of Neural Network Modularity

Høverstad, Boye Annfelt

doi:10.1162/artl_a_00016

Cited by 9 publications

(12 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The retina problem ( Figure 3a) is a pattern-recognition task which has been the focus of several previous studies on the evolution of modular neural network structures (Kashtan and Alon, 2005;Clune et al, 2010;Høverstad, 2011;Clune et al, 2013;Huizinga et al, 2014). In this task an 8-bit input is to be classified as 1 or 0.…”

Section: The Retina Problemmentioning

confidence: 99%

Guiding Neuroevolution with Structural Objectives

Ellefsen

Huizinga

Tørresen

2020

Evolutionary Computation

View full text Add to dashboard Cite

The structure and performance of neural networks are intimately connected, and by use of evolutionary algorithms, neural network structures optimally adapted to a given task can be explored. Guiding such neuroevolution with additional objectives related to network structure has been shown to improve performance in some cases, especially when modular neural networks are beneficial. However, apart from objectives aiming to make networks more modular, such structural objectives have not been widely explored. We propose two new structural objectives and test their ability to guide evolving neural networks on two problems which can benefit from decomposition into subtasks. The first structural objective guides evolution to align neural networks with a user-recommended decomposition pattern. Intuitively, this should be a powerful guiding target for problems where human users can easily identify a structure. The second structural objective guides evolution towards a population with a high diversity in decomposition patterns. This results in exploration of many different ways to decompose a problem, allowing evolution to find good decompositions faster. Tests on our target problems reveal that both methods perform well on a problem with a very clear and decomposable structure. However, on a problem where the optimal decomposition is less obvious, the structural diversity objective is found to outcompete other structural objectives -and this technique can even increase performance on problems without any decomposable structure at all.

show abstract

Section: The Retina Problemmentioning

confidence: 99%

Guiding Neuroevolution with Structural Objectives

Ellefsen

Huizinga

Tørresen

2020

Evolutionary Computation

View full text Add to dashboard Cite

show abstract

“…Along with many ongoing theoretical explorations [24–28], modularity-based partitioning has become popular recently in a broad range of applications [24,29–38]. Unlike traditional clustering methods that seek to minimize weighted combinations of the number of edges running between the modules, such as minimum cuts or normalized cuts [13], modularity-driven clustering methods compare each edge against its expected value when clustering nodes into corresponding modules.…”

Section: Introductionmentioning

confidence: 99%

Modularity-based graph partitioning using conditional expected models

2012

View full text Add to dashboard Cite

Modularity-based partitioning methods divide networks into modules by comparing their structure against random networks conditioned to have the same number of nodes, edges, and degree distribution. We propose a novel way to measure modularity and divide graphs, based on conditional probabilities of the edge strength of random networks. We provide closed-form solutions for the expected strength of an edge when it is conditioned on the degrees of the two neighboring nodes, or alternatively on the degrees of all nodes comprising the network. We analytically compute the expected network under the assumptions of Gaussian and Bernoulli distributions. When the Gaussian distribution assumption is violated, we prove that our expression is the best linear unbiased estimator. Finally, we investigate the performance of our conditional expected model in partitioning simulated and real-world networks.

show abstract

“…In recent studies on evolution of modularity, the condition of MVG was mainly applied to logic circuit models with functional components such as AND, OR, and XOR [7,8,20] and to linear mapping models with inputs and outputs [6]. We followed the definition in the linear mapping model because our network model comprises a continuous dynamical system.…”

Section: E Goal Settingmentioning

confidence: 99%

“…Because networks mediate both nominal signals and noise, the link topology of a network affects its robustness to noise [17]. Several computer simulation studies have shown that robustness increases with the emergence of modular networks [18][19][20]. However, there have been few discussions regarding a mathematical framework establishing how robustness to noise can contribute to network evolution.…”

Section: Introductionmentioning

confidence: 99%

Modular network evolution under selection for robustness to noise

Ikemoto

Sekiyama

2014

Phys. Rev. E

View full text Add to dashboard Cite

Real networks often exhibit modularity, which is defined as the degree to which a network can be decomposed into several subnetworks. The question of how a modular network arises is still open to discussion. The leading hypothesis is that high modularity evolves under multiple goals, which are decomposable to subproblems, as well as under the evolutionary constraint that selection prefers sparse links in a network. In the present study, we investigate an alternative evolutionary constraint entailing increased robustness to noise. To examine this, we present noise-interfused network models involving an analytically solvable linear system and biologically inspired nonlinear systems. The models demonstrate that it is possible to evolve a modular network under both modularly changing goal orientations and enhancing robustness to noise, thereby reducing sensitivity to noise. By performing theoretical analyses of linear systems, it is shown that the evolutionary constraint enforces the establishment of well-balanced noise sensitivities of multiple noise sources and leads to a modular network underlying a modular structure in goals. Moreover, computer simulations confirm that the presented mechanisms of modular network evolution are robust to variations of nonlinearity in network functions. Our findings suggest a positive role for the presence of noise in network evolution.

show abstract

Noise and the Evolution of Neural Network Modularity

Cited by 9 publications

References 28 publications

Guiding Neuroevolution with Structural Objectives

Guiding Neuroevolution with Structural Objectives

Modularity-based graph partitioning using conditional expected models

Modular network evolution under selection for robustness to noise

Contact Info

Product

Resources

About