An artificial modular neural network and its basic dynamical characteristics

Abstract

This work contains a proposition of an artificial modular neural network (MNN) in which every module network exchanges input/output information with others simultaneously. It further studies the basic dynamical characteristics of this network through both computer simulations and analytical considerations. A notable feature of this model is that it has generic representation with regard to the number of composed modules, network topologies, and classes of introduced interactions. The information processing of the MNN is described as the minimization of a total-energy function that consists of partial-energy functions for modules and their interactions, and the activity and weight dynamics are derived from the total-energy function under the Lyapunov stability condition. This concept was realized by Cross-Coupled Hopfield Nets (CCHN) that one of the authors proposed. In this paper, in order to investigate the basic dynamical properties of CCHN, we offer a representative model called Cross-Coupled Hopfield Nets with Local And Global Interactions (CCHN-LAGI) to which two distinct classes of interactions -local and global interactionsare introduced. Through a conventional test for associative memories, it is confirmed that our energy-function-based approach gives us proper dynamics of CCHN-LAGI even if the networks have different modularity. We also discuss the contribution of a single interaction and the joint contribution of the two distinct interactions through the eigenvalue analysis of connection matrices.

Untitled

“…The ANM model we will propose here is obtained from minimizing the energy function which has been defined for a modular neural network called Cross-Coupled Hopfield Nets (CCHN). The CCHN is composed of plural Hopfield networks (HNs) which are mutually connected via feedforward networks called internetworks [8,10,11]. The ANM model is derived from the simplest version of the CCHN: a single-module CCHN with two-layer internetwork (see Figure 1(a)).…”

“…In the context of modular neural networks, these terms are refereed as the energy functions for modules and their interactions. In this paper, we don't care about why such functions should be defined (see[11] for details).NPL97_Ozawa.tex; 30/09/2009; 10:04; no v.; p.4A continuous-time model of autoassociative neural memories…”

A transport equation approach for deep neural networks with quenched random weights

“…Indeed, in the past years, following experimental evidences about the existence of sub-units in brain networks (see e.g. [26][27][28][29]) associative neural networks embedded in modular topologies have attracted much attention [30][31][32][33][34]. Here, exploiting the above-mentioned interpolating techniques, we solve for the free energy at the RS and 1RSB levels of a Hopfield model made of several sub-units, Left: schematic representation of a DBM, made of K = 3 layers (the depth is limited for readability purposes), made of N 1 = N 2 = 3 and N 3 = 2 spins; spins belonging to adjacent layers, say layers p and p + 1 are connected to the hidden neuron z p , thus, the overall number of hidden neurons is L(K − 1).…”

Pattern recognition in Deep Boltzmann machines

“…2 (right panel). Indeed, in the past years, following experimental evidences about the existence of sub-units in brain networks (see e.g., [16,17,18,19]) associative neural networks embedded in modular topologies have attracted much attention [20,21,22,23,24]. Here, exploiting the above-mentioned interpolating techniques, we solve for the free energy of a Hopfield model made of several sub-units, each made of fully-connected neurons.…”