The effect of periodic changes in the fitness landscape on brain structure and function

Nitash, C G; Lundrigan, Barbara L.; Smale, Laura; Hintze, Arend

doi:10.1162/isal_a_00087

Cited by 2 publications

(1 citation statement)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…One approach to determine functional modularity in the context of ANNs is to determine the degree of modularity from the weights that connect nodes [ 22 ], by determining how compartmentalized information is [ 23 , 24 ], or by performing a knockout analysis that allows tasks to be associated with the processing nodes of the neural network [ 25 ]. However, results from such a knockout analysis are often not conclusive.…”

Section: Introductionmentioning

confidence: 99%

Detecting Information Relays in Deep Neural Networks

Hintze

Adami

2023

Entropy

View full text Add to dashboard Cite

Deep learning of artificial neural networks (ANNs) is creating highly functional processes that are, unfortunately, nearly as hard to interpret as their biological counterparts. Identification of functional modules in natural brains plays an important role in cognitive and neuroscience alike, and can be carried out using a wide range of technologies such as fMRI, EEG/ERP, MEG, or calcium imaging. However, we do not have such robust methods at our disposal when it comes to understanding functional modules in artificial neural networks. Ideally, understanding which parts of an artificial neural network perform what function might help us to address a number of vexing problems in ANN research, such as catastrophic forgetting and overfitting. Furthermore, revealing a network’s modularity could improve our trust in them by making these black boxes more transparent. Here, we introduce a new information-theoretic concept that proves useful in understanding and analyzing a network’s functional modularity: the relay information IR. The relay information measures how much information groups of neurons that participate in a particular function (modules) relay from inputs to outputs. Combined with a greedy search algorithm, relay information can be used to identify computational modules in neural networks. We also show that the functionality of modules correlates with the amount of relay information they carry.

show abstract