Dynamic node architecture learning: An information theoretic approach

To facilitate decision support in freshwater ecosystem protection and restoration management, habitat suitability models can be very valuable. Data driven methods such as artificial neural networks (ANNs) are particularly useful in this context, seen their time-efficient development and relatively high reliability. However, specialized and technical literature on neural network modelling offers a variety of model development criteria to select model architecture, training procedure, etc. This may lead to confusion among ecosystem modellers and managers regarding the optimal training and validation methodology. This paper focuses on the analysis of ANN development and application for predicting macroinvertebrate communities, a species group commonly used in freshwater assessment worldwide. This review reflects on the different aspects regarding model development and application based on a selection of 26 papers reporting the use of ANN models for the prediction of macroinvertebrates. This analysis revealed that the applied model training and validation methodologies can often be improved and moreover crucial steps in the modelling process are often poorly documented. Therefore, suggestions to improve model development, assessment and application in ecological river management are presented. In particular, data pre-processing determines to a high extent the reliability of the induced models and their predictive relevance. This also counts for the validation criteria, that need to be better tuned to the practical simulation requirements. Moreover, the use of sensitivity methods can help to extract knowledge on the habitat preference of species and allow peer-review by ecological experts. The selection of relevant input variables remains a critical challenge as well. Model coupling is a missing crucial step to link human activities, hydrology, physical habitat conditions, water quality and ecosystem status. This last aspect is probably the most valuable aspect to enable decision support in water management based on ANN models.

show abstract

“…Such techniques were proposed by e.g. Bartlett (1994), Nabhan and Zomaya (1994) and Anders and Korn (1999).…”

Section: Number Of Hidden Neuronsmentioning

confidence: 99%

Applications of artificial neural networks predicting macroinvertebrates in freshwaters

et al. 2007

View full text Add to dashboard Cite

show abstract

“…Although dynamic network architectures have in the past been limited to supervised paradigms [24][25][26][27][28][29], DLVQ recently provided a means of adding nodes by utilising information fed to the network by an external signal. CDUL however, provides an independent dynamic unsupervised network architecture, by using an integral class representation, balanced against computed classifications, to act as a basis upon which new nodes may be generated.…”

Section: Class Directed Unsupervised Learning and Related Networkmentioning

confidence: 99%

Enhancements to the dynamic self-organising neural network

Mackenzie¹

1996

Neural Comput & Applic

View full text Add to dashboard Cite

Class Directed Unsupervised Learning (CD UL) [7,8], having an input layer and a fully connected one-dimensional Kohonen layer which is used to conceptually separate data classes. These networks, along with CDUL, have all been developed from the Kohonen Self-Organising Map (SOM) [9,10] and use variations of the associated Kohonen learning rule. In each of these networks, the Kohonen nodes act as exemplars within an adaptive look-up table, and take on values corresponding to the centroids of each distinct data cluster in the feature space. Where CDUL differs is that the input layer is extended to include a class description of each training vector (see Fig. 1), and it is this additional information that provides CDUL with its interesting properties.The reason for the success of CDUL stems from two factors: firstly, the inclusion of class information as an integral section of the input vector, can be used to prevent Kohonen nodes from being trained by vectors of more than one class; and secondly, the size of the Kohonen layer can be computed

show abstract

“…Approaches to dynamically changing the structure of an MLP during training involve adding or removing weights or neurons and their associated set of weights. Bartlett [4], for example proposed an algorithm that added hidden units each time the training error flattened, and removed units based on an information theoretic measure. He also pointed out that the network weights were often optimised to the structure, and adding new ones didn't allow the network to escape the local optimum it was in.…”

Section: Multi Layer Perceptronsmentioning

confidence: 99%

Structure discovery in mixed order hyper networks

Swingler

2016

Big Data Anal

View full text Add to dashboard Cite

Background: Mixed Order Hyper Networks (MOHNs) are a type of neural network in which the interactions between inputs are modelled explicitly by weights that can connect any number of neurons. Such networks have a human readability that networks with hidden units lack. They can be used for regression, classification or as content addressable memories and have been shown to be useful as fitness function models in constraint satisfaction tasks. They are fast to train and, when their structure is fixed, do not suffer from local minima in the cost function during training. However, their main drawback is that the correct structure (which neurons to connect with weights) must be discovered from data and an exhaustive search is not possible for networks of over around 30 inputs. Results: This paper presents an algorithm designed to discover a set of weights that satisfy the joint constraints of low training error and a parsimonious model. The combined structure discovery and weight learning process was found to be faster, more accurate and have less variance than training an MLP. Conclusions:There are a number of advantages to using higher order weights rather than hidden units in a neural network but discovering the correct structure for those weights can be challenging. With the method proposed in this paper, the use of high order networks becomes tractable.

show abstract

Dynamic node architecture learning: An information theoretic approach

Cited by 52 publications

References 12 publications

Applications of artificial neural networks predicting macroinvertebrates in freshwaters

Applications of artificial neural networks predicting macroinvertebrates in freshwaters

Enhancements to the dynamic self-organising neural network

Structure discovery in mixed order hyper networks

Contact Info

Product

Resources

About