Extrapolation limitations of multilayer feedforward neural networks

Haley, P.; Soloway, Don

doi:10.1109/ijcnn.1992.227294

Cited by 46 publications

(38 citation statements)

References 1 publication

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A crucial requirement for ANN‐based predictions is that the training data form a representative subset of all the cases we wish to predict. Predictions that fall outside the ranges of input and output values defined by the training data require extrapolation and hence are unreliable [ Haley and Soloway , ]. We used the ANN implementation provided by the free MATLAB toolbox NETLAB [ Nabney , ].…”

Section: Methodsmentioning

confidence: 99%

Estimating the fill thickness and bedrock topography in intermontane valleys using artificial neural networks

et al. 2015

View full text Add to dashboard Cite

Thick sedimentary fills in intermontane valleys are common in formerly glaciated mountain ranges but difficult to quantify. Yet knowledge of the fill thickness distribution could help to estimate sediment budgets of mountain belts and to decipher the role of stored material in modulating sediment flux from the orogen to the foreland. Here we present a new approach to estimate valley fill thickness and bedrock topography based on the geometric properties of a landscape using artificial neural networks. We test the potential of this approach following a four-tiered procedure. First, experiments with synthetic, idealized landscapes show that increasing variability in surface slopes requires successively more complex network configurations. Second, in experiments with artificially filled natural landscapes, we find that fill volumes can be estimated with an error below 20%. Third, in natural examples with valley fill surfaces that have steeply inclined slopes, such as the Unteraar and the Rhône Glaciers in the Swiss Alps, for example, the average deviation of cross-sectional area between the measured and the modeled valley fill is 26% and 27%, respectively. Finally, application of the method to the Rhône Valley, an overdeepened glacial valley in the Swiss Alps, yields a total estimated sediment volume of 97 ± 11 km 3 and an average deviation of crosssectional area between measurements and model estimates of 21.5%. Our new method allows for rapid assessment of sediment volumes in intermontane valleys while eliminating most of the subjectivity that is typically inherent in other methods where bedrock reconstructions are based on digital elevation models.

show abstract

Section: Methodsmentioning

confidence: 99%

Estimating the fill thickness and bedrock topography in intermontane valleys using artificial neural networks

et al. 2015

View full text Add to dashboard Cite

show abstract

“…Developers of new training algorithms and feedforward network architectures frequently test their creations by fitting a polynomial or other simple relationship [see, for example, Almeida (1987), Barton (1991), Haley and Soloway (1992) and Webb and Lowe (1988)j. However, oral tradition has alerted practitioners to training difficulties in such circumstances.…”

Section: Introductionmentioning

confidence: 99%

Why Some Feedforward Networks Cannot Learn Some Polynomials

1994

View full text Add to dashboard Cite

It seems natural to test feedforward networks on deterministic functions. Yet, some simple functions, notably polynomials, present some difficult problems for approximation by feedforward networks. The estimated parameters become unbounded and fail to follow any unique pattern. Furthermore, as the fit to the specified functions becomes closer, numerical problems may develop in the algorithm. This paper explains why these problems occur for polynomials of order less than or equal to the number of hidden units of a feedforward network. We show that other examples occur for functions mathematically related to the network's squashing function. These difficulties do not indicate problems with the training algorithm, but occur as an inherent consequence of the role of the connection weights in feedforward networks.

show abstract

“…The question rises whether such a metamodel is able to predict cases that the network was not trained on. In general, neural networks do not extrapolate well: they are used to fit a function on the provided training data, but outside the subspace populated with training points that function does not necessarily represent the correct relation [13][14][15]. As such, data fed to the network during inference (employing a trained network to make new predictions) must always lie within the training subspace.…”

Section: Introductionmentioning

confidence: 99%

The impact of a reduced training subspace on the prediction accuracy of neural networks for hygrothermal predictions

Tijskens

Janssen

Roels

2020

Journal of Building Performance Simulation

View full text Add to dashboard Cite

Performing a probabilistic assessment of a building component can easily become computationally inhibitive.To solve this issue, the hygrothermal model can be replaced by a metamodel, which mimics the original model with a strongly reduced calculation time. In this paper, convolutional neural networks are used to predict hygrothermal performance. Because neural networks do not extrapolate well outside their training subspace, it is important to select the training data wisely so that the network can be used to predict for a wide variety of cases, while keeping training time as low as possible. The impact of a reduced training subspace is investigated by training a network on a limited number of wall types or exterior climates and evaluate its prediction accuracy for different wall geometries or other climates. The results showed that is indeed possible to train on a well-considered reduced subspace, while maintaining high accuracy, though it does not necessarily save training time.

show abstract

Extrapolation limitations of multilayer feedforward neural networks

Cited by 46 publications

References 1 publication

Estimating the fill thickness and bedrock topography in intermontane valleys using artificial neural networks

Estimating the fill thickness and bedrock topography in intermontane valleys using artificial neural networks

Why Some Feedforward Networks Cannot Learn Some Polynomials

The impact of a reduced training subspace on the prediction accuracy of neural networks for hygrothermal predictions

Contact Info

Product

Resources

About