Towards the Validation of Dynamical Models in Regions where there is no Data

Abstract

The creation of computer models is often driven by the need to make predictions in regions where there is no data (i.e. extrapolations). This makes validation challenging as it is difficult to ensure that a model will be suitable when it is applied in a region where there are no observations of the system of interest. The current paper proposes a method that can reveal flaws in a model which may be difficult to identify using traditional approaches for model calibration and validation. The method specifically targets the situation where one is attempting to model a dynamical system that is believed to possess time-invariant calibration parameters. The proposed approach allows these parameters to vary with time, even though it is believed that they are time-invariant. The of such an analysis is to identify key discrepancies - indications that a model has inherent flaws and, as a result, should not be used to influence decisions in regions where there is no data. The proposed method isn't necessarily a predictor of extrapolation performance, rather, it is a stringent test that, the authors believe, should be applied before extrapolation is attempted. The approach could therefore form a useful part of wider validation frameworks in the future.