Input Variable Selection (IVS) is an essential step in data-driven modelling and is particularly relevant in environmental applications, where potential input variables are often collinear and redundant. While methods for IVS continue to emerge, each has its own advantages and limitations and no single method is best suited to all datasets and modelling purposes. Rigorous evaluation of IVS methods would allow their effectiveness to be properly identified in various circumstances. However, such evaluations are largely neglected due to the lack of guidelines to facilitate consistent and standardised assessment. This work proposes a new evaluation framework, which consists of benchmark datasets with the typical properties of environmental data, a recommended set of evaluation criteria and a website for sharing data and code. The framework is demonstrated on four IVS algorithms commonly used in environmental modelling studies. The results indicate interesting differences in the algorithms' performance that have not been identified previously.Response to Reviewers: Editor I have now received reviews of the above paper and these lead me to recommend that revision according to all the reviewers' comments is necessary. I may not send it back to reviewers, trusting that you will cut it down, otherwise few people will not bother reading it.Response to Editor comment No. 1. We significantly reduced the manuscript length by mostly focusing on Section 2 and 3, as also suggested by reviewer #2. Where possible, we also tried to reduce Section 5. Overall, we obtained a reduction of about 6 pages (from the introduction to the conclusion) with respect to the previous version of the manuscript. Furthermore, we removed Appendix A, since this material can be directly accessed from the framework website. This gives an overall reduction of 21 pages.Another issue is that I'd like it to fit better with EMS being a generic journal and so link to our key outputs. Most citations to EMS papers are to the authors themselves! Just one way to do this is to link with/refer to other key modelling concepts and issues in the journal. For example see the next paragraph.On model evaluation: that it is credible and addressed well. In this connection, I would like you to justify, and if pertinent expand or comment upon, your choice of evaluation metrics and methods among the ones, for example, in the recent EMS Position paper of Bennett et al (2013) on performance evaluation (they propose a 5-step procedure for evaluating the performance of models). You could also add/comment on visual methods and quantitative measures used to examine model quantities and residuals, including visual inspection. There are several other evaluation issues you could address/compare as well and the paper by Robson and cited below presents an excellent example in Section 13 of that paper. One of our aims for EMS is to strengthen the credibility and relevance of the modelling reported and do this whatever the environmental problem sector. That way your paper is mor...

No abstract

No abstract

Abstract. Monthly to seasonal streamflow forecasts provide useful information for a range of water resource management and planning applications. This work focuses on improving such forecasts by considering the following two aspects: (1) state updating to force the models to match observations from the start of the forecast period, and (2) selection of a shorter calibration period that is more representative of the forecast period, compared to a longer calibration period traditionally used. The analysis is undertaken in the context of using streamflow forecasts for environmental flow water management of an open channel drainage network in southern Australia. Forecasts of monthly streamflow are obtained using a conceptual rainfall-runoff model combined with a post-processor error model for uncertainty analysis. This model set-up is applied to two catchments, one with stronger evidence of non-stationarity than the other. A range of metrics are used to assess different aspects of predictive performance, including reliability, sharpness, bias and accuracy. The results indicate that, for most scenarios and metrics, state updating improves predictive performance for both observed rainfall and forecast rainfall sources. Using the shorter calibration period also improves predictive performance, particularly for the catchment with stronger evidence of non-stationarity. The results highlight that a traditional approach of using a long calibration period can degrade predictive performance when there is evidence of non-stationarity. The techniques presented can form the basis for operational monthly streamflow forecasting systems and provide support for environmental decision-making.

No abstract