Augmenting adaptation with retrospective model correction for non-stationary regression problems

Abstract

Abstract-Existing adaptive predictive methods often use multiple adaptive mechanisms as part of their coping strategy in nonstationary environments. We address a scenario when selective deployment of these adaptive mechanisms is possible. In this case, deploying each adaptive mechanism results in different candidate models, and only one of these candidates is chosen to make predictions on the subsequent data. After observing the error of each of candidate, it is possible to revert the current model to the one which had the least error. We call this strategy retrospective model correction. In this work we aim to investigate the benefits of such approach. As a vehicle for the investigation we use an adaptive ensemble method for regression in batch learning mode which employs several adaptive mechanisms to react to changes in the data. Using real world data from the process industry we show empirically that the retrospective model correction is indeed beneficial for the predictive accuracy, especially for the weaker adaptive mechanisms.

Automated adaptation strategies for stream learning

“…Kadlec and Gabrys (2009) present a plug and play architecture for pre-processing, adaptation and prediction which foresees the possibility of using different adaptation methods in a modular fashion, but does not address the method of AM selection. Bakirov et al (2015Bakirov et al ( , 2016 have presented several such methods for AM selection for their adaptive algorithm, which are discussed in detail in Sect. 3.2.…”

“…The next strategy can be used in combination with any of the above strategies as it focuses on the history of the adaptation sequence and retrospectively adapts two steps back. This is called the retrospective model correction (Bakirov et al, 2016). Specifically, we set the current model to the output of the AM at batch k − 1 which would have produced the best estimate in block k:…”

“…This removes the need to design custom adaptive strategies, resulting in automation of adaptation process. In this work we empirically show the viability of the automated adaptation strategies based on cross-validation (Bakirov et al, 2015) with the optional use of retrospective model correction (Bakirov et al, 2016).…”

Multiple adaptive mechanisms for data-driven soft sensors

“…The next strategy can be used in combination with any of the above strategies as it focuses on the history of the adaptation sequence and retrospectively adapts two steps back. This is called the retrospective model correction [33]. Specifically, we estimate which adaptation at batch k − 1 would have produced an optimal estimate in block k:…”

Automated Adaptation Strategies for Stream Learning

“…This removes the need to design custom adaptive strategies which results in automation of adaptation process. In this work we empirically show the viability of the generic adaptive strategies based upon techniques shown in (Bakirov et al, 2015(Bakirov et al, , 2016, specifically a cross-validatory adaptation strategy with the optional use of retrospective model correction.…”