Forecasting error amending is a universal solution to improve short-term wind power forecasting accuracy no matter what specific forecasting algorithms are applied. The error correction model should be presented considering not only the nonlinear and non-stationary characteristics of forecasting errors but also the field application adaptability problems. The kernel recursive least-squares (KRLS) model is introduced to meet the requirements of online error correction. An iterative error modification approach is designed in this paper to yield the potential benefits of statistical models, including a set of error forecasting models. The teleconnection in forecasting errors from aggregated wind farms serves as the physical background to choose the hybrid regression variables. A case study based on field data is found to validate the properties of the proposed approach. The results show that our approach could effectively extend the modifying horizon of statistical models and has a better performance than the traditional linear method for amending short-term forecasts.