Additive Ensemble Neural Networks

Park, Min-Young; Lee, Seungyeon; Hwang, Sangheum; Kim, Dohyun

doi:10.1109/access.2020.3003748

Cited by 7 publications

(6 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…There are several aggregation strategies, and boosting is one of the most important obtaining state-of-the-art estimators. Bringing together boosting and deep learning has shown very good results in other classification/regression problems [ 9 , 16 ]. The additive ensemble models considered in this work will follow the gaNet architecture [ 9 ], a deep learning boosting ensemble model specifically intended for time-series forecasting.…”

Section: Methodsmentioning

confidence: 99%

“…The only requirement for a base model is to be trainable end-to-end by gradient descent and support the addition of a final layer in both the training and prediction stages. Thus, we have considered as base models several configurations of 1D and 2D convolutional neural networks (CNN) [ 13 , 14 ], long short-term memory (LSTM) [ 15 ] networks and their combination, as well as several additive ensembles (AE) deep learning models especially suitable for time-series forecasting [ 9 , 16 ]. We do not include sequence-to-sequence (Seq2seq) models as a base model since the forward pass for the training, and test stages are different with added complexity for the proposed extension.…”

Section: Introductionmentioning

confidence: 99%

“…It is worth noting the excellent results of the additive ensemble (AE) deep learning models [ 9 , 16 , 30 ] and how they excel in average results and in longer-term (most difficult) forecasts. The good behavior of deep ensembles has lately attracted considerable attention from different points of view.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Additive Ensemble Neural Network with Constrained Weighted Quantile Loss for Probabilistic Electric-Load Forecasting

López-Martín

Sánchez-Esguevillas

Hernández-Callejo

et al. 2021

Sensors

View full text Add to dashboard Cite

This work proposes a quantile regression neural network based on a novel constrained weighted quantile loss (CWQLoss) and its application to probabilistic short and medium-term electric-load forecasting of special interest for smart grids operations. The method allows any point forecast neural network based on a multivariate multi-output regression model to be expanded to become a quantile regression model. CWQLoss extends the pinball loss to more than one quantile by creating a weighted average for all predictions in the forecast window and across all quantiles. The pinball loss for each quantile is evaluated separately. The proposed method imposes additional constraints on the quantile values and their associated weights. It is shown that these restrictions are important to have a stable and efficient model. Quantile weights are learned end-to-end by gradient descent along with the network weights. The proposed model achieves two objectives: (a) produce probabilistic (quantile and interval) forecasts with an associated probability for the predicted target values. (b) generate point forecasts by adopting the forecast for the median (0.5 quantiles). We provide specific metrics for point and probabilistic forecasts to evaluate the results considering both objectives. A comprehensive comparison is performed between a selection of classic and advanced forecasting models with the proposed quantile forecasting model. We consider different scenarios for the duration of the forecast window (1 h, 1-day, 1-week, and 1-month), with the proposed model achieving the best results in almost all scenarios. Additionally, we show that the proposed method obtains the best results when an additive ensemble neural network is used as the base model. The experimental results are drawn from real loads of a medium-sized city in Spain.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Additive Ensemble Neural Network with Constrained Weighted Quantile Loss for Probabilistic Electric-Load Forecasting

López-Martín

Sánchez-Esguevillas

Hernández-Callejo

et al. 2021

Sensors

View full text Add to dashboard Cite

show abstract

“…Examples of leveraging model complexity for better generalisation abound in the machine learning community -the most relatable being better generalisation through efficient dataset utilisation by using an ensemble of less and more complex models on different subsets of data points, each subset characterised by different difficulty of learning (Maini et al, 2022). Historically, the commonly used techniques to improve model generalisability, such as dropout (Srivastava et al, 2014) and pruning (Han et al, 2015), were inspired by reducing model complexity (Park et al, 2020). Thus, a model complexity for traffic prediction tasks furthers the research into better generalisable DL models for traffic prediction.…”

Section: Introductionmentioning

confidence: 99%

Enhancing deep learning-based city-wide traffic prediction pipelines through complexity analysis

Kumar,

Martin,

Raubal

2024

Preprint

View full text Add to dashboard Cite

Deep learning models can effectively capture the non-linear spatiotemporal dynamics of city-wide traffic forecasting. Evidence of varying deep learning model performance between different cities, different prediction horizons, different scales, specific city regions, and during particular hours of the day abounds in the literature on deep learning-based traffic prediction, yet a unified metric to quantify the complexity of different prediction tasks does not exist. This paper proposes two metrics - Model Complexity ($MC$) and Intrinsic Complexity ($IC$). While $MC$ quantifies the complexity of deep learning models for city-wide traffic prediction tasks, the $IC$ quantifies the underlying complexity of the prediction task. Both metrics are validated through systematic experimentation using traffic volume data from three cities. Finally, we demonstrate how these metrics can improve the workflows for deep learning-based data-driven traffic prediction pipelines and deployment by reducing the hyperparameter search scope and comparing the effectiveness of different design pathways.

show abstract

“…Bagging is the most famous representative of the parallel ensemble learning strategy [20]. Bagging can be combined with almost any learning algorithm to form an ensemble learning system, such as a neural network [21] or decision tree [22].…”

mentioning

confidence: 99%

Wind Turbine Condition Monitoring Based on Bagging Ensemble Strategy and KNN Algorithm

Zhang¹,

Niu²,

et al. 2022

IEEE Access

View full text Add to dashboard Cite

The gearbox is an important component of a wind turbine (WT). Once the gearbox is damaged, problems such as long-term maintenance and high maintenance costs will occur. Therefore, it is necessary to carry out on-line condition monitoring (CM) of WTs. Because a large amount of data is accumulated by the supervisory control and data acquisition (SCADA) system, CMs based on data-driven methods have been widely investigated. In this paper, a CM method that is based on the KNN regression method and bagging ensemble strategy is proposed. The proposed method is validated by SCADA data collected from a field WT. The results show that the ensemble model can achieve the desired estimation accuracy and improve the operation efficiency by approximately 30%. INDEX TERMSWind turbine gearbox, data-driven method, condition monitoring, KNN, bagging. I. INTRODUCTIONTo cope with global climate change, China has announced that it will achieve peak carbon dioxide emissions by 2030 and carbon neutrality by 2060 [1]. Therefore, clean energy power generation technology has broad development potential. As a kind of clean energy, wind energy has been widely utilized worldwide. According to the Global Wind Energy Council (GWEC)ś report [2], although the global newly installed capacity reached 93 GW in 2020, a large number of wind turbines (WTs) still need to be installed. With an increase in the number of wind turbines in service and the extension of operation times, the possibility of component failure also increases. According to the statistical data [3], due to the harsh operating environment and other conditions, the gearbox is the component of WTs with a high incidence of faults. Once the gearbox is damaged, problems such as high maintenance costs, complex maintenance processes, and long maintenance times due to structural constraints will ensue [4]. Therefore, it is necessary to carry out online condition monitoring (CM) of gearboxes.The CM of gearboxes is divided into vibration signal analysis [5], oil quality analysis [6] and supervisory control and data acquisition (SCADA) system data analysis [7] The associate editor coordinating the review of this manuscript and approving it for publication was Dipankar Deb . according to different signal sources. However, vibration 33 signal analysis requires the installation of professional 34 sensors to collect high-frequency vibration data, resulting 35 in additional expenses. Oil quality analysis is an invasive 36 method that cannot realize online monitoring. Presently, 37 almost all wind turbines are equipped with the SCADA 38 system [8], which can collect a large amount of operational 39 and record fault data. Therefore, WTCM based on SCADA 40 data has been widely employed by scholars. 41 235 a small constant, so the complexity of the conventional KNN 236 algorithm and bagging integrated KNN algorithm has the 237 same order of magnitude. 238 C. THRESHOLD SETTING METHOD 239 The output of condition monitoring is often a continuous 240 value, but we cannot judge whether the gearbox is faulty...

show abstract

Additive Ensemble Neural Networks

Cited by 7 publications

References 22 publications

Additive Ensemble Neural Network with Constrained Weighted Quantile Loss for Probabilistic Electric-Load Forecasting

Additive Ensemble Neural Network with Constrained Weighted Quantile Loss for Probabilistic Electric-Load Forecasting

Enhancing deep learning-based city-wide traffic prediction pipelines through complexity analysis

Wind Turbine Condition Monitoring Based on Bagging Ensemble Strategy and KNN Algorithm

Contact Info

Product

Resources

About