BD-ELM: A Regularized Extreme Learning Machine Using Biased DropConnect and Biased Dropout

Lai, Jie; Wang, Xiaodan; Li, Rui; Song, Yafei; Lei, Lei

doi:10.1155/2020/3604579

Cited by 7 publications

(7 citation statements)

References 16 publications

(21 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The Extreme Learning Machine is a learning algorithm with high speed for the single hidden layer feed-forward neural networks (SLFN) [ 17 ]. The ELM network structure is shown in Fig.…”

Section: Methodsmentioning

confidence: 99%

“…The input weights and the hidden layer biases are determined randomly and only the output layer is trained [ 6 , 17 ].…”

Section: Methodsmentioning

confidence: 99%

“…Thus, the weight matrix values of

calculate by this approach.

where

is used as the estimated value of b and

is the Mooree-Penrose generalized inverse of matrix H [ 6 , 17 ].…”

Section: Methodsmentioning

confidence: 99%

See 2 more Smart Citations

Modeling and forecasting number of confirmed and death caused COVID-19 in IRAN: A comparison of time series forecasting methods

Talkhi

Fatemi

Amirghofran

et al. 2021

Biomedical Signal Processing and Control

View full text Add to dashboard Cite

Background The COVID-19 pandemic conditions are still prevalent in Iran and other countries and the monitoring system is gradually discovering new cases every day. Therefore, it is a cause for concern around the world, and forecasting the number of future patients and death cases, although not entirely accurate, helps the governments and health-policy makers to make the necessary decisions and impose restrictions to reduce prevalence. Methods In this study, we aimed to find the best model for forecasting the number of confirmed and death cases in Iran. For this purpose, we applied nine models including NNETAR, ARIMA, Hybrid, Holt-Winter, BSTS, TBATS, Prophet, MLP, and ELM network models. The quality of forecasting models is evaluated by three performance metrics, RMSE, MAE, and MAPE. The best model is selected by the lowest value of performance metrics. Then, the number of confirmed and the death cases forecasted for the 30 next days. The used data in this study is the absolute number of confirmed, death cases from February 20 to August 15, 2020. Results Our findings suggested that based on existing data in Iran, the suitable model with the lowest performance metrics for confirmed cases data obtained MLP network and the Holt-Winter model is the suitable model for forecasting death cases in the future. These models forecasted on September 14, 2020, we will have 2,484 new confirmed and 114 new death cases of COVID-19. Conclusion According to the results of this study and the existing data, we concluded that the MLP and Holt-Winter models had the lowest error in forecasting in comparison to other methods. Some models had fitted poorly in the test phase and this is because many other factors that are either not available or have been ignored in this study and can affect the accuracy of forecast results. Based on the trend of data and forecast results, the number of confirmed cases and death cases are almost constant and decreasing, respectively. However, due to disease progression and ignoring the recommendations and protocols of the Ministry of health, there is a possibility of re-emerging this disease more seriously in Iran and this requires more preventive care.

show abstract

“…The Extreme Learning Machine is a learning algorithm with high speed for the single hidden layer feed-forward neural networks (SLFN) [ 17 ]. The ELM network structure is shown in Fig.…”

Section: Methodsmentioning

confidence: 99%

“…The input weights and the hidden layer biases are determined randomly and only the output layer is trained [ 6 , 17 ].…”

Section: Methodsmentioning

confidence: 99%

See 1 more Smart Citation

Modeling and forecasting number of confirmed and death caused COVID-19 in IRAN: A comparison of time series forecasting methods

Talkhi

Fatemi

Amirghofran

et al. 2021

Biomedical Signal Processing and Control

View full text Add to dashboard Cite

show abstract

“…Authors of ELM claim that time of assigning weights for ELM is significantly lower than when using standard network training methods and with comparable or even better performance. Extreme learning machines were used as a baseline for many further models, such as stacked ELM [51], on-line sequential ELM [14], LRF-ELM designed for image classification [17], and biased dropout ELM [25]. ELM can also be considered as a non-recurrent equivalent of reservoir computing (RC) models [40].…”

Section: Motivationmentioning

confidence: 99%

Extreme learning machine versus classical feedforward network

Markowska–Kaczmar

Kosturek

2021

Neural Comput & Applic

View full text Add to dashboard Cite

Our research is devoted to answering whether randomisation-based learning can be fully competitive with the classical feedforward neural networks trained using backpropagation algorithm for classification and regression tasks. We chose extreme learning as an example of randomisation-based networks. The models were evaluated in reference to training time and achieved efficiency. We conducted an extensive comparison of these two methods for various tasks in two scenarios: $$\bullet$$ ∙ using comparable network capacity and $$\bullet$$ ∙ using network architectures tuned for each model. The comparison was conducted on multiple datasets from public repositories and some artificial datasets created for this research. Overall, the experiments covered more than 50 datasets. Suitable statistical tests supported the results. They confirm that for relatively small datasets, extreme learning machines (ELM) are better than networks trained by the backpropagation algorithm. But for demanding image datasets, like ImageNet, ELM is not competitive to modern networks trained by backpropagation; therefore, in order to properly address current practical needs in pattern recognition entirely, ELM needs further development. Based on our experience, we postulate to develop smart algorithms for the inverse matrix calculation, so that determining weights for challenging datasets becomes feasible and memory efficient. There is a need to create specific mechanisms to avoid keeping the whole dataset in memory to compute weights. These are the most problematic elements in ELM processing, establishing the main obstacle in the widespread ELM application.

show abstract

“…However, the nodes with the smaller weights tend to learn the noise in the data resulting in poor generalizing capabilities. Reducing overfitting while maintaining a sufficient number of hidden nodes to capture nonlinear input-output relationships using ELM has received a significant amount of attention in recent years (Yu et al, 2014;Shukla et al, ;Feng et al, 2017;Zhou et al, 2018;Duan et al, 2018;Lai et al, 2020).…”

Section: Regularization For Robust Estimation For Hidden Node Selectionmentioning

confidence: 99%

Deep, Wide, or Shallow? Artificial Neural Network Topologies for Predicting Intermittent Flows

Forghanparast

Hernandez

Uddameri

2021

Preprint

View full text Add to dashboard Cite

Abstract. Intermittent Rivers and Ephemeral Streams (IRES) comprise 60 % of all streams in the US and about 50 % of the streams worldwide. Furthermore, climate-driven changes are expected to force a shift towards intermittency in currently perennial streams. Most modeling studies have treated intermittent streamflows as a continuum. However, it is better to envision flow data of IRES as a “mixture-type”, comprised of both flow and no-flow regimes. It is therefore hypothesized that data-driven models with both classification and regression cells can improve the streamflow forecasting abilities in these streams. Deep and wide Artificial Neural Networks (ANNs) comprising of classification and regression cells were developed here by stacking them in series and parallel configurations. These deep and wide network architectures were compared against the commonly used single hidden layer ANNs (shallow), as a baseline, for modeling IRES flow series under the continuum assumption. New metrics focused on no-flow persistence and transitions between flow and no-flow states were formulated using contingency tables and Markov chain analysis. Nine IRES across the state of Texas, US, were used as a wide range of testbeds with different hydro-climatic characteristics. Model overfitting and the curse-of-dimensionality were reduced using extreme learning machines (ELM), and balancing training data using the synthetic minority oversampling technique (SMOTE), greedy learning and Least Absolute Shrinkage and Selection Operator (LASSO). The addition of classifier cells greatly improved the ability to distinguish between no-flow and flow states, in turn, improving the ability to capture no-flow persistence (dryness) and transitions to and from flow states (dryness initiation and cessation). The wide network topology provided better results when the focus was on capturing low flows and the deep topology did well in capturing extreme flows (zero and > 75th percentile).

show abstract

BD-ELM: A Regularized Extreme Learning Machine Using Biased DropConnect and Biased Dropout

Cited by 7 publications

References 16 publications

Modeling and forecasting number of confirmed and death caused COVID-19 in IRAN: A comparison of time series forecasting methods

Modeling and forecasting number of confirmed and death caused COVID-19 in IRAN: A comparison of time series forecasting methods

Extreme learning machine versus classical feedforward network

Deep, Wide, or Shallow? Artificial Neural Network Topologies for Predicting Intermittent Flows

Contact Info

Product

Resources

About