Downsampling for Binary Classification with a Highly Imbalanced Dataset Using Active Learning

Lee, Wonjae; Seo, Kangwon

doi:10.1016/j.bdr.2022.100314

Cited by 35 publications

(12 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Such class imbalances increase the likelihood of developing ‘naïve’ learners featuring low false‐positive counts (type I error) but high false‐negative counts (type II error). To mitigate the effects of such a strongly zero‐inflated distribution and potentially decrease type II error, additional BRT and ANN models were generated using a downsampled version of the training dataset with the original number of positive cases, but with 50% of the negative cases randomly excluded (Barros et al, 2019; Lee & Seo, 2021), increasing the proportion of positive cases from 5.8% to 11.0% of complete observations (those without any missing values for any predictor variable).…”

Section: Methodsmentioning

confidence: 99%

Comparison of two machine learning frameworks for predicting aggregatory behaviour of sharks

Wade

Fisher

Matich

2022

Journal of Applied Ecology

View full text Add to dashboard Cite

Ecological monitoring is critical for conservation efforts, particularly in marine ecosystems. Recently, greater emphasis on analyses of data resultant from long‐term ecological research has allowed for broader questions pertaining to climate change, species range shifts and reproductive phenology to be assessed, which may improve ecosystem management. Data collected via ecological monitoring, however, often feature strong class imbalances, complicating the development of models to predict such events. Here, we propose two modelling frameworks—a boosted regression tree (BRT) model and an artificial neural network (ANN)—for predicting exceptionally rare aggregatory behaviour of bull, Carcharhinus leucas and blacktip sharks, C. limbatus along the Gulf coast of Texas. In tandem with aggressive techniques for handling zero‐inflated data, both methods produced accurate predictions of aggregations of three or more individuals in one survey event, with the BRT outperforming the ANN in minimizing type I error. Additionally, both models maintained relatively high area under the receiver operating characteristic (ROC AUC) values when the threshold for defining aggregative behaviour was raised, though there was a measurable drop‐off in the precision‐recall (PR) AUC at each successive threshold increase. These results provide support for both modelling approaches as highly viable in generating predictions based on monitoring data, even in situations where negative cases outweigh positive cases by more than 10‐fold. This is promising for conservation and management of species that exhibit biologically and ecologically significant, but rare, behaviours like aggregations, and species that are rare in abundance and thus vulnerable to future declines. More accurately predicting aggregation events provides the information necessary to improve the protection of species that gather during key life‐history events (e.g. mating, parturition, migration), and assess the spatiotemporal consistency of such events, thereby improving the efficacy of adaptive management. Synthesis and applications. Conservation and management programmes serve a critical role in maintaining the health of ecosystems, and regularly make use of ecological monitoring to collect data relevant to population studies. In marine systems, particularly, monitoring is often expensive and time‐consuming, which can result in incomplete or sparse datasets. Especially where an ecological event of interest is rare, subsequent analyses are particularly affected by the many limitations of zero‐inflated datasets. When training predictive models to classify these events, special care must be taken to avoid simply predicting negative events in every case. Here, we propose and compare two machine learning approaches: gradient boosting and the artificial neural network (ANN), to predict aggregatory behaviour of two shark species: bull, Carcharhinus leucas and blacktip sharks, C. limbatus in the Gulf of Mexico. Applied to a large dataset considering many spatial and environ...

show abstract

Section: Methodsmentioning

confidence: 99%

Comparison of two machine learning frameworks for predicting aggregatory behaviour of sharks

Wade

Fisher

Matich

2022

Journal of Applied Ecology

View full text Add to dashboard Cite

show abstract

“…The metrics are Geometric Mean (GM) and Area Under the ROC Curve (AUC). GM is the result of calculating the geometric mean of sensitivity and specificity [18]. The plot that visualizes the balance achieved between the True Positive rate and the False Positive rate can be seen from the ROC curve [19].…”

Section: Evaluation Metricsmentioning

confidence: 99%

Prediction of Retweets Based on User, Content, and Time Features Using EUSBoost

Khoerunnisa

Jondri²,

Astuti³

2022

J. RESTI (Rekayasa Sist. Teknol. Inf.)

View full text Add to dashboard Cite

Twitter is one of the popular microblogs that allow users to write posts. Retweeting is one of the mechanisms for the diffusion of information on Twitter. One way to understand the spread of information is to learn about retweet predictions. This study focuses on predicting retweets using Evolutionary Undersampling Boosting (EUSBoost) based on user, content, and time-based features. We also consider the vector of text as a predictive feature. Models with EUSBoost are able to outperform models using the AdaBoost method. The evaluation results show that the best model can achieve an AUC performance score of 77.21% and a GM score of 77.18%. While the Adaboost-based models achieved AUC scores ranging from 68% to 69% and GM scores ranging from 62% to 63%. In addition, we found that there was no significant difference between using numeric features only and combining numeric and text features.

show abstract

“…Sebagai catatan penting bahwa sampel informatif dapat berupa minoritas atau mayoritas, bukan hanya memilih sampel mayoritas. Hasil dari pengembangan algorima menunjukkan kinerja yang lebih baik dibandingkan dengan metode resampling lainnya dengan ukuran sampel yang lebih kecil (Lee & Seo, 2022).…”

Section: Pendahuluanunclassified

Sintesis Fitur Density Based Feature Selection (DBFS) dan AdaBoots dengan XGBoost Untuk Meningkatkan Performa Model Prediksi

Nurhendratno

2022

SNST

View full text Add to dashboard Cite

Abstrak Ketidakseimbangan kelas (Class Imbalance) merupakan masalah yang krusial pada performa berbagai algoritma klasifikasi di bidang machine learning. Kelompok kelas dengan banyak data disebut kelas mayoritas, sebaliknya disebut kelas minoritas. Perbandingan antara kelas minoritas dan kelas mayoritas disebut rasio ketidakseimbangan (IR). Semakin besar perbedaan antara kelas minoritas dan kelas mayoritas nilai rasio ketidakseimbangan (IR) semakin besar. Ketidakseimbangan kelas memberikan dampak yang buruk pada hasil klasifikasi dimana kelas minoritas sering disalah klasifikasikan sebagai kelas mayoritas. Ketidak seimbangan kelas akan menghasilkan akurasi prediksi yang baik pada kelas mayoritas tetapi menjadi tidak konduktif dalam memprediksi kelas minoritas, sehingga nilai hasil akurasi pengklasifikasian (classifier) menjadi tidak optimal. Masalah ketidakseimbangan kelas tersebut secara umum dapat ditangani dengan dua pendekatan, yaitu level data dan level algoritma. Pendekatan level data ditujukan untuk memperbaiki keseimbangan kelas, sedangkan pendekatan level algoritma ditujukan untuk memperbaiki algoritma atau menggabungkan (ensemble) pengklasifikasi agar lebih konduktif terhadap kelas minoritas. Beberapa metode telah diusulkan para peneliti untuk memecahkan masalah tersebut seperti metode smote, sampling, cost-sensitive learning, bagging dan boosting. Kebanyakan metode yang dikembangkan hanya pada salah satu level data atau pada level algoritma saja. Maka pada penelitian ini, akan dilakukan kombinasi ensemble baik pada level data maupun pada level algoritma. Pada level data akan menggabungkan metode seleksi fitur (yaitu algoritma Adaptive Boosting (Adaboost) dan metode Density Based Feature Selection (DBFS). Sedangkan pada level algoritma menggunakan salah satu model ensemble klasifikasi XGBoost. Model kombinasi ensemble baik dari level data maupun pada level algoritma tersebut digunakan untuk menagani ketidak seimbangan kelas agar didapatkan performa model prediksi. Penerapan algoritma adaboost dalam seleksi fitur dilakukan untuk memberi bobot pada setiap fitur yang direkomendasikan, sehingga ditemukan fitur yang merupakan classifier yang kuat. Algoritma DBFS berfokus dalam mengidentifikasi kelas minoritas dan mengevaluasi dampak dari sebuah fitur yang bermanfaat berdasarkan rangking fitur. Hasil dari penggabungan (ensemble) kedua algoritma tersebut adalah dataset yang seimbang untuk selanjutnya disintesiskan dengan algoritma XGBoost dalam melakukan perhitungan model prediksi. Hasil prediksi akan di evaluasi dengan confusion matrix dan AUC-ROC. Kata kunci: Ensemble, DBFS, AdaBoost, XGBoost, Confusion Matrix, AUC-ROC

show abstract

Downsampling for Binary Classification with a Highly Imbalanced Dataset Using Active Learning

Cited by 35 publications

References 18 publications

Comparison of two machine learning frameworks for predicting aggregatory behaviour of sharks

Comparison of two machine learning frameworks for predicting aggregatory behaviour of sharks

Prediction of Retweets Based on User, Content, and Time Features Using EUSBoost

Sintesis Fitur Density Based Feature Selection (DBFS) dan AdaBoots dengan XGBoost Untuk Meningkatkan Performa Model Prediksi

Contact Info

Product

Resources

About