Handling Imbalanced Datasets by Partially Guided Hybrid Sampling for Pattern Recognition

Sandhan, Tushar; Choi, Jin Young

doi:10.1109/icpr.2014.258

Cited by 38 publications

(10 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The authors utilize concurrent oversampling and undersampling to deal with heavily skewed data distributions. 68 Additionally, LVQ-SMOTE integrates the SMOTE ovesampler with feature codebooks learned by vector quantization to create synthetic samples that use more feature space than the other SMOTE variants. 69 Also, a comparable performance can be obtained using the Polynom-fit SMOTE, which oversamples the minority class using polynomial fitting functions.…”

Section: Oversampling Methodsmentioning

confidence: 99%

“…Moreover, the G‐SMOTE approach pioneered hybrid oversampling that is driven by previously unknown patterns derived from the class with minor samples and randomization. The authors utilize concurrent oversampling and undersampling to deal with heavily skewed data distributions 68 . Additionally, LVQ‐SMOTE integrates the SMOTE ovesampler with feature codebooks learned by vector quantization to create synthetic samples that use more feature space than the other SMOTE variants 69 .…”

Section: End‐to‐end Machine Learning Framework: Design and Implementationmentioning

confidence: 99%

See 1 more Smart Citation

An intelligent framework for end‐to‐end rockfall detection

Zoumpekas

Puig

Salamó

et al. 2021

Int J of Intelligent Sys

View full text Add to dashboard Cite

Rockfall detection is a crucial procedure in the field of geology, which helps to reduce the associated risks. Currently, geologists identify rockfall events almost manually utilizing point cloud and imagery data obtained from different caption devices such as Terrestrial Laser Scanner (TLS) or digital cameras. Multitemporal comparison of the point clouds obtained with these techniques requires a tedious visual inspection to identify rockfall events which implies inaccuracies that depend on several factors such as human expertize and the sensibility of the sensors. This paper addresses this issue and provides an intelligent framework for rockfall event detection for any individual working in the intersection of the geology domain and decision support systems. The development of such an analysis framework presents major research challenges and justifies exhaustive experimental analysis. In particular, we propose an intelligent system that utilizes multiple machine learning algorithms to detect rockfall clusters of point cloud data. Due to the extremely imbalanced nature of the problem, a plethora of state‐of‐the‐art resampling techniques accompanied by multiple models and feature selection procedures are being investigated. Various machine learning pipeline combinations have been examined and benchmarked applying well‐known metrics to be incorporated into our system. Specifically, we developed machine learning techniques and applied them to analyze point cloud data extracted from TLS in two distinct case studies, involving different geological contexts: the basaltic cliff of Castellfollit de la Roca and the conglomerate Montserrat Massif, both located in Spain. Our experimental results indicate that some of the above‐mentioned machine learning pipelines can be utilized to detect rockfall incidents on mountain walls, with experimentally validated accuracy.

show abstract

Section: Oversampling Methodsmentioning

confidence: 99%

Section: End‐to‐end Machine Learning Framework: Design and Implementationmentioning

confidence: 99%

An intelligent framework for end‐to‐end rockfall detection

Zoumpekas

Puig

Salamó

et al. 2021

Int J of Intelligent Sys

View full text Add to dashboard Cite

show abstract

“…Moreover, the G-SMOTE method introduced hybrid oversampling, by the notion of doing partially sampling guided by the hidden patterns obtained from minority class and randomization. Highly skewed data distributions are handled by simultaneous over and undersampling (Sandhan & Choi (2014)). LVQ-SMOTE uses a combination of the SMOTE ovesampler and the feature codebooks obtained by the learning vector quantization in order to generate synthetic samples which occupy more feature space than the other SMOTE variants (Nakamura et al (2013)).…”

Section: Oversampling Methodsmentioning

confidence: 99%

End-to-End Intelligent Framework for Rockfall Detection

Zoumpekas,

Puig,

Salamó

et al. 2021

Preprint

View full text Add to dashboard Cite

Rockfall detection is a crucial procedure in the field of geology, which helps to reduce the associated risks. Currently, geologists identify rockfall events almost manually utilizing point cloud and imagery data obtained from different caption devices such as Terrestrial Laser Scanner or digital cameras. Multi-temporal comparison of the point clouds obtained with these techniques requires a tedious visual inspection to identify rockfall events which implies inaccuracies that depend on several factors such as human expertise and the sensibility of the sensors. This paper addresses this issue and provides an intelligent framework for rockfall event detection for any individual working in the intersection of the geology domain and decision support systems. The development of such an analysis framework poses significant research challenges and justifies intensive experimental analysis. In particular, we propose an intelligent system that utilizes multiple machine learning algorithms to detect rockfall clusters of point cloud data. Due to the extremely imbalanced nature of the problem, a plethora of state-of-the-art resampling techniques accompanied by multiple models and feature selection procedures are being investigated. Various machine learning pipeline combinations have been benchmarked and compared applying well-known metrics to be incorporated into our system. Specifically, we developed statistical and machine learning techniques and applied them to analyze point cloud data extracted from Terrestrial Laser Scanner in two distinct case studies, involving different geological contexts: the basaltic cliff of Castellfollit de la Roca and the conglomerate Montserrat Massif, both located in Spain. Our experimental data suggest that some of the above-mentioned machine learning pipelines can be utilized to detect rockfall incidents on mountain walls, with experimentally proven accuracy.

show abstract

“…In addition to two previously proposed methods utilizing the concept of class potential, Radial-Based Oversampling (RBO) [17] and Radial-Based Undersampling (RBU) [34], we considered several other state-of-the-art resampling strategies. We based our choice on a recent ranking constructed by Kovács [48], out of which we selected the following best-performing methods: SMOTE [18], Polynomial Fitting SMOTE (pf-SMOTE) [49], Oversampling with Rejection (Lee) [50], Synthetic Minority Oversampling Based on Sample Density (SMOBD) [51], Partially Guided Oversampling (G-SMOTE) [52], Learning Vector Quantization-based SMOTE (LVQ-SMOTE) [53], Assembled SMOTE (A-SMOTE) [54] and SMOTE combined with Tomek Links (SMOTE-TL) [55]. With the exception of RBO and RBU, the implementations of the reference methods provided in the smotevariants library [56] were utilized.…”

Section: Set-upmentioning

confidence: 99%

Potential Anchoring for imbalanced data classification

Koziarski¹

2021

Preprint

View full text Add to dashboard Cite

Data imbalance remains one of the factors negatively affecting the performance of contemporary machine learning algorithms. One of the most common approaches to reducing the negative impact of data imbalance is preprocessing the original dataset with data-level strategies. In this paper we propose a unified framework for imbalanced data over-and undersampling. The proposed approach utilizes radial basis functions to preserve the original shape of the underlying class distributions during the resampling process. This is done by optimizing the positions of generated synthetic observations with respect to the potential resemblance loss. The final Potential Anchoring algorithm combines over-and undersampling within the proposed framework. The results of the experiments conducted on 60 imbalanced datasets show outperformance of Potential Anchoring over state-of-the-art resampling algorithms, including previously proposed methods that utilize radial basis functions to model class potential. Furthermore, the results of the analysis based on the proposed data complexity index show that Potential Anchoring is particularly well suited for handling naturally complex (i.e. not affected by the presence of noise) datasets.

show abstract

Handling Imbalanced Datasets by Partially Guided Hybrid Sampling for Pattern Recognition

Cited by 38 publications

References 20 publications

An intelligent framework for end‐to‐end rockfall detection

An intelligent framework for end‐to‐end rockfall detection

End-to-End Intelligent Framework for Rockfall Detection

Potential Anchoring for imbalanced data classification

Contact Info

Product

Resources

About