Pre-processing approaches for imbalanced distributions in regression

Branco, Paula; Torgo, Luı́s; Ribeiro, Rita P.

doi:10.1016/j.neucom.2018.11.100

Cited by 138 publications

(152 citation statements)

References 13 publications

Supporting

Mentioning

151

Contrasting

Unclassified

Order By: Relevance

“…In this respect, learning a multi-dimensional LF raises the issue of learning from highly imbalanced data. This issue is extensively studied in the ML literature for classification problems, but has gathered much less attention from the regression point of view [39][40][41].…”

Section: Learning From Imbalanced Datamentioning

confidence: 99%

The DNNLikelihood: enhancing likelihood distribution with Deep Learning

et al. 2020

View full text Add to dashboard Cite

We introduce the DNNLikelihood, a novel framework to easily encode, through deep neural networks (DNN), the full experimental information contained in complicated likelihood functions (LFs). We show how to efficiently parametrise the LF, treated as a multivariate function of parameters of interest and nuisance parameters with high dimensionality, as an interpolating function in the form of a DNN predictor. We do not use any Gaussian approximation or dimensionality reduction, such as marginalisation or profiling over nuisance parameters, so that the full experimental information is retained. The procedure applies to both binned and unbinned LFs, and allows for an efficient distribution to multiple software platforms, e.g. through the framework-independent ONNX model format. The distributed DNNLikelihood can be used for different use cases, such as re-sampling through Markov Chain Monte Carlo techniques, possibly with custom priors, combination with other LFs, when the correlations among parameters are known, and re-interpretation within different statistical approaches, i.e. Bayesian vs frequentist. We discuss the accuracy of our proposal and its relations with other approximation techniques and likelihood distribution frameworks. As an example, we apply our procedure to a pseudo-experiment corresponding to a realistic LHC search for new physics already considered in the literature.

show abstract

Section: Learning From Imbalanced Datamentioning

confidence: 99%

The DNNLikelihood: enhancing likelihood distribution with Deep Learning

et al. 2020

View full text Add to dashboard Cite

show abstract

“…Applying the SMOTER strategy undersamples the normal values and oversamples the rare values by generating synthetic data points through interpolation between each rare value and a random selection of one of its k-nearest neighbors. 41,42,55 The feature vector and target value of a synthetic instance, + and + , respectively, are determined as follows: 41 41 where , is the feature vector of an instance in ) , 88 is one of k-nearest neighbors of , , ∈ [0, 1] is a random number, , and 88 are the target values of , and 88 , respectively, and , and 88 are the Euclidean distances between + and , , and between + and 88 , respectively. The amount of undersampling and oversampling was automatically determined according to the following options: Optimal hyperparameters were similarly determined by a grid search ( Table 2).…”

Section: Synthetic Minority Oversampling Technique For Regression (Smmentioning

confidence: 99%

“…30,32 Few methods have been proposed for working with imbalanced distributions in regression domains including: SMOTE for regression (SMOTER), 41 SMOGN, 42 meta learning for utility maximization (MetaUtil), 43 resampled bagging (REBAGG), 44 and weighted relevance-based combination strategy (WERCS). 45 In many bioinformatic and cheminformatic supervised-learning regression problems, the data often follows a normal distribution, and the rare extreme values may be more important to the user than the abundant values centered about the median of the distribution. For example, in predicting Topt for practical applications, higher Topt values are generally more relevant since thermostable enzymes are desired for enhanced biochemical reaction rates.…”

Section: Introductionmentioning

confidence: 99%

Improving enzyme optimum temperature prediction with resampling strategies and ensemble learning

Gado

Beckham

Payne

2020

Preprint

View full text Add to dashboard Cite

Accurate prediction of the optimal catalytic temperature (Topt) of enzymes is vital in biotechnology, as enzymes with high Topt values are desired for enhanced reaction rates. Recently, a machine-learning method (TOME) for predicting Topt was developed. TOME was trained on a normally-distributed dataset with a median Topt of 37˚C and less than five percent of Topt values above 85˚C, limiting the method's predictive capabilities for thermostable enzymes. Due to the distribution of the training data, the mean squared error on Topt values greater than 85°C is nearly an order of magnitude higher than the error on values between 30 and 50°C. In this study, we apply ensemble learning and resampling strategies that tackle the data imbalance to significantly decrease the error on high Topt values (>85˚C) by 60% and increase the overall R 2 value from 0.527 to 0.632. The revised method, TOMER, and the resampling strategies applied in this work are freely available to other researchers as a Python package on GitHub.

show abstract

“…Ribeiro's thesis also introduces a new set of performance measures for regression; they substitute the calculated utility in place of the ground truth in the traditional precision and recall metrics, and the F1 measure derived from them. SmoteR is used as the basis for resampling imbalanced time series forecasting problems in Moniz et al Extensions to the SmoteR algorithm include refining the results of utility‐based regression by considering the utility of a point's nearest neighbors, or using Gaussian noise to create synthetic examples if a neighbor is too distant . It should, however, be noted that Torgo and his coauthors employ their utility‐based F1 measure as the sole figure of merit in their experimental evaluations.…”

Section: Literature Reviewmentioning

confidence: 99%

Synthetic minority oversampling for function approximation problems

Pelayo

Dick

2019

Int J Intell Syst

View full text Add to dashboard Cite

Imbalanced data sets are a common occurrence in important machine learning problems. Research in improving learning under imbalanced conditions has largely focused on classification problems (ie, problems with a categorical dependent variable). However, imbalanced data also occur in function approximation, and far less attention has been paid to this case. We present a novel stratification approach for imbalanced function approximation problems. Our solution extends the SMOTE oversampling preprocessing technique to continuous-valued dependent variables by identifying regions of the feature space with a low density of examples and high variance in the dependent variable.Synthetic examples are then generated between nearest neighbors in these regions. In an empirical validation, our approach reduces the normalized mean-squared prediction error in 18 out of 21 benchmark data sets, and compares favorably with state-of-the-art approaches. K E Y W O R D S data mining, learning from imbalanced data sets, machine learning, sample selection bias, stratification PELAYO AND DICK | 2743

show abstract

Pre-processing approaches for imbalanced distributions in regression

Cited by 138 publications

References 13 publications

The DNNLikelihood: enhancing likelihood distribution with Deep Learning

The DNNLikelihood: enhancing likelihood distribution with Deep Learning

Improving enzyme optimum temperature prediction with resampling strategies and ensemble learning

Synthetic minority oversampling for function approximation problems

Contact Info

Product

Resources

About