$$\text {ALR}^n$$ ALR n : accelerated higher-order logistic regression

Zaidi, Nayyar Abbas; Webb, Geoffrey I.; Carman, Mark James; Petitjean, François; Cerquides, Jesús

doi:10.1007/s10994-016-5574-8

Cited by 13 publications

(5 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Recently it has been shown that for big datasets, one can train an LR by building all higherorder features [19]. One can speed-up the convergence of this higher-order LR by integrating the softmax TRON algorithm proposed in this work.…”

Section: Discussionmentioning

confidence: 99%

A Fast Trust-Region Newton Method for Softmax Logistic Regression

Zaidi

Webb

2017

Proceedings of the 2017 SIAM International Conference on Data Mining

Self Cite

View full text Add to dashboard Cite

With the emergence of big data, there has been a growing interest in optimization routines that lead to faster convergence of Logistic Regression (LR). Among many optimization methods such as Gradient Descent, Quasi-Newton, Conjugate Gradient, etc., the Trust-region based truncated Newton method (TRON) algorithm has been shown to converge the fastest. The TRON algorithm also forms an important component of the highly efficient and widely used liblinear package. It has been shown that the WANBIA-C trick of scaling with the log of the naive Bayes conditional probabilities can greatly accelerate the convergence of LR trained using (first-order) Gradient Descent and (approximate secondorder) Quasi-Newton optimization. In this work we study the applicability of the WANBIA-C trick to TRON. We first devise a TRON algorithm optimizing the softmax objective function and then demonstrate that WANBIA-C style preconditioning can be beneficial for TRON, leading to an extremely fast (batch) LR algorithm. Second, we present a comparative analysis of one-vs-all LR and softmax LR in terms of the 0-1 Loss, Bias, Variance, RMSE, Log-Loss, Training and Classification time, and show that softmax LR leads to significantly better RMSE and Log-Loss. We evaluate our proposed approach on 51 benchmark datasets.

show abstract

Section: Discussionmentioning

confidence: 99%

A Fast Trust-Region Newton Method for Softmax Logistic Regression

Zaidi

Webb

2017

Proceedings of the 2017 SIAM International Conference on Data Mining

Self Cite

View full text Add to dashboard Cite

show abstract

“…Furthermore, it is important to address the impact of high correlation among predictor variables, which can lead to challenges related to multicollinearity, potentially affecting the stability and interpretability of models [ 39 ]. However, it is essential to acknowledge that removing predictor variables from the dataset may result in information loss [ 40 ].…”

Section: Literature Reviewmentioning

confidence: 99%

Enhancing credit scoring accuracy with a comprehensive evaluation of alternative data

Hlongwane,

Ramaboa,

Mongwe

2024

PLoS ONE

View full text Add to dashboard Cite

This study explores the potential of utilizing alternative data sources to enhance the accuracy of credit scoring models, compared to relying solely on traditional data sources, such as credit bureau data. A comprehensive dataset from the Home Credit Group’s home loan portfolio is analysed. The research examines the impact of incorporating alternative predictors that are typically overlooked, such as an applicant’s social network default status, regional economic ratings, and local population characteristics. The modelling approach applies the model-X knockoffs framework for systematic variable selection. By including these alternative data sources, the credit scoring models demonstrate improved predictive performance, achieving an area under the curve metric of 0.79360 on the Kaggle Home Credit default risk competition dataset, outperforming models that relied solely on traditional data sources, such as credit bureau data. The findings highlight the significance of leveraging diverse, non-traditional data sources to augment credit risk assessment capabilities and overall model accuracy.

show abstract

“…Again, we follow 1 In case of ties, we assign the average (or fractional) ranking. For example, if there is one winner, two seconds and a loser [1,2,2,4], then the fractional ranking will be [1,2.5,2.5,4]. standard practices for classifier comparison [19] and perform a Nemenyi test to compare pairs of methods.…”

Section: A Cold Startmentioning

confidence: 99%

“…We can influence the variance by adding/removing representation bias to the classifier (see e.g. [4]); adding representation bias typically reduces variance and conversely. However, in many cases it can be easier to express our knowledge of the problem (i.e.…”

Section: Introductionmentioning

confidence: 99%

Generating Synthetic Time Series to Augment Sparse Datasets

Forestier¹,

Petitjean²,

Dau³

et al. 2017

2017 IEEE International Conference on Data Mining (ICDM)

126

View full text Add to dashboard Cite

In machine learning, data augmentation is the process of creating synthetic examples in order to augment a dataset used to learn a model. One motivation for data augmentation is to reduce the variance of a classifier, thereby reducing error. In this paper, we propose new data augmentation techniques specifically designed for time series classification, where the space in which they are embedded is induced by Dynamic Time Warping (DTW). The main idea of our approach is to average a set of time series and use the average time series as a new synthetic example. The proposed methods rely on an extension of DTW Barycentric Averaging (DBA), the averaging technique that is specifically developed for DTW. In this paper, we extend DBA to be able to calculate a weighted average of time series under DTW. In this case, instead of each time series contributing equally to the final average, some can contribute more than others. This extension allows us to generate an infinite number of new examples from any set of given time series. To this end, we propose three methods that choose the weights associated to the time series of the dataset. We carry out experiments on the 85 datasets of the UCR archive and demonstrate that our method is particularly useful when the number of available examples is limited (e.g. 2 to 6 examples per class) using a 1-NN DTW classifier. Furthermore, we show that augmenting full datasets is beneficial in most cases, as we observed an increase of accuracy on 56 datasets, no effect on 7 and a slight decrease on only 22.

show abstract

$$\text {ALR}^n$$ ALR n : accelerated higher-order logistic regression

Cited by 13 publications

References 25 publications

A Fast Trust-Region Newton Method for Softmax Logistic Regression

A Fast Trust-Region Newton Method for Softmax Logistic Regression

Enhancing credit scoring accuracy with a comprehensive evaluation of alternative data

Generating Synthetic Time Series to Augment Sparse Datasets

Contact Info

Product

Resources

About