Asymptotic Analysis of an Ensemble of Randomly Projected Linear Discriminants

Niyazi, Lama B.; Kammoun, Abla; Dahrouj, Hayssam; Alouini, Mohamed‐Slim; Al-Naffouri, Tareq Y.

doi:10.1109/jsait.2020.3042137

Cited by 4 publications

(4 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…under the growth regime assumptions (a)-(e), it is [20] for proof). Thus, the deterministic equivalent ε is itself a function of deterministic equivalents m0 , m1 , σ2 0 , and σ2 1 which are also functions of only true statistics.…”

Section: A Asymptotic Analysismentioning

confidence: 98%

Weight Vector Tuning and Asymptotic Analysis of Binary Linear Classifiers

Niyazi¹,

Kammoun²,

Dahrouj³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Unlike its intercept, a linear classifier's weight vector cannot be tuned by a simple grid search.Hence, this paper proposes weight vector tuning of a generic binary linear classifier through the parameterization of a decomposition of the discriminant by a scalar which controls the trade-off between conflicting informative and noisy terms. By varying this parameter, the original weight vector is modified in a meaningful way. Applying this method to a number of linear classifiers under a variety of data dimensionality and sample size settings reveals that the classification performance loss due to non-optimal native hyperparameters can be compensated for by weight vector tuning. This yields computational savings as the proposed tuning method reduces to tuning a scalar compared to tuning the native hyperparameter, which may involve repeated weight vector generation along with its burden of optimization, dimensionality reduction, etc., depending on the classifier. It is also found that weight vector tuning significantly improves the performance of Linear Discriminant Analysis (LDA) under high estimation noise. Proceeding from this second finding, an asymptotic study of the misclassification probability of the parameterized LDA classifier in the growth regime where the data dimensionality and sample size are comparable is conducted. Using random matrix theory, the misclassification probability is shown to converge to a quantity that is a function of the true statistics of the data. Additionally, an estimator of the misclassification probability is derived. Finally, computationally efficient tuning of the parameter using this estimator is demonstrated on real data.

show abstract

Section: A Asymptotic Analysismentioning

confidence: 98%

Weight Vector Tuning and Asymptotic Analysis of Binary Linear Classifiers

Niyazi¹,

Kammoun²,

Dahrouj³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…However, the estimator in ( 16) is an inconsistent estimator in the regime where n and p grow at constant rate [24]. To clarify, Fig.…”

Section: The Proposed Consistent Vector-based Mse Estimatormentioning

confidence: 99%

“…Both LW-RSCM and Ell2-RSCM are computationally more efficient than Ell1-RSCM as pointed in [5], especially for the high-dimension setup. For the proposed method, we observe that implementing the VB-MSE estimator according to the steps presented in III-A requires computing (23), (24) and the last term in (22), each of O(p 3 ) complexity. Also, the line search method used to obtain the regularization parameter in the VB-MSE approach increases the time complexity.…”

Section: Time Complexitymentioning

confidence: 99%

Portfolio Optimization Using a Consistent Vector-Based MSE Estimation Approach

et al. 2022

View full text Add to dashboard Cite

This paper is concerned with optimizing the weights of the global minimum-variance portfolio (GMVP) in high-dimensional settings where both observation and population dimensions grow at a bounded ratio. Optimizing the GMVP weights is highly influenced by the data covariance matrix estimation. In a high-dimensional setting, it is well known that the sample covariance matrix is not a proper estimator of the true covariance matrix since it is not invertible when we have fewer observations than the data dimension. Even with more observations, the sample covariance matrix may not be well-conditioned. This paper determines the GMVP weights based on a regularized covariance matrix estimator to overcome the abovementioned difficulties. Unlike other methods, the proper selection of the regularization parameter is achieved by minimizing the mean squared error of an estimate of the noise vector that accounts for the uncertainty in the data mean estimation. Using random-matrix-theory tools, we derive a consistent estimator of the achievable mean squared error that allows us to find the optimal regularization parameter using a simple line search. Simulation results demonstrate the effectiveness of the proposed method when the data dimension is larger than, or of the same order of, the number of data samples.INDEX TERMS Portfolio optimization, global minimum-variance portfolio, GMVP, random matrix theory, RMT, consistent estimator.

show abstract

“…(see Lemma 2 in [22] for proof). Thus, the DE ε is itself a function of DEs m0 , m1 , σ2 0 , and σ2 1 which are also functions of only true statistics.…”

Section: A Asymptotic Analysismentioning

confidence: 99%

Weight Vector Tuning and Asymptotic Analysis of Binary Linear Classifiers

Niyazi

Kammoun

Dahrouj

et al. 2022

IEEE Open J. Signal Process.

Self Cite

View full text Add to dashboard Cite

Unlike its intercept, a linear classifier's weight vector cannot be tuned by a simple grid search. Hence, this paper proposes weight vector tuning of a generic binary linear classifier through the parameterization of a decomposition of the discriminant by a scalar which controls the trade-off between conflicting informative and noisy terms. By varying this parameter, the original weight vector is modified in a meaningful way. Applying this method to a number of linear classifiers under a variety of data dimensionality and sample size settings reveals that the classification performance loss due to non-optimal native hyperparameters can be compensated for by weight vector tuning. This yields computational savings as the proposed tuning method reduces to tuning a scalar compared to tuning the native hyperparameter, which may involve repeated weight vector generation along with its burden of optimization, dimensionality reduction, etc., depending on the classifier. It is also found that weight vector tuning significantly improves the performance of Linear Discriminant Analysis (LDA) under high estimation noise. Proceeding from this second finding, an asymptotic study of the misclassification probability of the parameterized LDA classifier in the growth regime where the data dimensionality and sample size are comparable is conducted. Using random matrix theory, the misclassification probability is shown to converge to a quantity that is a function of the true statistics of the data. Additionally, an estimator of the misclassification probability is derived. Finally, computationally efficient tuning of the parameter using this estimator is demonstrated on real data.

show abstract

Asymptotic Analysis of an Ensemble of Randomly Projected Linear Discriminants

Cited by 4 publications

References 34 publications

Weight Vector Tuning and Asymptotic Analysis of Binary Linear Classifiers

Weight Vector Tuning and Asymptotic Analysis of Binary Linear Classifiers

Portfolio Optimization Using a Consistent Vector-Based MSE Estimation Approach

Weight Vector Tuning and Asymptotic Analysis of Binary Linear Classifiers

Contact Info

Product

Resources

About