Truncated $L_1$ Regularized Linear Regression: Theory and Algorithm

Dai, Mingwei

doi:10.4208/cicp.oa-2020-0250

Cited by 2 publications

(3 citation statements)

References 20 publications

(30 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…(B) As shown in Shen et al (2012), the truncated

ℓ_{1}

‐function

\min false(\frac{false| β_{l} false|}{τ}, 1 false)

can be regarded as a non‐convex and non‐smooth surrogate of

ℓ_{0}

‐function

I false(β_{l} \neq 0 false)

when

τ \to 0

. Besides, the selection consistency can be achieved by the

ℓ_{0}

‐penalty and its surrogate—the truncated

ℓ_{1}

‐penalty (Dai et al, 2021; Shen et al, 2013). Therefore, the sparse PCA with the

ℓ_{1}

penalty cannot achieve selection consistency.…”

Section: The Methodsmentioning

confidence: 99%

See 1 more Smart Citation

Feature grouping and sparse principal component analysis with truncated regularization

Jiang

Qin

Padilla

2023

Stat

View full text Add to dashboard Cite

We propose a new method for principal component analysis (PCA) called feature grouping and sparse principal component analysis (FGSPCA). This method is designed to capture both grouping and sparsity structures in factor loadings simultaneously. To achieve this, we use a non‐convex truncated regularization that can adjust for sparsity and grouping effects automatically. This regularization encourages factor loadings with similar values to be either grouped together for feature grouping or be zero for feature selection, helping to reduce model complexity and improve interpretation. While other structured PCA methods often require prior knowledge to construct the regularization term, FGSPCA can capture grouping and sparsity structures without any prior information. We solve the resulting non‐convex optimization problem using an alternating algorithm that combines difference‐of‐convex programming, the augmented Lagrange method, and coordinate descent method. Our experiments show that FGSPCA performs well and efficiently on both synthetic and real‐world datasets. An implementation of FGSPCA is available on GitHub https://github.com/HaiyanJiang/FGSPCA.

show abstract

“…(B) As shown in Shen et al (2012), the truncated

ℓ_{1}

‐function

\min false(\frac{false| β_{l} false|}{τ}, 1 false)

can be regarded as a non‐convex and non‐smooth surrogate of

ℓ_{0}

‐function

I false(β_{l} \neq 0 false)

when

τ \to 0

. Besides, the selection consistency can be achieved by the

ℓ_{0}

‐penalty and its surrogate—the truncated

ℓ_{1}

‐penalty (Dai et al, 2021; Shen et al, 2013). Therefore, the sparse PCA with the

ℓ_{1}

penalty cannot achieve selection consistency.…”

Section: The Methodsmentioning

confidence: 99%

“…0. Besides, the selection consistency can be achieved by the ℓ 0 -penalty and its surrogate-the truncated ℓ 1 -penalty (Dai et al, 2021;Shen et al, 2013). Therefore, the sparse PCA with the ℓ 1 penalty cannot achieve selection consistency.…”

Section: Feature Grouping and Sparse Loadingsmentioning

confidence: 99%

Feature grouping and sparse principal component analysis with truncated regularization

Jiang

Qin

Padilla

2023

Stat

View full text Add to dashboard Cite

show abstract

“…As shown in , the truncated L 1 -function min( |β l | τ , 1) can be regarded as a non-convex and non-smooth surrogate of L 0 -function I(β l = 0) when τ → 0. Besides, the selection consistency can be achieved by the L 0 -penalty and its surrogate-the truncated L 1penalty (Dai et al, 2021, Shen et al, 2013. B).…”

Section: The Proposed Fgspca Methodsmentioning

confidence: 99%

Feature Grouping and Sparse Principal Component Analysis

Jiang¹,

Qin²,

Padilla³

2021

Preprint

View full text Add to dashboard Cite

Sparse Principal Component Analysis (SPCA) is widely used in data processing and dimension reduction; it uses the lasso to produce modified principal components with sparse loadings for better interpretability. However, sparse PCA never considers an additional grouping structure where the loadings share similar coefficients (i.e., feature grouping), besides a special group with all coefficients being zero (i.e., feature selection). In this paper, we propose a novel method called Feature Grouping and Sparse Principal Component Analysis (FGSPCA) which allows the loadings to belong to disjoint homogeneous groups, with sparsity as a special case. The proposed FGSPCA is a subspace learning method designed to simultaneously perform grouping pursuit and feature selection, by imposing a non-convex regularization with naturally adjustable sparsity and grouping effect. To solve the resulting non-convex optimization problem, we propose an alternating algorithm that incorporates the difference-of-convex programming, augmented Lagrange and coordinate descent methods. Additionally, the experimental results on real data sets show that the proposed FGSPCA benefits from the grouping effect compared with methods without grouping effect.

show abstract

Truncated $L_1$ Regularized Linear Regression: Theory and Algorithm

Cited by 2 publications

References 20 publications

Feature grouping and sparse principal component analysis with truncated regularization

Feature grouping and sparse principal component analysis with truncated regularization

Feature Grouping and Sparse Principal Component Analysis

Contact Info

Product

Resources

About