Hard Thresholding Regularised Logistic Regression: Theory and Algorithms

Abstract

The hard thresholding regularised logistic regression in high dimensions with larger number of features than samples is considered. The sharp oracle inequality for the global solution is established. If the target signal is detectable, it is proven that with a high probability the estimated and true supports coincide. Starting with the KKT condition, we introduce the primal and dual active sets algorithm for fitting and also consider a sequential version of this algorithm with a warm-start strategy. Simulations and a real data analysis show that SPDAS outperforms LASSO, MCP and SCAD methods in terms of computational efficiency, estimation accuracy, support recovery and classification.