Minimax optimal high‐dimensional classification using deep neural networks

Wang, Shuoyang; Shang, Zuofeng

doi:10.1002/sta4.482

Cited by 1 publication

(1 citation statement)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It is interesting to explore whether minimax optimal DNN approaches with dimension-free properties can be established in the classification setting. Existing literature on this front is limited, and most works either treat classification as regression by estimating the conditional class probability instead of the decision boundary Kohler and Langer, 2020;Bos and Schmidt-Hieber, 2021;Hu et al, 2021;Wang et al, 2022b,a;Wang and Shang, 2022) or settle for an upper bound on the misclassification risk (Kim et al, 2021;Steinwart et al, 2007;Hamm and Steinwart, 2020). Unlike regression problems where one intends to estimate the unknown regression functions, the goal of classification is to recover the unknown decision boundary separating different classes.…”

Section: Introductionmentioning

confidence: 99%

Minimax Optimal Deep Neural Network Classifiers Under Smooth Decision Boundary

Hu¹,

Liu²,

Shang³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Deep learning has gained huge empirical successes in large-scale classification problems. In contrast, there is a lack of statistical understanding about deep learning methods, particularly in the minimax optimality perspective. For instance, in the classical smooth decision boundary setting, existing deep neural network (DNN) approaches are rate-suboptimal, and it remains elusive how to construct minimax optimal DNN classifiers. Moreover, it is interesting to explore whether DNN classifiers can circumvent the "curse of dimensionality" in handling highdimensional data. The contributions of this paper are two-fold. First, based on a localized margin framework, we discover the source of suboptimality of existing DNN approaches. Motivated by this, we propose a new deep learning classifier using a divide-and-conquer technique: DNN classifiers are constructed on each local region and then aggregated to a global one. We further propose a localized version of the classical Tsybakov's noise condition, under which statistical optimality of our new classifier is established. Second, we show that DNN classifiers can adapt to low-dimensional data structures and circumvent the "curse of dimensionality" in the sense that the minimax rate only depends on the effective dimension, potentially much smaller than the actual data dimension. Numerical experiments are conducted on simulated data to corroborate our theoretical results.

show abstract