Classification and Feature Selection in Sparse High-Dimensional Models

Abstract

This work deals with binary classification in a sparse high-dimensional model. Essentially, we have two predetermined subclasses of the normally distributed population and we want to assign a new observation to one of the two subclasses. In this thesis, we look at a classification problem in which the sample size is less than the dimension of the data. The goal is to find good methods for classification that can minimize our chances of misclassification under these circumstances. A typical high-dimensional problem, like the one studied in this thesis, has a large number of unknown parameters and not enough data to make reliable inferences about these parameters. The classical statistical methods were not designed to cope with high-dimensional problems. Therefore, when studying the classification problem of our interest, we have proposed original statistical ideas and tools based on the notion of 'sparsity'.Much of the research done on classification in high-dimensional settings has been under the assumption that the covariance matrix of the observed vectors has a diagonal structure. In this thesis, we have a more general condition for this covariance matrix.We only require that the matrix has block-diagonal structure which makes this study more relevant to practical situations.In this thesis, we propose new classification procedures and investigate one existing classification method from the literature in terms of classification accuracy. Even in a simpler setup of a diagonal covariance matrix, there has been no rigorous justification for the existing method under consideration which makes this research crucial for furthering our understanding of how to minimize the chances of misclassification under the conditions mentioned above.The motivation behind much of the research done in the realm of high-dimensional statistics is in the pursuit of knowledge. While this is the case here as well, it may also have relevance to the biomedical field as the concept of classification (especially, binary classification) is indistinguishable from the notion of diagnostics in the medical field. i Aknowledgements I would first like to thank my supervisor, Dr. Natalia Stepanova, for all her help over the past two years. Her guidance, knowledge and patience has been invaluable over that time, but it is her devotion to her students that I have appreciated the most.