Sparse Recovery, Classification, and Data Compression via Improved Maximum Feasible Subsystem Algorithms

Abstract

Two new strategies are developed in this thesis to increase the speed of the state-ofthe-art Maximum Feasible Subsystem (MAX FS) algorithms. The newly developed strategies can be combined with any MAX FS algorithm to increase its speed while preserving or improving solution quality. The improved algorithms apply in the case of dense constraint matrices such as those found when various data compression/dimensionality reduction, classification, and sparse recovery problems are converted to MAX FS problems. This approach is used for sparse recovery in Compressive Sensing (CS) and data compression via Nonnegative Matrix Factorization (NNMF) for the first time in this thesis. In CS, the new algorithm can successfully recover real-world signals such as speech and Electrocardiogram (ECG) signals that have been more greatly compressed than existing methods can handle, and with greater recovered signal quality. They also deliver sparser solutions when compared with those obtained using traditional sparse recovery algorithms. The new algorithm significantly reduces the solution time of the existing MAX FS methods. It reduces the solution time of Method B by 80.64% and 77.82% for recovery of compressively sensed ECG and speech signals for instance, respectively. Three novel MAX FS-based methods for NNMF are developed and investigated using synthetic data and real-world image datasets. The new NNMF algorithms, unlike the majority of existing NNMF methods, do not require an initialization method or prior knowledge of the matrix rank. iii These new NNMF solutions methods are also more robust to outliers and provide higher quality solution compared to the state-of-the-art NNMF algorithms. To illustrate, the new NNMF algorithm improves the relative approximation quality of the existing column subset selection NNMF algorithm by 5.2% for the ORL facial image dataset. A faster MAX FS-based algorithm for binary classification is also proposed, which improves the processing time of the existing MAX FS algorithm (Algorithm 2) by 94.77% on average. In comparison, the developed algorithm outperforms widely used binary classification algorithms by providing better results for recall-oriented machine learning tasks such as disease diagnosis. For instance, the new algorithm improves the average recall score of Support Vector Machine (SVM) by 18.16% for binary classification task.