The classification of aerial and satellite remote sensing data has become a challenging problem due to the recent advances in remote sensor technology that led to higher spatial and spectral resolutions. This research paper presents novel sensor independent algorithms and techniques for dealing with the challenges of classification of high volume remote sensor data. A fast unsupervised band reduction method is proposed to lower the dimensionality of the input image. The band reduced image is then split into two mutually disjoint pure and mixed pixel subsets by a pixel segregator built using extended mathematical morphology techniques. A novel hierarchical spectral-spatial support vector machine based classifier that adaptively includes the usage of expensive spatial information based on the pixel categorization is proposed. The final thematic map is obtained after merging the classification results of the two subsets and fixed spatial neighborhood homogenization. The accuracy, efficiency and flexibility of the developed system are demonstrated by evaluating the classification results using several hyperspectral and multispectral data sets. The obtained results demonstrate that the proposed method performs significantly better than conventional classifiers while alleviating the computational complexity involved in generating spatial information.