Hyperspectral image (HSI) clustering is generally a challenging task because of the complex spectral-spatial structure. Based on the assumption that all the pixels are sampled from the union of subspaces, recent works have introduced a robust technique-the sparse subspace clustering (SSC) algorithm and its enhanced versions (SSC models incorporating spatial information)-to cluster HSIs, achieving excellent performances. However, these methods are all based on the linear representation model, which conflicts with the well-known nonlinear structure of HSIs and limits their performance to a large degree. In this paper, to overcome these obstacles, we present a new kernel sparse subspace clustering algorithm with a spatial max pooling operation (KSSC-SMP) for hyperspectral remote sensing data interpretation. The proposed approach maps the feature points into a much higher dimensional kernel space to extend the linear sparse subspace clustering model to nonlinear manifolds, which can better fit the complex nonlinear structure of HSIs. With the help of the kernel sparse representation, a more accurate representation coefficient matrix can be obtained. A spatial max pooling operation is utilized for the representation coefficients to generate more discriminant features by integrating the spatial-contextual information, which is essential for the accurate modeling of HSIs. This paper is an extension of our previous conference paper, and a number of enhancements are put forward. The proposed algorithm was evaluated on two well-known hyperspectral data sets-the Salinas image and the University of Pavia image-and the experimental results clearly indicate that the newly developed KSSC-SMP algorithm can obtain very competitive clustering results for HSIs, outperforming the current state-of-the-art clustering methods.