At present, the classification of the hyperspectral image (HSI) based on the deep convolutional network has made great progress. Due to the high dimensionality of spectral features, limited samples of ground truth, and high nonlinearity of hyperspectral data, effective classification of HSI based on deep convolutional neural networks is still difficult. This paper proposes a novel deep convolutional network structure, namely, a hybrid depth-separable residual network, for HSI classification, called HDSRN. The HDSRN model organically combines 3D CNN, 2D CNN, multiresidual network ROR, and depth-separable convolutions to extract deeper abstract features. On the one hand, due to the addition of multiresidual structures and skip connections, this model can alleviate the problem of over fitting, help the backpropagation of gradients, and extract features more fully. On the other hand, the depth-separable convolutions are used to learn the spatial feature, which reduces the computational cost and alleviates the decline in accuracy. Extensive experiments on the popular HSI benchmark datasets show that the performance of the proposed network is better than that of the existing prevalent methods.