To address the problems of defective pixels and strips in hyperspectral images affecting subsequent processing and applications, we modeled the hyperspectral image (HSI) inpainting problem as a sparse signal reconstruction problem with incomplete observations using the theory of sparse representation, and proposed an HSI inpainting algorithm based on spectral dictionary learning. First, we studied the HSI observation model under the assumption of additive noise. We subsequently proposed a new algorithm for constructing a spectral dictionary directly from hyperspectral data by introducing an online learning optimization method and performing dictionary learning using a robust function. Afterwards, the image was sparsely encoded by applying the variable decomposition and augmented Lagrangian sparse regression method. Finally, the inpainted HSI was obtained by sparse reconstruction. The experimental results showed that compared with the existing algorithms, the algorithm proposed herein could effectively inpaint the defective HSI under different noise conditions with a shorter calculation time than those of existing methods and other dictionary learning inpainting algorithms.