In order to solve the problems of long-term image acquisition time and massive data processing in a terahertz time domain spectroscopy imaging system, a novel fast terahertz imaging model, combined with group sparsity and nonlocal self-similarity (GSNS), is proposed in this paper. In GSNS, the structure similarity and sparsity of image patches in both two-dimensional and three-dimensional space are utilized to obtain high-quality terahertz images. It has the advantages of detail clarity and edge preservation. Furthermore, to overcome the high computational costs of matrix inversion in traditional split Bregman iteration, an acceleration scheme based on conjugate gradient method is proposed to solve the terahertz imaging model more efficiently. Experiments results demonstrate that the proposed approach can lead to better terahertz image reconstruction performance at low sampling rates.