The deep-learning steganography of current hotspots can conceal an image secret message in a cover image of the same size. While the steganography secret message is primarily removed via active steganalysis. The document image as the secret message in deep-learning steganography can deliver a considerable amount of effective information in a secret communication process. This study builds and implements deep-learning steganography removal models of document image secret messages based on the idea of adversarial perturbation removal: feed-forward denoising convolutional neural networks (DnCNN) and high-level representation guided denoiser (HGD). Further—considering the large computation cost and storage overheads of the above model—we use the document image-quality assessment (DIQA) as threshold, calculate the importance of filters using geometric median and prune redundant filters as extensively as possible through the overall iterative pruning and artificial bee colony (ABC) automatic pruning algorithms to reduce the size of the network structure of the existing vast and over-parameterized deep-learning steganography removal model, while maintaining the good removal effects of the model in the pruning process. Experiment results showed that the model generated by this method has better adaptability and scalability. Compared with the original deep-learning steganography removal model without pruning in this paper, the classic indicators params and flops are reduced by more than 75%.