Seed sorting based on deep neural networks is one of the important applications of seed variety identification and quality purification. However, DNNs is difficult to deploy on embedded devices since the consumption of computational and storage resource. To address these problems, this paper proposes a pipeline‐style neural network framework for real‐time seed sorting. First, we propose a novel algorithm, 2D information entropy, pruning redundant filters to realize structured pruning. Then, the pruning rate of each convolution layer is determined by visualizing the results of 2D entropy. Meanwhile, the pruned network is fine‐tuned to recover the performance. Finally, TensorRT is utilized to optimize and accelerate the pruned model for deployment in Jeston Nano. Experiments on two large‐scale seed‐sorting datasets demonstrate the significant improvement of the proposed method over existing model compression methods. Experimental results on Jeston Nano show that the pruned model 2EFP‐E achieves a single image inference speed of 107 FPS, with the best accuracy of 95.94% on the red kidney bean dataset.