Aggregate grading is an important link in aggregate production. Aiming at solving the problems of low efficiency of traditional artificial quality detection method and the detection accuracy affected by the subjective consciousness of the tester, combined with the method of deep learning, the intelligent grading of aggregates is realized in this paper. According to the aggregate provided by the local gravel company, 200 digital images of aggregate were taken and marked. Using FCN, DeepLabv3+, Attention-UNet, DANet, PSPNet to achieve the segmentation of aggregates. The particle size of the aggregate is calculated by drawing the minimum circumscribed circle of the aggregate, and then the aggregate is classified. The experimental results show that the method combined with deep learning has an accuracy rate of more than 95% in aggregate detection, and can accurately measure the particle size of aggregates, which is of great significance to the intelligent development of the industry.