In this study, a deep learning-based system for apple defect detection and quality grading was developed, integrating various advanced image-processing technologies and machine learning algorithms to enhance the automation and accuracy of apple quality monitoring. Experimental validation demonstrated the superior performance of the proposed model in handling complex image tasks. In the defect-segmentation experiments, the method achieved a precision of 93%, a recall of 90%, an accuracy of 91% and a mean Intersection over Union (mIoU) of 92%, significantly surpassing traditional deep learning models such as U-Net, SegNet, PSPNet, UNet++, DeepLabv3+ and HRNet. Similarly, in the quality-grading experiments, the method exhibited high efficiency with a precision of 91%, and both recall and accuracy reaching 90%. Additionally, ablation experiments with different loss functions confirmed the significant advantages of the Jump Loss in enhancing model performance, particularly in addressing class imbalance and improving feature learning. These results not only validate the effectiveness and reliability of the system in practical applications but also highlight its potential in automating the detection and grading processes in the apple industry. This integration of advanced technologies provides a new automated solution for quality control of agricultural products like apples, facilitating the modernization of agricultural production.