Lychee is an economically important crop with widespread popularity. However, lychee diseases significantly impact both the yield and fruit quality of lychee. Existing lychee disease detection models face challenges such as large parameter sizes, slow processing speeds, and deployment complexities. To address these challenges, this paper proposes an improved lightweight network, named YOLOv7-MGPC (YOLOv7-Mosaic-GhostNet-Pruning-CBAM), that enables real-time lychee disease detection. In this study, we collected datasets of lychee diseases, covering four types of leaf diseases, and employed Mosaic data augmentation for data preprocessing. Building upon the YOLOv7 framework, we replaced the original backbone network with the lightweight GhostNetV1 and applied channel pruning to effectively reduce the parameter overhead. Subsequently, an attention mechanism called CBAM was incorporated to enhance the detection accuracy. The resultant model was then deployed to edge devices (Nvidia Jetson Nano) for real-world applications. Our experiments showed that our enhanced YOLOv7 variant outperforms the original model by a large margin, achieving a speed increase from 120 frames/s to 217 frames/s while maintaining an accuracy of 88.6%. Furthermore, the parameter size was substantially reduced from 36.5 M to 7.8 M, which firmly demonstrates the effectiveness of our methods in enabling model deployment on edge devices for lychee disease detection.