The separation of walnut kernels and shells has long been regarded as a bottleneck, limiting processing efficiency, product quality, and industry advancement. In response to the challenges of improving separation accuracy and the inadequacy of existing equipment for meeting industry demands, this paper proposes an innovative walnut shell–kernel separation device based on machine vision technology. An experimental system was constructed, and key parameters were optimized to enhance its performance. The device comprises five main modules: material conveyance, image acquisition, control module, sorting module, and frame. Differential separation technology is used to convert the walnut material group into a stable particle flow, addressing the issue of missed selections due to material blockages. An enhanced YOLOv8n algorithm improves small object detection and interference resistance, enabling accurate identification of walnut kernels. The Box–Behnken Design and Artificial Neural Network prediction model was used to determine the optimal operating parameters for the device. Experimental results showed that effective differential separation was achieved when the dual-stage conveyor system operated at speeds of 0.2 m/s and 1 m/s. The improved I-YOLOv8n algorithm reached an accuracy of 98.8%. Using the neural network model, the optimal operational parameters were determined: an air pressure of 0.72 MPa, a jetting component angle of 10.16°, and a sorting height of 105.12 cm. Under these conditions, the device achieved an actual cleaning rate of 93.56%, demonstrating outstanding separation performance. Compared to traditional separation methods, this device offers significant advantages in terms of efficiency, quality, and ease of operation, providing new technological pathways and support for the automation and intelligent transformation of the walnut processing industry. In the future, the device is expected to undergo further improvements to meet broader market demand and serve as a reference for the separation of other agricultural products.