Since the mature green tomatoes have color similar to branches and leaves, some are shaded by branches and leaves, and overlapped by other tomatoes, the accurate detection and location of these tomatoes is rather difficult. This paper proposes to use the Mask R-CNN algorithm for the detection and segmentation of mature green tomatoes. A mobile robot is designed to collect images round-the-clock and with different conditions in the whole greenhouse, thus, to make sure the captured dataset are not only objects with the interest of users. After the training process, RestNet50-FPN is selected as the backbone network. Then, the feature map is trained through the region proposal network to generate the region of interest (ROI), and the ROIAlign bilinear interpolation is used to calculate the target region, such that the corresponding region in the feature map is pooled to a fixed size based on the position coordinates of the preselection box. Finally, the detection and segmentation of mature green tomatoes is realized by the parallel actions of ROI target categories, bounding box regression and mask. When the Intersection over Union is equal to 0.5, the performance of the trained model is the best. The experimental results show that the F1-Score of bounding box and mask region all achieve 92.0%. The image acquisition processes are fully unobservable, without any user preselection, which are a highly heterogenic mix, the selected Mask R-CNN algorithm could also accurately detect mature green tomatoes. The performance of this proposed model in a real greenhouse harvesting environment is also evaluated, thus facilitating the direct application in a tomato harvesting robot.

The maturity level of tomato is a key factor of tomato picking, which directly determines the transportation distance, storage time, and market freshness of postharvest tomato. In view of the lack of studies on tomato maturity classification under nature greenhouse environment, this paper proposes a SE-YOLOv3-MobileNetV1 network to classify four kinds of tomato maturity. The proposed maturity classification model is improved in terms of speed and accuracy: (1) Speed: Depthwise separable convolution is used. (2) Accuracy: Mosaic data augmentation, K-means clustering algorithm, and the Squeeze-and-Excitation attention mechanism module are used. To verify the detection performance, the proposed model is compared with the current mainstream models, such as YOLOv3, YOLOv3-MobileNetV1, and YOLOv5 in terms of accuracy and speed. The SE-YOLOv3-MobileNetV1 model is able to distinguish tomatoes in four kinds of maturity, the mean average precision value of tomato reaches 97.5%. The detection speed of the proposed model is 278.6 and 236.8 ms faster than the YOLOv3 and YOLOv5 model. In addition, the proposed model is considerably lighter than YOLOv3 and YOLOv5, which meets the need of embedded development, and provides a reference for tomato maturity classification of tomato harvesting robot.

In this paper, we propose a tree trunk and obstacle detection method in a semistructured apple orchard environment based on improved YOLOv5s, with an aim to improve the real-time detection performance. The improvement includes using the K-means clustering algorithm to calculate anchor frame and adding the Squeeze-and-Excitation module and 10% pruning operation to ensure both detection accuracy and speed. Images of apple orchards in different seasons and under different light conditions are collected to better simulate the actual operating environment. The Gradient-weighted Class Activation Map technology is used to visualize the performance of YOLOv5s network with and without improvement to increase interpretability of improved network on detection accuracy. The detected tree trunk can then be used to calculate the traveling route of an orchard carrier platform, where the centroid coordinates of the identified trunk anchor are fitted by the least square method to obtain the endpoint of the next time traveling rout. The mean average precision values of the proposed model in spring, summer, autumn, and winter were 95.61%, 98.37%, 96.53%, and 89.61%, respectively. The model size of the improved model is reduced by 13.6 MB, and the accuracy and average accuracy on the test set are increased by 5.60% and 1.30%, respectively. The average detection time is 33 ms, which meets the requirements of real-time detection of an orchard carrier platform.