Segmenting Purple Rapeseed Leaves in the Field from UAV RGB Imagery Using Deep Learning as an Auxiliary Means for Nitrogen Stress Detection

Abstract

Crop leaf purpling is a common phenotypic change when plants are subject to some biotic and abiotic stresses during their growth. The extraction of purple leaves can monitor crop stresses as an apparent trait and meanwhile contributes to crop phenotype analysis, monitoring, and yield estimation. Due to the complexity of the field environment as well as differences in size, shape, texture, and color gradation among the leaves, purple leaf segmentation is difficult. In this study, we used a U-Net model for segmenting purple rapeseed leaves during the seedling stage based on unmanned aerial vehicle (UAV) RGB imagery at the pixel level. With the limited spatial resolution of rapeseed images acquired by UAV and small object size, the input patch size was carefully selected. Experiments showed that the U-Net model with the patch size of 256 × 256 pixels obtained better and more stable results with a F-measure of 90.29% and an Intersection of Union (IoU) of 82.41%. To further explore the influence of image spatial resolution, we evaluated the performance of the U-Net model with different image resolutions and patch sizes. The U-Net model performed better compared with four other commonly used image segmentation approaches comprising support vector machine, random forest, HSeg, and SegNet. Moreover, regression analysis was performed between the purple rapeseed leaf ratios and the measured N content. The negative exponential model had a coefficient of determination (R²) of 0.858, thereby explaining much of the rapeseed leaf purpling in this study. This purple leaf phenotype could be an auxiliary means for monitoring crop growth status so that crops could be managed in a timely and effective manner when nitrogen stress occurs. Results demonstrate that the U-Net model is a robust method for purple rapeseed leaf segmentation and that the accurate segmentation of purple leaves provides a new method for crop nitrogen stress monitoring.

Ramie Yield Estimation Based on UAV RGB Images

“…At present, the method based on target detection is mainly used for crop plant count [ 24 , 25 , 26 ]. However, plant counting using target detection technology based on UAV images poses a great challenge due to limited spatial resolution, small object size, and complex background features [ 27 ]. Previous studies have confirmed that combining vegetation indices, plant height, canopy cover, plant density, and other multiple features can improve the yield estimation.…”

UAVs for Vegetation Monitoring: Overview and Recent Scientific Contributions

“…Lower altitudes result in images with finer spatial resolution, but require longer flights and demand higher computational times for image processing [35]. Therefore, evaluation of flight altitude and spatial resolution was a central point in some investigations of this SI [16,24,27,29,36], being a usual objective to resample UAV images to spatial resolutions similar to those of common satellites, e.g., 5 m of Rapideye, 10 m of Sentinel-2, and 30 m of LANDSAT, in order to study the potential of using such satellite platforms instead of UAV for specific agro-forestry goals. Thus, Iizuka et al (2019) [16] discovered that CC estimation was affected by spatial resolution and coarser resolutions showed stronger correlation with manually delineated data in an RGB orthomosaic, concluding that the satellite images could be feasible for this purpose.…”

“…Thus, Guo et al (2021) [29] and Ye et al (2020) [27] evaluated several pixel sizes to select the optimal resolution for mapping disease areas, yielding 0.01 m for yellow rust in wheat and a minimum of 0.5 m in the case of Fusarium wilt in banana, respectively. In addition, Zhang et al (2020) [36] explored the influence of image spatial resolution on the segmentation process prior to identify N deficiency in rapeseed fields, concluding that the optimal value by using the U-Net convolutional neural network architecture will depend on the target size and stating an appropriate patch size of 256x256 pixels in the purple rapeseed leaves.…”

“…Leaf N is directly related to the chlorophyll content, which is a key indicator of photosynthetic capacity and nutrient status. Zhang et al (2020) [36] developed a method to monitor crop N stress in winter oilseed rape using UAV RGB imagery and ML-based segmentation. Their method consisted in discriminating the rapeseed leaves with purple color, a symptom of N deficiency, using a binary semantic segmentation process and, then, relating the purple leaf area ratio to the N content of the plants.…”

DBA_SSD ： A Novel End-to-End Object Detection Using Deep Attention Module for Helping Smart Device with Vegetable and Fruit Leaf Plant Disease Detection

“…Mohamed Kerkech et al [37] proposed a new method of grape disease detection based on the SegNet [38] architecture for visible light and infrared image segmentation to identify shadows, ground, healthy and symptomatic vines, and finally merge the segmentation obtained from visible light and infrared images to generate the whole disease map of grapes. Literature [39] , a U-Net method for pixel-level purple rapeseed segmentation was proposed to calculate the model parameters by adjusting the sample size. In the literature [40] , a new thermal imaging method was proposed to calculate the color similarity problem between unripe citrus fruits and leaves, which were prone to temperature differences between fruit and leaf surfaces because of the varying rates of temperature change between the fruit and leaf surfaces caused by water mist and to build a deep learning model based on the thermal imaging system.…”