An image restoration and detection method for picking robot based on convolutional auto-encoder

Chen, Jiqing; Zhang, Hongdu; Wang, Zhikui; Wu, Jiahua; Luo, Tian; Wang, Huabin; Li, Teng

doi:10.1016/j.compag.2022.106896

Cited by 11 publications

(4 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Firstly, directly detecting images of occluded and overlapping fruits. Secondly, performing classification recognition of obstacles and unoccluded fruits [12]. Thirdly, conducting image restoration by repairing occluded fruits before recognition.…”

Section: The Dependence Of Recognition and Localization Functions On ...mentioning

confidence: 99%

“…In practical harvesting operations, the precision of target recognition and localization is influenced by various factors, and the resolution of target recognition and localization problems needs to be considered from multiple perspectives. (1) Interference from complex work environments, including interference from complex backgrounds [10]; variations in natural lighting [11]; fruit overlapping and obstruction from branches and leaves [12].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

An Overview of the Application of Machine Vision in Recognition and Localization of Fruit and Vegetable Harvesting Robots

Hou,

Chen,

Jiang

et al. 2023

Agriculture

View full text Add to dashboard Cite

Intelligent agriculture imposes higher requirements on the recognition and localization of fruit and vegetable picking robots. Due to its unique visual information and relatively low hardware cost, machine vision is widely applied in the recognition and localization of fruit and vegetable picking robots. This article provides an overview of the application of machine vision in the recognition and localization of fruit and vegetable picking robots. Firstly, the advantages, disadvantages, and the roles of different visual sensors and machine vision algorithms in the recognition and localization of fruit and vegetable picking robots are introduced, including monocular cameras, stereo cameras, structured light cameras, multispectral cameras, image segmentation algorithms, object detection algorithms, and 3D reconstruction algorithms. Then, the current status and challenges faced by machine vision in the recognition and localization of fruit and vegetable picking robots are summarized. These challenges include the stability of fast recognition under complex background interference, stability of recognition under different lighting environments for the same crop, the reliance of recognition and localization on prior information in the presence of fruit overlap and occlusions caused by leaves and branches, and the uncertainty of picking caused by complex working environments. In current research on algorithms dealing with complex background interference and various occlusion disturbances, good results have been achieved. Different lighting environments have a significant impact on the recognition and positioning of fruits and vegetables, with a minimum accuracy of 59.2%. Finally, this article outlines future research directions to address these challenges.

show abstract

Section: The Dependence Of Recognition and Localization Functions On ...mentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

An Overview of the Application of Machine Vision in Recognition and Localization of Fruit and Vegetable Harvesting Robots

Hou,

Chen,

Jiang

et al. 2023

Agriculture

View full text Add to dashboard Cite

show abstract

“…Automated agricultural systems require counting fruits and leaves for yield estimation or disease monitoring, and amodal segmentation technology provides an effective solution for this. In the agricultural realm, Chen et al (2022) leveraged the robust feature extraction and reconstruction capabilities of convolutional autoencoders to recover pixels in obscured regions. To address the challenge of reduced accuracy in detecting obscured citrus fruits, the convolutional autoencoder skillfully extracted meaningful features from the surrounding background information, restoring the integrity of the original image.…”

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Application of amodal segmentation for shape reconstruction and occlusion recovery in occluded tomatoes

Yang,

Deng,

Zhang

et al. 2024

Front. Plant Sci.

View full text Add to dashboard Cite

Common object detection and image segmentation methods are unable to accurately estimate the shape of the occluded fruit. Monitoring the growth status of shaded crops in a specific environment is challenging, and certain studies related to crop harvesting and pest detection are constrained by the natural shadow conditions. Amodal segmentation can focus on the occluded part of the fruit and complete the overall shape of the fruit. We proposed a Transformer-based amodal segmentation algorithm to infer the amodal shape of occluded tomatoes. Considering the high cost of amodal annotation, we only needed modal dataset to train the model. The dataset was taken from two greenhouses on the farm and contains rich occlusion information. We introduced boundary estimation in the hourglass structured network to provide a priori information about the completion of the amodal shapes, and reconstructed the occluded objects using a GAN network (with discriminator) and GAN loss. The model in this study showed accuracy, with average pairwise accuracy of 96.07%, mean intersection-over-union (mIoU) of 94.13% and invisible mIoU of 57.79%. We also examined the quality of pseudo-amodal annotations generated by our proposed model using Mask R-CNN. Its average precision (AP) and average precision with intersection over union (IoU) 0.5 (AP50) reached 63.91%,86.91% respectively. This method accurately and rationally achieves the shape of occluded tomatoes, saving the cost of manual annotation, and is able to deal with the boundary information of occlusion while decoupling the relationship of occluded objects from each other. Future work considers how to complete the amodal segmentation task without overly relying on the occlusion order and the quality of the modal mask, thus promising applications to provide technical support for the advancement of ecological monitoring techniques and ecological cultivation.

show abstract

LmGa: Combining label mapping method with graph attention network for agricultural recognition

Tran-Anh,

Vu,

Bui-Quoc

et al. 2024

Knowl Inf Syst

View full text Add to dashboard Cite

An image restoration and detection method for picking robot based on convolutional auto-encoder

Cited by 11 publications

References 20 publications

An Overview of the Application of Machine Vision in Recognition and Localization of Fruit and Vegetable Harvesting Robots

An Overview of the Application of Machine Vision in Recognition and Localization of Fruit and Vegetable Harvesting Robots

Application of amodal segmentation for shape reconstruction and occlusion recovery in occluded tomatoes

LmGa: Combining label mapping method with graph attention network for agricultural recognition

Contact Info

Product

Resources

About