Logistics box recognition in robotic industrial de-palletising procedure with systematic RGB-D image processing supported by multiple deep learning methods

Yoon, Jonghun; Han, Jooyeop; Nguyen, Thong

doi:10.1016/j.engappai.2023.106311

Cited by 10 publications

(2 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The main object detection algorithms used are YOLO (Redmon et al [ 17 ]) and R-CNN (He et al [ 18 ]). YOLO is a grid-based approach that predicts boundaries and selects the highest-class probability after an object passes through a CNN, and R-CNN classifies the pixels that constitute the object in the identified boundaries, proposes the potential position of the object using a neural network, and classifies and detects the object based on the proposed area [ 19 , 20 , 21 , 22 , 23 , 24 , 25 , 26 , 27 , 28 , 29 , 30 ]. In the case of Zhao et al [ 19 ], they propose a planar parcel detection method using R-CNN reinforcement.…”

Section: Introductionmentioning

confidence: 99%

“…Finally, we propose a parallel algorithm of image processing and deep learning to improve the detection speed of the courier of a 5-ton truck. In a previous study by Yoon et al [ 28 ], the depth map included segment boundaries and utilized CycleGAN to standardize the appearance of boxes, enabling box recognition through Mask-RCNN. In contrast, our proposed method focuses on box recognition through boundary segmentation rather than direct box detection using YOLACT.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Revolutionizing Robotic Depalletizing: AI-Enhanced Parcel Detecting with Adaptive 3D Machine Vision and RGB-D Imaging for Automated Unloading

Kim,

Truong,

Lee

et al. 2024

Sensors

Self Cite

View full text Add to dashboard Cite

Detecting parcels accurately and efficiently has always been a challenging task when unloading from trucks onto conveyor belts because of the diverse and complex ways in which parcels are stacked. Conventional methods struggle to quickly and accurately classify the various shapes and surface patterns of unordered parcels. In this paper, we propose a parcel-picking surface detection method based on deep learning and image processing for the efficient unloading of diverse and unordered parcels. Our goal is to develop a systematic image processing algorithm that emphasises the boundaries of parcels regardless of their shape, pattern, or layout. The core of the algorithm is the utilisation of RGB-D technology for detecting the primary boundary lines regardless of obstacles such as adhesive labels, tapes, or parcel surface patterns. For cases where detecting the boundary lines is difficult owing to narrow gaps between parcels, we propose using deep learning-based boundary line detection through the You Only Look at Coefficients (YOLACT) model. Using image segmentation techniques, the algorithm efficiently predicts boundary lines, enabling the accurate detection of irregularly sized parcels with complex surface patterns. Furthermore, even for rotated parcels, we can extract their edges through complex mathematical operations using the depth values of the specified position, enabling the detection of the wider surfaces of the rotated parcels. Finally, we validate the accuracy and real-time performance of our proposed method through various case studies, achieving mAP (50) values of 93.8% and 90.8% for randomly sized and rotationally covered boxes with diverse colours and patterns, respectively.

show abstract