Design and Acceleration of Field Programmable Gate Array-Based Deep Learning for Empty-Dish Recycling Robots

Abstract

As the proportion of the working population decreases worldwide, robots with artificial intelligence have been a good choice to help humans. At the same time, field programmable gate array (FPGA) is generally used on edge devices including robots, and it greatly accelerates the inference process of deep learning tasks, including object detection tasks. In this paper, we build a unique object detection dataset of 16 common kinds of dishes and use this dataset for training a YOLOv3 object detection model. Then, we propose a formalized process of deploying a YOLOv3 model on the FPGA platform, which consists of training and pruning the model on a software platform, and deploying the pruned model on a hardware platform (such as FPGA) through Vitis AI. According to the experimental results, we successfully realize acceleration of the dish detection using a YOLOv3 model based on FPGA. By applying different sparse training and pruning methods, we test the pruned model in 18 different situations on the ZCU102 evaluation board. In order to improve detection speed as much as possible while ensuring detection accuracy, for the pruned model with the highest comprehensive performance, compared to the original model, the comparison results are as follows: the model size is reduced from 62 MB to 12 MB, which is only 19% of the origin; the number of parameters is reduced from 61,657,117 to 9,900,539, which is only 16% of the origin; the running time is reduced from 14.411 s to 6.828 s, which is only less than half of the origin, while the detection accuracy is decreased from 97% to 94.1%, which is only less than 3%.

An Ultralightweight Object Detection Network for Empty-Dish Recycling Robots

“…Wang et al [4] propose a YOLOv3-based dish detection network on an FPGA platform, and through different sparse training and pruning methods, the model size is reduced from 62 to 12 MB. Koubaa et al [28] present a real-world case study of deploying a face recognition application using the MTCNN detector and FaceNet recognizer and demonstrate that TensorRT optimization provides the fastest execution on edge devices.…”

“…Yue et al [1] use traditional YOLOv4 to detect dishes and achieve more than 96.00% high accuracy on precision, recall, and F1 values. Wang et al [4] use traditional YOLOv3 to detect 16 classes of dishes and achieve a mean average precision (mAP) of 96.40%. Yue et al [5] propose a dish grasp point extraction algorithm based on image processing technology, which can extract the grasp point coordinates of dishes in a 2-D plane.…”