Robotic 3D Vision-Guided System for Half-Sheep Cutting Robot

Abstract

Sheep body segmentation robot can improve production hygiene, product quality, and cutting accuracy, which is a huge change for traditional manual segmentation. With reference to the New Zealand sheep body segmentation specification, a vision system for Cartesian coordinate robot cutting half-sheep was developed and tested. The workflow of the vision system was designed and the image acquisition device with an Azure Kinect sensor was developed. Furthermore, a LabVIEW software with the image processing algorithm was then integrated with the RGBD image acquisition device in order to construct an automatic vision system. Based on Deeplab v3+ networks, an image processing system for locating ribs and spine was employed. Taking advantage of the location characteristics of ribs and spine in the split half-sheep, a calculation method of cutting line based on the key points is designed to determine five cutting curves. The seven key points are located by convex points of ribs and spine and the root of hind leg. Using the conversion relation between depth image and the space coordinates, the 3D coordinates of the curves were computed. Finally, the kinematics equation of the rectangular coordinate robot arm is established, and the 3D coordinates of the curves are converted into the corresponding motion parameters of the robot arm. The experimental results indicated that the automatic vision system had a success rate of 98.4% in the cutting curves location, 4.2 s time consumption per half-sheep, and approximately 1.3 mm location error. The positioning accuracy and speed of the vision system can meet the requirements of the sheep cutting production line. The vision system shows that there is potential to automate even the most challenging processing operations currently carried out manually by human operators.

Posicionamiento de ayuda visual utilizando compresión de nube de puntos y cámaras RGB-D para manipuladores robóticos

“…Actualmente, el posicionamiento de los manipuladores robóticos se realiza mediante sensores inerciales, tales como IMU y enconder, que se encargan determinar la orientación del robot, cámaras visuales con imágenes RGB, sensores LIDAR [10]- [12], entre otros. Por otro lado, el uso de cámaras que permiten determinar la pose de diferentes objetos en entornos variados ha permitido la penetración de estas tecnologías a nivel industrial [13]- [17].…”

Cutting of sheep carcass using 3D point cloud with dual-robot system

“…Machine vision technologies are widely used in meat processing to solve problems of meat positioning and robot work trajectory planning. Mu et al [28] designed a 3D vision-guided system for half sheep carcass cutting robot. The images of the half-sheep carcass were obtained by the Azure Kinect camera, and the ribs and spine of the half-sheep carcass were identified using the Deeplab v3+ model.…”

A Real-Time Semantic Segmentation Method of Sheep Carcass Images Based on ICNet

“…Current, the CNN shows excellent performance in object detection [12][13][14][15], image recognition [16,17], and natural language processing [18] due to the automatic extraction of deep and shallow layers features of the image. At the same time, it has high precision and strong robustness.…”