Teat Pose Estimation via RGBD Segmentation for Automated Milking

Abstract

We present initial results in the development of a novel robot using RGBD cameras, image segmentation, and a simple teat pose estimation algorithm for automated milking. We relate on the analysis of the accuracy of different commercial RGBD cameras in realistic conditions. Although preliminary, our initial implementation shows that 2D image segmentation combined with point cloud processing can achieve repeatable millimeter-scale precision in estimating (synthetic) teat tip positions and cup attachment approach. The solution is also applicable in a cloud robotics setup, with GPU-based segmentation executed on an edge device or cloud.

Demo:Smart Waste Disposal with Edge-Cloud Continuum Architecture

“…Owing to the emerging use of robotic sorting facilities, recent research on cloud robotics suggests that the continuum can be extended to robotic arms that autonomously detect objects and their poses and subsequently grab and sort them [2].…”

Research on tracking algorithm of milking robot arm based on improved SURF-KCF

“…At present, the automatic milking system (AMS) [2][3] has been commercialized, and the positioning is realized by irradiating the cow's teat with laser light. In recent years, researchers have started to detect the position of cow teats from a visual perspective [4] to achieve higher milking efficiency [5] . Akhloufi M.A [6] development of a 3D vision system using a 3D-TOF camera and an RGBD camera for the detection of bovine udder position information; Aleksey Dorokhov [7] obtained an image point cloud using a 3D-TOF camera and segmented the point cloud using KNN nearest neighbor algorithm and Euclidean distance to achieve detection of cow teats; Pal [8] proposed a conceptual idea to combine TOF, thermography and RGBD imaging for teat identification; Ben Azouz [9] developed a vision system integrating optical stereo vision and thermal imaging, which was tested in various laboratory situations using a dummy thermal breast; Rastogi [10] applied the Haar feature extraction algorithm to milk cow teat detection, although there are differences in the lighting, color, position and size of the object, the teat can be detected from the cluttered background.…”