Automated Assembly Using 3D and 2D Cameras

Kleppe, Adam Leon; Bjørkedal, Asgeir; Larsen, Kristoffer; Egeland, Olav

doi:10.3390/robotics6030014

Cited by 5 publications

(4 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The model predictions reached parity with human accuracy levels. This indicated that the quality and consistency of demonstrations became a limiting factor and was a likely reason that our reported accuracy numbers were in line with some results reported for some similar systems [12,51,52] and significantly worse than some others [50]. The differences in applications and the accuracy of the ground truths made a comparison across applications difficult.…”

Section: Discussionsupporting

confidence: 85%

“…Finally, translation and rotation error are widely used. For an industrial assembly task, accuracies of <1 mm and <1 • were reported [50]; the prediction error in the pose of automotive objects was 1 cm and <5 • [12]; pose estimation for industrial work-pieces achieved an accuracy <10 mm and <2 • [51]; and for a bin-picking task, translation error was <23 mm and rotation <2.26 • [52].…”

mentioning

confidence: 99%

See 1 more Smart Citation

Cutting Pose Prediction from Point Clouds

Philipsen

Moeslund

2020

Sensors

View full text Add to dashboard Cite

The challenge of getting machines to understand and interact with natural objects is encountered in important areas such as medicine, agriculture, and, in our case, slaughterhouse automation. Recent breakthroughs have enabled the application of Deep Neural Networks (DNN) directly to point clouds, an efficient and natural representation of 3D objects. The potential of these methods has mostly been demonstrated for classification and segmentation tasks involving rigid man-made objects. We present a method, based on the successful PointNet architecture, for learning to regress correct tool placement from human demonstrations, using virtual reality. Our method is applied to a challenging slaughterhouse cutting task, which requires an understanding of the local geometry including the shape, size, and orientation. We propose an intermediate five-Degree of Freedom (DoF) cutting plane representation, a point and a normal vector, which eases the demonstration and learning process. A live experiment is conducted in order to unveil issues and begin to understand the required accuracy. Eleven cuts are rated by an expert, with 8 / 11 being rated as acceptable. The error on the test set is subsequently reduced through the addition of more training data and improvements to the DNN. The result is a reduction in the average translation from 1.5 cm to 0.8 cm and the orientation error from 4.59° to 4.48°. The method’s generalization capacity is assessed on a similar task from the slaughterhouse and on the very different public LINEMOD dataset for object pose estimation across view points. In both cases, the method shows promising results. Code, datasets, and other materials are available in Supplementary Materials.

show abstract

Section: Discussionsupporting

confidence: 85%

mentioning

confidence: 99%

Cutting Pose Prediction from Point Clouds

Philipsen

Moeslund

2020

Sensors

View full text Add to dashboard Cite

show abstract

“…The system first found a rough estimate of the target object posed with the 3D camera, and then the two-dimensional camera on the robotic arm was utilized to define a fine estimate of the target. This approach avoids the viewing range limitation of robotic cameras by estimating the rough location first to guide the robotic arm to the working area and demonstrates a cooperative structure of fixed cameras and robotic cameras to position the target accurately [14]. In addition, with the popularization of deep learning approaches based on computer vision, a learning-based approach to hand-eye coordination for robotic grasping has been developed to enhance the awareness of collaborative pick and place robotic operations.…”

Section: Vision-guided Robots In Manufacturingmentioning

confidence: 99%

Vision-Based Guiding System for Autonomous Robotic Corner Cleaning of Window Frames

Tung,

Al-Hussein,

Martinez

2023

Buildings

View full text Add to dashboard Cite

Corner cleaning is the most important manufacturing step of window framing to ensure aesthetic quality. After the welding process, the current methods to clean the welding seams lack quality control and adaptability. This increases rework, cost, and the waste produced in manufacturing and is largely due to the use of CNC cutting machines, as well as the reliance on manual inspection and weld seam cleaning. Dealing with manufacturing imperfections becomes a challenging task, as CNC machines rely on predetermined cleaning paths and frame information. To tackle such challenges using Industry 4.0 approaches and automation technology, such as robots and sensors, in this paper, a novel intelligent system is proposed to increase the process capacity to adapt to variability in weld cleaning conditions while ensuring quality through a combined approach of robot arms and machine vision that replaces the existing manual-based methods. Using edge detection to identify the window position and its orientation, artificial intelligence image processing techniques (Mask R-CNN model) are used to detect the window weld seam and to guide the robot manipulator in its cleaning process. The framework is divided into several modules, beginning with the estimation of a rough position for the purpose of guiding the robot toward the window target, followed by an image processing and detection module used in conjunction with instance segmentation techniques to segment the target area of the weld seam, and, finally, the generation of cleaning paths for further robot manipulation. The proposed robotic system is validated two-fold: first, in a simulated environment and then, in a real-world scenario, with the results obtained demonstrating the effectiveness and adaptability of the proposed system. The evaluation of the proposed framework shows that the trained Mask R-CNN can locate and quantify weld seams with 95% mean average precision (less than 1 cm).

show abstract

“…Feature matching algorithm can be divided into point-based matching algorithm (Kleppe et al , 2017; Yu et al , 2019), line-based matching algorithm (Miraldo et al , 2015; Lopez et al , 2015) and edge-based matching algorithm (Cai et al , 2016). The feature points are easy to extract by using point feature operator.…”

Section: Literature Reviewmentioning

confidence: 99%

A contour-guided pose alignment method based on Gaussian mixture model for precision assembly

Guo

Zhang

Shi

et al. 2021

View full text Add to dashboard Cite

Purpose The purpose of this study was to solve the problem of pose measurement of various parts for a precision assembly system. Design/methodology/approach A novel alignment method which can achieve high-precision pose measurement of microparts based on monocular microvision system was developed. To obtain the precise pose of parts, an area-based contour point set extraction algorithm and a point set registration algorithm were developed. First, the part positioning problem was transformed into a probability-based two-dimensional point set rigid registration problem. Then, a Gaussian mixture model was fitted to the template point set, and the contour point set is represented by hierarchical data. The maximum likelihood estimate and expectation-maximization algorithm were used to estimate the transformation parameters of the two point sets. Findings The method has been validated for accelerometer assembly on a customized assembly platform through experiments. The results reveal that the proposed method can complete letter-pedestal assembly and the swing piece-basal part assembly with a minimum gap of 10 µm. In addition, the experiments reveal that the proposed method has better robustness to noise and disturbance. Originality/value Owing to its good accuracy and robustness for the pose measurement of complex parts, this method can be easily deployed to assembly system.

show abstract

Automated Assembly Using 3D and 2D Cameras

Cited by 5 publications

References 19 publications

Cutting Pose Prediction from Point Clouds

Cutting Pose Prediction from Point Clouds

Vision-Based Guiding System for Autonomous Robotic Corner Cleaning of Window Frames

A contour-guided pose alignment method based on Gaussian mixture model for precision assembly

Contact Info

Product

Resources

About