Visual servoing for low-cost SCARA robots using an RGB-D camera as the only sensor

Ðurović, Petra; Grbić, Ratko; Cupec, Robert

doi:10.1080/00051144.2018.1461771

Cited by 6 publications

(4 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…There are mainly two types of eye-to-hand methods to reconstruct the contour points from RGB-D image to 3D manipulator workspace: geometrical methods (Meng and Zhuang, 2001; Boby and Saha, 2016; Ðurović et al , 2017) and neural network (NN) method (Sun et al , 2021; Wu et al , 2013; Jones and Vernon, 1994; Levine et al , 2018). The camera position setting and camera inner parameters are not strictly demanded in the NN eye-to-hand method, as compared with geometry methods.…”

Section: Related Workmentioning

confidence: 99%

Enhance pick-and-place performance using multimodal interaction in operation environment

Guo

Chen

2023

View full text Add to dashboard Cite

Purpose Currently, the vision and depth information obtained from the eye-to-hand RGB-D camera can apply to the reconstruction of the three-dimensional (3D) environment for a robotic operation workspace. The reconstructed 3D space contributes to a symmetrical and equal observation view for robots and humans, which can be considered a digital twin (DT) environment. The purpose of this study is to enhance the robot skill in the physical workspace, although the artificial intelligence (AI) technique has high performance of the robotic operation in the known environments. Design/methodology/approach A multimodal interaction framework is proposed in DT operation environments. Findings A fast image-based target segmentation technique is combined in the 3D reconstruction of the robotic operation environment from the eye-to-hand camera, thus expediting the 3D DT environment generation without accuracy loss. A multimodal interaction interface is integrated into the DT environment. Originality/value The users are supported to operate the virtual objects in the DT environment using speech, mouse and keyboard simultaneously. The humans’ operations in 3D DT virtual space are recorded, and cues are provided for the robot’s operations in practice.

show abstract

Section: Related Workmentioning

confidence: 99%

Enhance pick-and-place performance using multimodal interaction in operation environment

Guo

Chen

2023

View full text Add to dashboard Cite

show abstract

“…In recent years, technologies such as deep learning and computer vision have gradually become research hotspots and have been applied to tasks such as intelligent assembly and automatic fetching. P. Ðurović [3] proposed a two-step hand-eye calibration visual servo system, and a simple grasp planning method based on vision. The method is designed for low-cost visually oriented robots, the positioning function is realized based on a visual servo, the tag tracking and depth information is provided by the rgb-d camera, and there are no encoders or other sensors.…”

Section: Computer Visionmentioning

confidence: 99%

Trajectory Prediction of Assembly Alignment of Columnar Precast Concrete Members with Deep Learning

2019

View full text Add to dashboard Cite

During the construction of prefabricated building, there are some problems such as a time consuming, low-level of automation when precast concrete members are assembled and positioned. This paper presents vision-based intelligent assembly alignment guiding technology for columnar precast concrete members. We study the video images of assembly alignment of the hole at the bottom of the precast concrete members and the rebar on the ground. Our goal is to predict the trajectory of the moving target in a future moment and the movement direction at each position during the alignment process by assembly image sequences. However, trajectory prediction is still subject to the following challenges: (1) the effect of external environment (illumination) on image quality; (2) small target detection in complex backgrounds; (3) low accuracy of trajectory prediction results based on the visual context model. In this paper, we use mask and adaptive histogram equalization to improve the quality of the image and improved method to detect the targets. In addition, aiming at the low position precision of trajectory prediction based on the context model, we propose the end point position-matching equation according to the principle of end point pixel matching of the moving target and fixed target, as the constraint term of the loss function to improve the prediction accuracy of the network. In order to evaluate comprehensively the performance of the proposed method on the trajectory prediction in the assembly alignment task, we construct the image dataset, use Hausdorff distance as the evaluation index, and compare with existing prediction methods. The experimental results show that, this framework is better than the existing methods in accuracy and robustness at the prediction of assembly alignment motion trajectory of columnar precast concrete members.

show abstract

“…In addition, depth cameras have also been applied. In hand-eye calibration ( Ðurović et al., 2017 ; Du et al., 2018 ; Ge et al., 2022 ). TOF (Time of flight) depth camera is a camera with active vision measurement function.…”

Section: Introductionmentioning

confidence: 99%

A novel hand-eye calibration method of picking robot based on TOF camera

Zhang

Cheng

et al. 2023

Front. Plant Sci.

View full text Add to dashboard Cite

Aiming at the stability of hand-eye calibration in fruit picking scene, a simple hand-eye calibration method for picking robot based on optimization combined with TOF (Time of Flight) camera is proposed. This method needs to fix the TOF depth camera at actual and calculated coordinates of the peach the end of the robot, operate the robot to take pictures of the calibration board from different poses, and record the current photographing poses to ensure that each group of pictures is clear and complete, so as to use the TOF depth camera to image the calibration board. Obtain multiple sets of calibration board depth maps and corresponding point cloud data, that is, “eye” data. Through the circle center extraction and positioning algorithm, the circle center points on each group of calibration plates are extracted, and a circle center sorting method based on the vector angle and the center of mass coordinates is designed to solve the circle center caused by factors such as mirror distortion, uneven illumination and different photographing poses. And through the tool center point of the actuator, the coordinate value of the circle center point on the four corners of each group of calibration plates in the robot end coordinate system is located in turn, and the “hand” data is obtained. Combined with the SVD method, And according to the obtained point residuals, the weight coefficients of the marker points are redistributed, and the hand-eye parameters are iteratively optimized, which improves the accuracy and stability of the hand-eye calibration. the method proposed in this paper has a better ability to locate the gross error under the environment of large gross errors. In order to verify the feasibility of the hand-eye calibration method, the indoor picking experiment was simulated, and the peaches were identified and positioned by combining deep learning and 3D vision to verify the proposed hand-eye calibration method. The JAKA six-axis robot and TuYang depth camera are used to build the experimental platform. The experimental results show that the method is simple to operate, has good stability, and the calibration plate is easy to manufacture and low in cost. work accuracy requirements.

show abstract

Visual servoing for low-cost SCARA robots using an RGB-D camera as the only sensor

Cited by 6 publications

References 24 publications

Enhance pick-and-place performance using multimodal interaction in operation environment

Enhance pick-and-place performance using multimodal interaction in operation environment

Trajectory Prediction of Assembly Alignment of Columnar Precast Concrete Members with Deep Learning

A novel hand-eye calibration method of picking robot based on TOF camera

Contact Info

Product

Resources

About