Real-time extraction of surface patches with associated uncertainties by means of Kinect cameras

Olesen, SØren Maagaard; Lyder, Simon; Kraft, Dirk; Krüger, Norbert; Jessen, Jeppe Barsøe

doi:10.1007/s11554-012-0261-x

Cited by 22 publications

(30 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Our pipeline starts with initial surface normals estimation at every input point location, much like previous work does [41,43]. This is a prerequisite for the MLS process.…”

Section: Normal Estimationmentioning

confidence: 99%

Real-time scene reconstruction and triangle mesh generation using multiple RGB-D cameras

Meerits

Nozick

Saitō

2017

J Real-Time Image Proc

View full text Add to dashboard Cite

We present a novel 3D reconstruction system that can generate a stable triangle mesh using data from multiple RGB-D sensors in real time for dynamic scenes. The first part of the system uses moving least squares (MLS) point set surfaces to smooth and filter point clouds acquired from RGB-D sensors. The second part of the system generates triangle meshes from point clouds. The whole pipeline is executed on the GPU and is tailored to scale linearly with the size of the input data. Our contributions include changes to the MLS method for improving meshing, a fast triangle mesh generation method and GPU implementations of all parts of the pipeline.

show abstract

“…Our pipeline starts with initial surface normals estimation at every input point location, much like previous work does [41,43]. This is a prerequisite for the MLS process.…”

Section: Normal Estimationmentioning

confidence: 99%

Real-time scene reconstruction and triangle mesh generation using multiple RGB-D cameras

Meerits

Nozick

Saitō

2017

J Real-Time Image Proc

View full text Add to dashboard Cite

show abstract

“…In computer vision, extraction processes for structures such as edges [10] and patchlets (or texlets) [57,60] have been discussed. They represent edge and surface information which require a different kind of parameterization.…”

Section: Retinamentioning

confidence: 99%

“…Due to page number limitations we cannot give a precise definition of the uncertainties here but refer to [60]. In the following paragraphs we sketch the computation in both cases.…”

Section: D Texlet π Tmentioning

confidence: 99%

“…In [60] the point-wise reconstruction uncertainty has been investigated, leading to a Kinect noise model dependent on the distance to the observed object and distance to the principal point. This noise model is used to add Gaussian noise to the reconstructed points and, as in the stereo case, by using Monte Carlo simulation, calculate the uncertainty covariance matrix.…”

Section: D Texlet π Tmentioning

confidence: 99%

“…This noise model is used to add Gaussian noise to the reconstructed points and, as in the stereo case, by using Monte Carlo simulation, calculate the uncertainty covariance matrix. We can achieve a computational speed of around 5-10 Hz for scenes consisting of ∼4000 texlets, depending on the number of RANSAC iterations used in and the Monte Carlo simulation for uncertainty calculation (for details, see, see [60]). …”

Section: D Texlet π Tmentioning

confidence: 99%

See 2 more Smart Citations

Using surfaces and surface relations in an Early Cognitive Vision system

Kraft

Mustafa

Popovic

et al. 2015

Machine Vision and Applications

Self Cite

View full text Add to dashboard Cite

We present a deep hierarchical visual system with two parallel hierarchies for edge and surface information. In the two hierarchies, complementary visual information is represented on different levels of granularity together with the associated uncertainties and confidences. At all levels geometric and appearance information is coded explicitly in 2D and 3D allowing to access this information separately and to link between the different levels. We demonstrate the advantages of such hierarchies in three applications covering grasping, view-point independent object representation, and pose estimation.

show abstract

Robotic harvesting of the occluded fruits with a precise shape and position reconstruction approach

Gong

Wang

et al. 2021

Journal of Field Robotics

View full text Add to dashboard Cite

Occlusion is one of the key factors affecting the success rate of vision‐based fruit‐picking robots. It is important to accurately locate and grasp the occluded fruit in field applications, However, there is yet no universal and effective solution. In this paper, a high‐precision estimation method of spatial geometric features of occluded targets based on deep learning and multisource images is presented, enabling the selective harvest robot to envision the whole target fruit as if its occlusions do not exist. First, RGB, depth and infrared images are acquired. And pixel‐level matched RGB‐D‐I fusion images are obtained by image registration. Second, aiming at the problem of detecting the occluded tomatoes in the greenhouse, an extended Mask‐RCNN network is designed to extract the target tomato. The target segmentation accuracy is improved by 7.6%. Then, for partially occluded tomatoes, a shape and position restoration method is used to recover the obscured tomato. This algorithm can extract tomato radius and centroid coordinates directly from the restored depth image. The mean Intersection over Union is 0.895, and the centroid position error is 0.62 mm for the occluded rate under 25% and the illuminance between 1 and 12 KLux. And hereby a dual‐arm robotic harvesting system is improved to achieve a picking time of 11 s per fruit, an average gripping accuracy of 8.21 mm, and an average picking success rate of 73.04%. The proposed approach realizes a high‐fidelity geometrics reconstruction instead of mere image style restoration, which endows the robot with the ability to see through obstacles in the field scenes and improves its operational success rate in its result.

show abstract

Real-time extraction of surface patches with associated uncertainties by means of Kinect cameras

Cited by 22 publications

References 18 publications

Real-time scene reconstruction and triangle mesh generation using multiple RGB-D cameras

Real-time scene reconstruction and triangle mesh generation using multiple RGB-D cameras

Using surfaces and surface relations in an Early Cognitive Vision system

Robotic harvesting of the occluded fruits with a precise shape and position reconstruction approach

Contact Info

Product

Resources

About