A Mobile System for Scene Monitoring and Object Retrieval

Birkas, D.; Birkas, K.; Popa, Tiberiu

doi:10.1145/2915926.2915933

Cited by 4 publications

(5 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We have run our algorithm successfully on a large number and wide variety of point sets to ensure its robustness: e.g. segmented silhouettes of noisy sensed data [BBP16] (Figure ), but also contrived shapes representing multiply connected open/closed curves, also corrupted with high noise and outliers (Figures , , , and ). In addition, we show its improvements on prior work.…”

Section: Resultsmentioning

confidence: 99%

See 1 more Smart Citation

FitConnect: Connecting Noisy 2D Samples by Fitted Neighbourhoods

Ohrhallinger

Wimmer

2018

Computer Graphics Forum

View full text Add to dashboard Cite

We propose a parameter‐free method to recover manifold connectivity in unstructured 2D point clouds with high noise in terms of the local feature size. This enables us to capture the features which emerge out of the noise. To achieve this, we extend the reconstruction algorithm HNN‐Crust, which connects samples to two (noise‐free) neighbours and has been proven to output a manifold for a relaxed sampling condition. Applying this condition to noisy samples by projecting their k‐nearest neighbourhoods onto local circular fits leads to multiple candidate neighbour pairs and thus makes connecting them consistently an NP‐hard problem. To solve this efficiently, we design an algorithm that searches that solution space iteratively on different scales of k. It achieves linear time complexity in terms of point count plus quadratic time in the size of noise clusters. Our algorithm FitConnect extends HNN‐Crust seamlessly to connect both samples with and without noise, performs as local as the recovered features and can output multiple open or closed piecewise curves. Incidentally, our method simplifies the output geometry by eliminating all but a representative point from noisy clusters. Since local neighbourhood fits overlap consistently, the resulting connectivity represents an ordering of the samples along a manifold. This permits us to simply blend the local fits for denoising with the locally estimated noise extent. Aside from applications like reconstructing silhouettes of noisy sensed data, this lays important groundwork to improve surface reconstruction in 3D. Our open‐source algorithm is available online.

show abstract

Section: Resultsmentioning

confidence: 99%

“…Applications and Reconstruction from Noisy Samples Birkas et al . [BBP16] show a system for retrieving objects from mobile sensed data by segmenting them via clustering. From these point clusters, (partially occluded) silhouettes can be extracted, which are noisy due to sensor artefacts.…”

Section: Related Workmentioning

confidence: 99%

FitConnect: Connecting Noisy 2D Samples by Fitted Neighbourhoods

Ohrhallinger

Wimmer

2018

Computer Graphics Forum

View full text Add to dashboard Cite

show abstract

“…[HZCP16] jointly optimize segmentation and dense reconstruction to generate more accurate semantic models. Birkas et al [BBP16] take the single RGBD image as input and retrieve the appropriate 3D models from the pre-built database. Shi et al [SLX * 16] propose a data-driven approach to modeling contextual information covering both intra-object part relations and interobject object layouts.…”

Section: D Scene Segmentationmentioning

confidence: 99%

InstanceFusion: Real‐time Instance‐level 3D Reconstruction Using a Single RGBD Camera

Peng

et al. 2020

Computer Graphics Forum

View full text Add to dashboard Cite

We present InstanceFusion, a robust real-time system to detect, segment, and reconstruct instance-level 3D objects of indoor scenes with a hand-held RGBD camera. It combines the strengths of deep learning and traditional SLAM techniques to produce visually compelling 3D semantic models. The key success comes from our novel segmentation scheme and the efficient instancelevel data fusion, which are both implemented on GPU. Specifically, for each incoming RGBD frame, we take the advantages of the RGBD features, the 3D point cloud, and the reconstructed model to perform instance-level segmentation. The corresponding RGBD data along with the instance ID are then fused to the surfel-based models. In order to sufficiently store and update these data, we design and implement a new data structure using the OpenGL Shading Language. Experimental results show that our method advances the state-of-the-art (SOTA) methods in instance segmentation and data fusion by a big margin. In addition, our instance segmentation improves the precision of 3D reconstruction, especially in the loop closure. InstanceFusion system runs 20.5Hz on a consumer-level GPU, which supports a number of augmented reality (AR) applications (e.g., 3D model registration, virtual interaction, AR map) and robot applications (e.g., navigation, manipulation, grasping). To facilitate future research and reproduce our system more easily, the source code, data, and the trained model are released on Github: https://github.com/Fancomi2017/InstanceFusion. CCS Concepts • Computing methodologies → Scene understanding; Vision for robotics; Perception;

show abstract

“…Applications for reconstructing curves from noisy samples Birkas et al [BBP16] take sensed RGBD images and cluster points to extract silhouettes. With the reconstructed silhouette curve, the corresponding object can be segmented and visualized in the RGB part of the image.…”

Section: Related Workmentioning

confidence: 99%

“…Figure 9 shows segmented silhouettes of sensed 3D objects [BBP16]. Here we use the noise extent estimated by FITCON-NECT for denoising since we do not have information about the actual error from the sensor for these data.…”

Section: Silhouettes With Estimated Noisementioning

confidence: 99%

StretchDenoise: Parametric Curve Reconstruction with Guarantees by Separating Connectivity from Residual Uncertainty of Samples

Ohrhallinger¹,

Wimmer²

2018

Preprint

View full text Add to dashboard Cite

We reconstruct a closed denoised curve from an unstructured and highly noisy 2D point cloud. Our proposed method uses a twopass approach: Previously recovered manifold connectivity is used for ordering noisy samples along this manifold and express these as residuals in order to enable parametric denoising. This separates recovering low-frequency features from denoising high frequencies, which avoids over-smoothing. The noise probability density functions (PDFs) at samples are either taken from sensor noise models or from estimates of the connectivity recovered in the first pass. The output curve balances the signed distances (inside/outside) to the samples. Additionally, the angles between edges of the polygon representing the connectivity become minimized in the least-square sense. The movement of the polygon's vertices is restricted to their noise extent, i.e., a cut-off distance corresponding to a maximum variance of the PDFs. We approximate the resulting optimization model, which consists of higher-order functions, by a linear model with good correspondence. Our algorithm is parameter-free and operates fast on the local neighborhoods determined by the connectivity. We augment a least-squares solver constrained by a linear system to also handle bounds. This enables us to guarantee stochastic error bounds for sampled curves corrupted by noise, e.g., silhouettes from sensed data, and we improve on the reconstruction error from ground truth.

show abstract

A Mobile System for Scene Monitoring and Object Retrieval

Cited by 4 publications

References 23 publications

FitConnect: Connecting Noisy 2D Samples by Fitted Neighbourhoods

FitConnect: Connecting Noisy 2D Samples by Fitted Neighbourhoods

InstanceFusion: Real‐time Instance‐level 3D Reconstruction Using a Single RGBD Camera

StretchDenoise: Parametric Curve Reconstruction with Guarantees by Separating Connectivity from Residual Uncertainty of Samples

Contact Info

Product

Resources

About