SET: Stereo Evaluation Toolbox for Combined Performance Assessment of Camera Systems, 3D Reconstruction and Visual SLAM

Heide, Nina Felicitas; Albrecht, Alexander; Heizmann, Michael

doi:10.1109/icicsp48821.2019.8958548

Cited by 4 publications

(3 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Most recent sensor benchmarks evaluated low cost consumer sensors and/or LiDARs, such as an actuated SICK LMS-200 (ground truth), a SwissRanger SR-4000, a Fotonic B70 and a Microsoft Kinect camera [8], a Hokuyo URG-04LX, Hukuyo UTM-30LX and Sick LMS-151 [9], a Microsoft Kinect, ZESS MultiCam 3k and 19k, SoftKinetic DepthSense 311, PMD Technologies 3k-S and Camcube 41k [10], Kinect and Asus Xtion [13], Kinect II [16], [17], Microsoft Kinect II and Asus Xtion Pro [14], Intel Realsense SR300 [11], Intel Realsense D415 [12] and a comparison of ten different LiDARs for applications in Autonomouts driving [18]. One exception is the SET framework for the evaluation of stereo matching used to test a Stereolabs ZED and a Roboception rc_visard 160 [19]. Unfortunately, it is not applicable in our benchmark, as most of the included sensors obtain 3D information via structured light (SL) or time of flight (ToF).…”

Section: Related Workmentioning

confidence: 99%

New Metrics for Industrial Depth Sensors Evaluation for Precise Robotic Applications

Cop¹,

Peters

Žagar³

et al. 2021

2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

View full text Add to dashboard Cite

Precise perception is one of the key enablers of autonomous robotic operations. The right selection of sensors significantly influences the overall performance of the system. This paper provides a systematic approach for evaluation of various sensors available on the market. The main focus is to assess the performance in use cases of short to medium distance operations, especially relevant for precise manipulation and/or quality control. The evaluation is based solely on depth data (point clouds). We use six metrics to evaluate the sensors and propose a novel approach for low-cost fabrication of benchmark targets. The evaluation experiments are conducted on different materials to simulate various industrial environments. Our results provide a qualitative and quantitative comparison of different characteristics of various sensors and can be used to select an appropriate device for specific conditions.

show abstract

Section: Related Workmentioning

confidence: 99%

New Metrics for Industrial Depth Sensors Evaluation for Precise Robotic Applications

Cop¹,

Peters

Žagar³

et al. 2021

2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

View full text Add to dashboard Cite

show abstract

“…Also, a qualitative visual assessment of label smoothness is possible. A scoring from 0 to 10 allows a detailed rating for experts with domain knowledge [12]. Here, 10 stands for the highest ACC possible.…”

Section: Accuracy (Acc) Of Processed Datamentioning

confidence: 99%

A Step towards Explainable Artificial Neural Networks in Image Processing by Dataset Assessment

Heide,

Albrecht,

Heizmann

2020

Tagungsband "Forum Bildverarbeitung 2020

View full text Add to dashboard Cite

We propose a methodology for generalized exploratory data analysis focusing on artificial neural network (ANN) methods. Our method is denoted IC-ACC due to the combined assessment of information content (IC) and accuracy (ACC) and aims at answering a frequently posed question in ANN research: ”What is good data?” As the dataset has the primary influence on the development of the model, IC-ACC provides a step towards explainable ANN methods in the pre-modeling stage by a better insight in the dataset. With this insight, detrimental data can be eliminated before a negative influence on the ANN performance occurs. IC-ACC constitutes a guideline to generate efficient and accurate data for a specific, data-driven ANN method. Moreover, we show that training an ANN for the semantic segmentation of 3D data from unstructured environments with IC-ACC-assessed and -customized training data contributes to a more efficient training. The IC-ACC method is demonstrated on application examples for the visual perception of robotic platforms.

show abstract

“…On the other hand, Halmetschlager-Funek et al [12] analyzed the precision, bias and lateral noise of sensors on different lightning conditions when observing objects with different materials. Heide et al [13] evaluated stereo cameras, using as metrics the point cloud density, the smoothness of the surface points captured from the walls, the consistency of the edges, and the mean distance between the 3D points and the ground truth surfaces along with a comparison of their surface normals. For simultaneous localization and mapping (SLAM) use cases, Neto et al [14] relied on the mean distance between the poses estimated by the SLAM algorithm and the ground truth poses, which were known because the sensors were mounted on a robotic arm for following a pre-programmed trajectory.…”

Section: Introductionmentioning

confidence: 99%

Comparison of 3D Sensors for Automating Bolt-Tightening Operations in the Automotive Industry

Dias

Simões²,

Soares³

et al. 2023

Sensors

View full text Add to dashboard Cite

Machine vision systems are widely used in assembly lines for providing sensing abilities to robots to allow them to handle dynamic environments. This paper presents a comparison of 3D sensors for evaluating which one is best suited for usage in a machine vision system for robotic fastening operations within an automotive assembly line. The perception system is necessary for taking into account the position uncertainty that arises from the vehicles being transported in an aerial conveyor. Three sensors with different working principles were compared, namely laser triangulation (SICK TriSpector1030), structured light with sequential stripe patterns (Photoneo PhoXi S) and structured light with infrared speckle pattern (Asus Xtion Pro Live). The accuracy of the sensors was measured by computing the root mean square error (RMSE) of the point cloud registrations between their scans and two types of reference point clouds, namely, CAD files and 3D sensor scans. Overall, the RMSE was lower when using sensor scans, with the SICK TriSpector1030 achieving the best results (0.25 mm ± 0.03 mm), the Photoneo PhoXi S having the intermediate performance (0.49 mm ± 0.14 mm) and the Asus Xtion Pro Live obtaining the higher RMSE (1.01 mm ± 0.11 mm). Considering the use case requirements, the final machine vision system relied on the SICK TriSpector1030 sensor and was integrated with a collaborative robot, which was successfully deployed in an vehicle assembly line, achieving 94% success in 53,400 screwing operations.

show abstract

SET: Stereo Evaluation Toolbox for Combined Performance Assessment of Camera Systems, 3D Reconstruction and Visual SLAM

Cited by 4 publications

References 12 publications

New Metrics for Industrial Depth Sensors Evaluation for Precise Robotic Applications

New Metrics for Industrial Depth Sensors Evaluation for Precise Robotic Applications

A Step towards Explainable Artificial Neural Networks in Image Processing by Dataset Assessment

Comparison of 3D Sensors for Automating Bolt-Tightening Operations in the Automotive Industry

Contact Info

Product

Resources

About