A Comprehensive Survey of Depth Completion Approaches

Khan, Muhammad Ahmed Ullah; Nazir, Danish; Pagani, Alain; Mokayed, Hamam; Liwicki, Marcus; Stricker, Didier; Afzal, Muhammad Zeshan

doi:10.3390/s22186969

Cited by 8 publications

(8 citation statements)

References 78 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The guided techniques utilize auxiliary information, such as RGB information, to guide the depth completion process, while the non-guided techniques rely only on sparse depth measurements. It is worth mentioning that guided approaches usually produce better results and, thus, are commonly used [14]. Researchers have developed various techniques, primarily convolutional neural network (CNN) solutions.…”

Section: Related Workmentioning

confidence: 99%

“…For evaluation, we followed the most commonly used metrics: root mean squared error (RMSE), mean absolute error (MAE), root mean squared error of the inverse depth (iRMSE), and mean absolute error of the inverse depth (iMAE), formulated in Equations ( 7), (8) ( 9) and (10), respectively [14].…”

Section: Dataset and Evaluation Metricsmentioning

confidence: 99%

“…In image-guided methods, textural information from RGB images is utilized for more accurate depth estimation. Most of the ongoing research applies depth completion techniques on the entire field of view and designs complex learning models to improve depth estimation in challenging scenarios, such as when objects are small or far away, or to enhance depth measurements at objects' boundaries [14,15]. However, to our knowledge, none of the existing work has integrated prior knowledge about scene objects or utilized pre-processes that generate additional features that can guide the depth completion process with simpler deep networks.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Guided Depth Completion with Instance Segmentation Fusion in Autonomous Driving Applications

El-Yabroudi

Abdel‐Qader

Bazuin

et al. 2022

Sensors

View full text Add to dashboard Cite

Pixel-level depth information is crucial to many applications, such as autonomous driving, robotics navigation, 3D scene reconstruction, and augmented reality. However, depth information, which is usually acquired by sensors such as LiDAR, is sparse. Depth completion is a process that predicts missing pixels’ depth information from a set of sparse depth measurements. Most of the ongoing research applies deep neural networks on the entire sparse depth map and camera scene without utilizing any information about the available objects, which results in more complex and resource-demanding networks. In this work, we propose to use image instance segmentation to detect objects of interest with pixel-level locations, along with sparse depth data, to support depth completion. The framework utilizes a two-branch encoder–decoder deep neural network. It fuses information about scene available objects, such as objects’ type and pixel-level location, LiDAR, and RGB camera, to predict dense accurate depth maps. Experimental results on the KITTI dataset showed faster training and improved prediction accuracy. The proposed method reaches a convergence state faster and surpasses the baseline model in all evaluation metrics.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Dataset and Evaluation Metricsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Guided Depth Completion with Instance Segmentation Fusion in Autonomous Driving Applications

El-Yabroudi

Abdel‐Qader

Bazuin

et al. 2022

Sensors

View full text Add to dashboard Cite

show abstract

“…In today's world, machine learning (ML) applications have become integral to various aspects of our lives (Kim et al, 2018). Whether it is transportation, healthcare, document analysis, or any other industry (Adewumi et al, 2022;Alkhaled et al, 2023;Kanchi et al, 2022;Khan et al, 2022), the proliferation of ML applications has been rapid (Amershi et al, 2019). ML is a fundamental component of artificial intelligence (AI), where data and algorithms are utilized to enable AI systems to mimic human behavior and cognition.…”

Section: Introductionmentioning

confidence: 99%

Supportive Environment for Better Data Management Stage in the Cycle of ML Process

Alkhaled,

Khamis

2023

AIA

View full text Add to dashboard Cite

The objective of this study is to explore the process of developing Artificial Intelligence (AI) and machine learning (ML) applications to establish an optimal support environment. The primary stages of ML include problem understanding, data management, model building, model deployment, and maintenance. This paper specifically focuses on examining the data management stage of ML development and the challenges it presents, as it is crucial for achieving accurate end models. During this stage, the major obstacle encountered was the scarcity of adequate data for model training, particularly in domains where data confidentiality is a concern. The work aimed to construct and enhance a framework that would assist researchers and developers in addressing the insufficiency of data during the data management stage. The framework incorporates various data augmentation techniques, enabling the generation of new data from the original dataset along with all the required files for detection challenges. This augmentation process improves the overall performance of ML applications by increasing both the quantity and quality of available data, thereby providing the model with the best possible input. The tool can be accessed using the following link https://github.com/TahaKh99/Image_Augmentor.

show abstract

“…Many studies use a combination of LiDAR sensors and cameras. Table 1 shows a comparison of their use [ 22 ].…”

Section: Introductionmentioning

confidence: 99%

Optimization of Trash Identification on the House Compound Using a Convolutional Neural Network (CNN) and Sensor System

Naf’an

Sulaiman

Ali

2023

Sensors

View full text Add to dashboard Cite

This study aims to optimize the object identification process, especially identifying trash in the house compound. Most object identification methods cannot distinguish whether the object is a real image (3D) or a photographic image on paper (2D). This is a problem if the detected object is moved from one place to another. If the object is 2D, the robot gripper only clamps empty objects. In this study, the Sequential_Camera_LiDAR (SCL) method is proposed. This method combines a Convolutional Neural Network (CNN) with LiDAR (Light Detection and Ranging), with an accuracy of ±2 mm. After testing 11 types of trash on four CNN architectures (AlexNet, VGG16, GoogleNet, and ResNet18), the accuracy results are 80.5%, 95.6%, 98.3%, and 97.5%. This result is perfect for object identification. However, it needs to be optimized using a LiDAR sensor to determine the object in 3D or 2D. Trash will be ignored if the fast scanning process with the LiDAR sensor detects non-real (2D) trash. If Real (3D), the trash object will be scanned in detail to determine the robot gripper position in lifting the trash object. The time efficiency generated by fast scanning is between 13.33% to 59.26% depending on the object's size. The larger the object, the greater the time efficiency. In conclusion, optimization using the combination of a CNN and a LiDAR sensor can identify trash objects correctly and determine whether the object is real (3D) or not (2D), so a decision may be made to move the trash object from the detection location.

show abstract

A Comprehensive Survey of Depth Completion Approaches

Cited by 8 publications

References 78 publications

Guided Depth Completion with Instance Segmentation Fusion in Autonomous Driving Applications

Guided Depth Completion with Instance Segmentation Fusion in Autonomous Driving Applications

Supportive Environment for Better Data Management Stage in the Cycle of ML Process

Optimization of Trash Identification on the House Compound Using a Convolutional Neural Network (CNN) and Sensor System

Contact Info

Product

Resources

About