MonoCInIS: Camera Independent Monocular 3D Object Detection using Instance Segmentation

Heylen, Jonas; Wolf, Mark De; Dawagne, Bruno; Proesmans, Marc; Gool, Luc Van; Abbeloos, Wim; Abdelkawy, Hazem; Reino, Daniel Olmeda

doi:10.1109/iccvw54120.2021.00108

Cited by 17 publications

(5 citation statements)

References 56 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The association stage of the tracked objects depends on quasi-dense similarity learning to identify objects of interest in different positions and views based on visual features. MonoCInIS [111] presents a method that utilizes instance segmentation in order to estimate the object's pose. The proposed method is camera-independent in order to account for variable camera perspectives.…”

Section: D-data-based Techniquesmentioning

confidence: 99%

MonoGhost: Lightweight Monocular GhostNet 3D Object Properties Estimation for Autonomous Driving

El-Dawy,

El-Zawawi,

El-Habrouk

2023

Robotics

View full text Add to dashboard Cite

Effective environmental perception is critical for autonomous driving; thus, the perception system requires collecting 3D information of the surrounding objects, such as their dimensions, locations, and orientation in space. Recently, deep learning has been widely used in perception systems that convert image features from a camera into semantic information. This paper presents the MonoGhost network, a lightweight Monocular GhostNet deep learning technique for full 3D object properties estimation from a single frame monocular image. Unlike other techniques, the proposed MonoGhost network first estimates relatively reliable 3D object properties depending on efficient feature extractor. The proposed MonoGhost network estimates the orientation of the 3D object as well as the 3D dimensions of that object, resulting in reasonably small errors in the dimensions estimations versus other networks. These estimations, combined with the translation projection constraints imposed by the 2D detection coordinates, allow for the prediction of a robust and dependable Bird’s Eye View bounding box. The experimental outcomes prove that the proposed MonoGhost network performs better than other state-of-the-art networks in the Bird’s Eye View of the KITTI dataset benchmark by scoring 16.73% on the moderate class and 15.01% on the hard class while preserving real-time requirements.

show abstract

Section: D-data-based Techniquesmentioning

confidence: 99%

MonoGhost: Lightweight Monocular GhostNet 3D Object Properties Estimation for Autonomous Driving

El-Dawy,

El-Zawawi,

El-Habrouk

2023

Robotics

View full text Add to dashboard Cite

show abstract

“…Monoloco [4] tackles the fundamentally ill-posed problem of 3D human localization from monocular RGB images by predicting confidence intervals through a loss function based on the Laplace distribution with a lightweight 3D localization neural network. MonoIS [6] provides a category-level pose estimation method based on instance segmentation, using camera-independent geometric reasoning to compare with the varying camera viewpoints and intrinsic of different datasets. Additionally, utilizing segmentation masks results in enhanced outcomes for this strategy [17], [18].…”

Section: A Monocular-based Approach For Object Detection and 3d Local...mentioning

confidence: 99%

“…Monoloco achieved outstanding results on the pedestrian localization task with the KITTI-3D dataset benchmark, making real-time predictions of 3D position with a monocular camera as the only sensor. Recent work has exploited the rich semantic information from segmentation masks to leverage 2D object detection and use for 3D position prediction such as in [6]. The accuracy of these methods is limited to the training dataset such as KITTI-3D [7] and nuScenes [8].…”

Section: Introductionmentioning

confidence: 99%

MonoIS3DLoc: Simulation to Reality Learning Based Monocular Instance Segmentation to 3D Objects Localization From Aerial View

Tran

Nguyen

et al. 2023

IEEE Access

View full text Add to dashboard Cite

3D object detection and localization based on only a monocular camera always faces its fundamental ill-posed issue to estimate 3D information. In combination with deep neural networks, recent researches have shown encouraging results to tackle this issue. However, most of them are only applied to street-view cameras based on several available small-size datasets and the 3D prediction accuracy of these methods is still low in comparison to traditional estimation methods using stereo-cameras. With the development of drone delivery applications in city spaces, it is also necessary to have a similar method to detect objects and estimate their 3D position from an aerial view. We proposed a novel Simulation to Reality approach to predict the object's 3D position from an aerial view. An instance segmentation of an object is used as an intermediate representation not only to create a very large dataset for training by simulation but also to minimize the gap between simulation and reality. We designed a feed-forward neural network to predict the 3D position from instance segmentation and integrated it with a range-attention classification to improve accuracy, especially for 3D object detection at far distances. To evaluate our methods, we created two simulation datasets: one for cross-validation with other state-of-the-art methods and the other one for practical experiments on a real drone with a monocular camera. The experiment's results demonstrate that we not only achieve better accuracy than the state-of-the-art methods using the monocular camera by testing on the same KITTI-3D dataset but also reach close to the accuracy of a stereo-based technique. Since our model is lightweight, we successfully deployed it on a companion computer on the real drone and the results of practical experiences are promising.

show abstract

“…Moreover, these representation models impose challenges for generalizing the object shapes to the real‐world scene (Zakharov et al., 2020). Finally, object segmentation information with masks is helpful in shape reconstruction (Beker et al., 2020) and a 3D object location (Heylen et al., 2021). Likewise, these prior‐based methods suffer from the extra task network, domain shift, and 3D GT data as supervisory signals.…”

Section: Introductionmentioning

confidence: 99%

Monocular 3D object detection for construction scene analysis

Shen,

Jiao,

Zhang

et al. 2023

Computer aided Civil Eng

View full text Add to dashboard Cite

Three‐dimensional (3D) object detection, that is, localizing and classifying all critical objects in a 3D space, is essential for downstream construction scene analysis tasks. However, accurate instance segmentation, few 2D object segmentation and 3D object detection data sets, high‐quality feature representations for depth estimation, and limited 3D cues from a single red‐green‐blue (RGB) image pose significant challenges to 3D object detection and severely hinder its practical applications. In response to these challenges, an improved cascade‐based network with a transformer backbone and a boundary‐patch‐refinement method is proposed to build hierarchical features and refine object boundaries, resulting in better results in 2D object detection and instance segmentation. Furthermore, a novel self‐supervised monocular depth learning method is proposed to extract better feature representations for depth estimation from construction site video data with unknown camera parameters. Additionally, a pseudo‐LiDAR point cloud method and a 3D object detection method with a density‐based clustering algorithm are proposed to detect 3D objects in a construction scene without help from 3D labels, which will serve as a good foundation for other downstream 3D tasks. Finally, the proposed model is evaluated for object instance segmentation and depth estimation on the moving objects in construction sites (MOCS) and construction scene data sets. It brings a 9.16% gain in terms of mean average precision (mAP) for object detection and a 4.92% gain in mask mAP for object instance segmentation. The average order accuracy and relative mean error for depth estimation are improved by 0.94% and 60.56%, respectively. This study aims to overcome the challenges and limitations of 3D object detection and facilitate practical applications in construction scene analysis.

show abstract

MonoCInIS: Camera Independent Monocular 3D Object Detection using Instance Segmentation

Cited by 17 publications

References 56 publications

MonoGhost: Lightweight Monocular GhostNet 3D Object Properties Estimation for Autonomous Driving

MonoGhost: Lightweight Monocular GhostNet 3D Object Properties Estimation for Autonomous Driving

MonoIS3DLoc: Simulation to Reality Learning Based Monocular Instance Segmentation to 3D Objects Localization From Aerial View

Monocular 3D object detection for construction scene analysis

Contact Info

Product

Resources

About