Point-cloud based 3D object detection and classification methods for self-driving applications: A survey and taxonomy

Fernandes, Duarte; Silva, Alexandre Ferreira da; Névoa, Rafael; Simões, Cláudia; Gonzalez, Dibet Garcia; Guevara, Miguel Ángel; Nováis, Paulo; Monteiro, João L.; Melo‐Pinto, Pedro

doi:10.1016/j.inffus.2020.11.002

Cited by 182 publications

(97 citation statements)

References 113 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Semantic segmentation was inspired by the success of Deep Learning methods in producing an accurate result [ 10 , 13 , 14 ], but these techniques require an extremely large amount of data to train the network. Such large datasets may be difficult to obtain, or not provide adequate information, such as the case of man-made structures captured by sensors that only provide colourless point clouds.…”

Section: Introductionmentioning

confidence: 99%

Underwater Object Recognition Using Point-Features, Bayesian Estimation and Semantic Information

Himri

Ridao

Gracias

2021

Sensors

View full text Add to dashboard Cite

This paper proposes a 3D object recognition method for non-coloured point clouds using point features. The method is intended for application scenarios such as Inspection, Maintenance and Repair (IMR) of industrial sub-sea structures composed of pipes and connecting objects (such as valves, elbows and R-Tee connectors). The recognition algorithm uses a database of partial views of the objects, stored as point clouds, which is available a priori. The recognition pipeline has 5 stages: (1) Plane segmentation, (2) Pipe detection, (3) Semantic Object-segmentation and detection, (4) Feature based Object Recognition and (5) Bayesian estimation. To apply the Bayesian estimation, an object tracking method based on a new Interdistance Joint Compatibility Branch and Bound (IJCBB) algorithm is proposed. The paper studies the recognition performance depending on: (1) the point feature descriptor used, (2) the use (or not) of Bayesian estimation and (3) the inclusion of semantic information about the objects connections. The methods are tested using an experimental dataset containing laser scans and Autonomous Underwater Vehicle (AUV) navigation data. The best results are obtained using the Clustered Viewpoint Feature Histogram (CVFH) descriptor, achieving recognition rates of 51.2%, 68.6% and 90%, respectively, clearly showing the advantages of using the Bayesian estimation (18% increase) and the inclusion of semantic information (21% further increase).

show abstract

Section: Introductionmentioning

confidence: 99%

Underwater Object Recognition Using Point-Features, Bayesian Estimation and Semantic Information

Himri

Ridao

Gracias

2021

Sensors

View full text Add to dashboard Cite

show abstract

“…Regarding the object detection in point cloud data, models addressed in the literature have increasingly improved their detection capabilities. Generally, models in the literature have been positioned in two categories: 3D and 2D CNN-based approaches, where different data representation, backbone networks, and multi-scale feature learning techniques might be adopted [ 4 ].…”

Section: Related Workmentioning

confidence: 99%

“…They have a deeper pipeline and process a larger amount of data. For instance, typically a point cloud comprises between 100 k–120 k points [ 4 ], where each point contains data related to euclidean distance and signal reflection, i.e., 128 bits for translating each point information. Although solutions in the literature differ in the design of its pipeline, they share the same type of backbone, i.e., a stage responsible for extracting features from input data, which are based on CNN, and therefore due to the amount of data and arithmetic operations is the most computational demanding stage of the pipeline.…”

Section: Introductionmentioning

confidence: 99%

Resource-Constrained Onboard Inference of 3D Object Detection and Localisation in Point Clouds Targeting Self-Driving Applications

Silva

Fernandes

Névoa

et al. 2021

Sensors

Self Cite

View full text Add to dashboard Cite

Research about deep learning applied in object detection tasks in LiDAR data has been massively widespread in recent years, achieving notable developments, namely in improving precision and inference speed performances. These improvements have been facilitated by powerful GPU servers, taking advantage of their capacity to train the networks in reasonable periods and their parallel architecture that allows for high performance and real-time inference. However, these features are limited in autonomous driving due to space, power capacity, and inference time constraints, and onboard devices are not as powerful as their counterparts used for training. This paper investigates the use of a deep learning-based method in edge devices for onboard real-time inference that is power-effective and low in terms of space-constrained demand. A methodology is proposed for deploying high-end GPU-specific models in edge devices for onboard inference, consisting of a two-folder flow: study model hyperparameters’ implications in meeting application requirements; and compression of the network for meeting the board resource limitations. A hybrid FPGA-CPU board is proposed as an effective onboard inference solution by comparing its performance in the KITTI dataset with computer performances. The achieved accuracy is comparable to the PC-based deep learning method with a plus that it is more effective for real-time inference, power limited and space-constrained purposes.

show abstract

“…Firstly, it reduces the total processing required, facilitating the real-time analysis necessary for e.g, self-driving vehicles and machinery (cf. [109] for a detailed review). Secondly, it prevents the smoothing and distortion that inevitably occurs when data is projected onto a planar image grid, so retains the original data quality and resolution.…”

Section: From 2-d Raster To 3-d Point Cloudmentioning

confidence: 99%

The Potential of Machine Learning for a More Responsible Sourcing of Critical Raw Materials

Ghamisi¹,

Shahi²,

Duan³

et al. 2021

IEEE J. Sel. Top. Appl. Earth Observations Remote Sensing

View full text Add to dashboard Cite

The digitization and automation of the raw material sector is required to attain the targets set by the Paris Agreements and support the sustainable development goals defined by the United Nations. While many aspects of the industry will be affected, most of the technological innovations will require smart imaging sensors. In this review, we assess the relevant recent developments of Machine Learning for the processing of imaging sensor data. We first describe the main imagers and the acquired data types as well as the platforms on which they can be installed. We briefly describe radiometric and geometric corrections as these procedures have been already described extensively in previous works. We focus on the description of innovative processing workflows and illustrate the most prominent approaches with examples. We also provide a list of available resources, codes, and libraries for researchers at different levels, from students to senior researchers, willing to explore novel methodologies on the challenging topics of raw material extraction, classification, and process automatization.

show abstract

Point-cloud based 3D object detection and classification methods for self-driving applications: A survey and taxonomy

Cited by 182 publications

References 113 publications

Underwater Object Recognition Using Point-Features, Bayesian Estimation and Semantic Information

Underwater Object Recognition Using Point-Features, Bayesian Estimation and Semantic Information

Resource-Constrained Onboard Inference of 3D Object Detection and Localisation in Point Clouds Targeting Self-Driving Applications

The Potential of Machine Learning for a More Responsible Sourcing of Critical Raw Materials

Contact Info

Product

Resources

About