ObjectFusion: An object detection and segmentation framework with RGB-D SLAM and convolutional neural networks

Tian, Guanzhong; Liu, Liang; Ri, JongHyok; Liu, Yong; Sun, Yi-Ran

doi:10.1016/j.neucom.2019.01.088

Cited by 36 publications

(10 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The specific detection process is as follows. Firstly, the selective search algorithm is utilized to select the candidate target regions in the target image, and nearly 2000 candidate target regions are extracted [10,11]. Secondly, the selected 2000 candidate target regions are cut out from the original images, and the cut images are trimmed according to the unified size of 227 × 227.…”

Section: Research Methods Based On Region Proposalmentioning

confidence: 99%

See 1 more Smart Citation

Application of local fully Convolutional Neural Network combined with YOLO v5 algorithm in small target detection of remote sensing image

Liu

et al. 2021

PLoS ONE

268

View full text Add to dashboard Cite

This exploration primarily aims to jointly apply the local FCN (fully convolution neural network) and YOLO-v5 (You Only Look Once-v5) to the detection of small targets in remote sensing images. Firstly, the application effects of R-CNN (Region-Convolutional Neural Network), FRCN (Fast Region-Convolutional Neural Network), and R-FCN (Region-Based-Fully Convolutional Network) in image feature extraction are analyzed after introducing the relevant region proposal network. Secondly, YOLO-v5 algorithm is established on the basis of YOLO algorithm. Besides, the multi-scale anchor mechanism of Faster R-CNN is utilized to improve the detection ability of YOLO-v5 algorithm for small targets in the image in the process of image detection, and realize the high adaptability of YOLO-v5 algorithm to different sizes of images. Finally, the proposed detection method YOLO-v5 algorithm + R-FCN is compared with other algorithms in NWPU VHR-10 data set and Vaihingen data set. The experimental results show that the YOLO-v5 + R-FCN detection method has the optimal detection ability among many algorithms, especially for small targets in remote sensing images such as tennis courts, vehicles, and storage tanks. Moreover, the YOLO-v5 + R-FCN detection method can achieve high recall rates for different types of small targets. Furthermore, due to the deeper network architecture, the YOL v5 + R-FCN detection method has a stronger ability to extract the characteristics of image targets in the detection of remote sensing images. Meanwhile, it can achieve more accurate feature recognition and detection performance for the densely arranged target images in remote sensing images. This research can provide reference for the application of remote sensing technology in China, and promote the application of satellites for target detection tasks in related fields.

show abstract

Section: Research Methods Based On Region Proposalmentioning

confidence: 99%

“…MS-OPN can be solved by gradient descent method. The optimization objective function is shown in Eq (10).…”

Section: Plos Onementioning

confidence: 99%

Application of local fully Convolutional Neural Network combined with YOLO v5 algorithm in small target detection of remote sensing image

Liu

et al. 2021

PLoS ONE

268

View full text Add to dashboard Cite

show abstract

“…Accurate depth perception is a prerequisite for several computer vision and robotics applications, such as simultaneous localization and mapping [1]- [3], object recognition [4]- [6], and object semantic segmentation [7]- [9]. Recently, commercial RGB-D cameras (e.g., Kinect, Realsense, ASUS Xtion) have been widely adopted as single-view depth sensors owing to their affordable price and portability.…”

Section: Introductionmentioning

confidence: 99%

Accurate Ground-Truth Depth Image Generation via Overfit Training of Point Cloud Registration using Local Frame Sets

Kim¹,

Kim²,

Shin³

et al. 2022

Preprint

View full text Add to dashboard Cite

Accurate three-dimensional perception is a fundamental task in several computer vision applications. Recently, commercial RGB-depth (RGB-D) cameras have been widely adopted as single-view depth-sensing devices owing to their efficient depth-sensing abilities. However, the depth quality of most RGB-D sensors remains insufficient owing to the inherent noise from a single-view environment. Recently, several studies have focused on the single-view depth enhancement of RGB-D cameras. Recent research has proposed deep-learning-based approaches that typically train networks using high-quality supervised depth datasets. The performance of current learningbased depth enhancement techniques is highly dependent on the quality of the ground-truth (GT) depth dataset; however, such high-quality GT datasets are difficult to obtain. In this study, we developed a novel method for high-quality GT depth generation based on an RGB-D stream dataset. First, we defined consecutive depth frames in a local spatial region as a local frame set. Then, the depth frames were aligned to a certain frame in the local frame set using an unsupervised point cloud registration scheme. The registration parameters were trained based on an overfit-training scheme, which was primarily used to construct a single GT depth image for each frame set. The final GT depth dataset was constructed using several local frame sets, and each local frame set was trained independently. The primary advantage of this study is that a high-quality GT depth dataset can be constructed under various scanning environments using only the RGB-D stream dataset, which can be used as a single-view depth supervision dataset. Moreover, our proposed method can be used as a new benchmark GT dataset for accurate performance evaluations. We evaluated our GT dataset on previously benchmarked GT depth datasets and demonstrated that our method is superior to state-of-the-art depth enhancement frameworks.

show abstract

“…With the increasing amount of available low-cost consumer-grade depth sensors such as Microsoft Kinect [18], Intel RealSense [19], and depth cameras becoming a standard feature in some flagship mobile phones, we are moving towards an era where RGB-Depth (RGB-D) sensors are as common as regular RGB cameras. Object detection and segmentation with RGB-D sensors has been widely used in recent years, such as Canonical Correlation Analysis (CCA)-based multi-view Convolutional Neural Networks (CNN) [20], using regular point clouds in addition to multi-views for point cloud recognition [21], fusing CNNs with simultaneous localization and mapping in order to perform object segmentation [22], employing multi-modal deep neural networks and Dempster Shafer evidence theory to achieve the task of object recognition [23], or adopting multifoveated point clouds [24].…”

Section: Introductionmentioning

confidence: 99%

Reconstruction of 3D Object Shape Using Hybrid Modular Neural Network Architecture Trained on 3D Models from ShapeNetCore Dataset

Kulikajevas

Maskeliūnas

Damaševičius

et al. 2019

Sensors

View full text Add to dashboard Cite

Depth-based reconstruction of three-dimensional (3D) shape of objects is one of core problems in computer vision with a lot of commercial applications. However, the 3D scanning for point cloud-based video streaming is expensive and is generally unattainable to an average user due to required setup of multiple depth sensors. We propose a novel hybrid modular artificial neural network (ANN) architecture, which can reconstruct smooth polygonal meshes from a single depth frame, using a priori knowledge. The architecture of neural network consists of separate nodes for recognition of object type and reconstruction thus allowing for easy retraining and extension for new object types. We performed recognition of nine real-world objects using the neural network trained on the ShapeNetCore model dataset. The results evaluated quantitatively using the Intersection-over-Union (IoU), Completeness, Correctness and Quality metrics, and qualitative evaluation by visual inspection demonstrate the robustness of the proposed architecture with respect to different viewing angles and illumination conditions.

show abstract

ObjectFusion: An object detection and segmentation framework with RGB-D SLAM and convolutional neural networks

Cited by 36 publications

References 10 publications

Application of local fully Convolutional Neural Network combined with YOLO v5 algorithm in small target detection of remote sensing image

Application of local fully Convolutional Neural Network combined with YOLO v5 algorithm in small target detection of remote sensing image

Accurate Ground-Truth Depth Image Generation via Overfit Training of Point Cloud Registration using Local Frame Sets

Reconstruction of 3D Object Shape Using Hybrid Modular Neural Network Architecture Trained on 3D Models from ShapeNetCore Dataset

Contact Info

Product

Resources

About