Deep Neural Networks for Road Sign Detection and Embedded Modeling Using Oblique Aerial Images

Abstract

Oblique photogrammetry-based three-dimensional (3D) urban models are widely used for smart cities. In 3D urban models, road signs are small but provide valuable information for navigation. However, due to the problems of sliced shape features, blurred texture and high incline angles, road signs cannot be fully reconstructed in oblique photogrammetry, even with state-of-the-art algorithms. The poor reconstruction of road signs commonly leads to less informative guidance and unsatisfactory visual appearance. In this paper, we present a pipeline for embedding road sign models based on deep convolutional neural networks (CNNs). First, we present an end-to-end balanced-learning framework for small object detection that takes advantage of the region-based CNN and a data synthesis strategy. Second, under the geometric constraints placed by the bounding boxes, we use the scale-invariant feature transform (SIFT) to extract the corresponding points on the road signs. Third, we obtain the coarse location of a single road sign by triangulating the corresponding points and refine the location via outlier removal. Least-squares fitting is then applied to the refined point cloud to fit a plane for orientation prediction. Finally, we replace the road signs with computer-aided design models in the 3D urban scene with the predicted location and orientation. The experimental results show that the proposed method achieves a high mAP in road sign detection and produces visually plausible embedded results, which demonstrates its effectiveness for road sign modeling in oblique photogrammetry-based 3D scene reconstruction.

The Influence of Point Cloud Accuracy from Image Matching on Automatic Preparation of Training Datasets for Object Detection in UAV Images

“…The potential that oblique photogrammetry brings is quite significant. Firstly, it provides a data source with distinct advantages: multiple views from different perspectives and significantly different image scales [19]. In addition, oblique photogrammetry carries the possibility of obtaining information about the location of an object in a terrain system and the use of the multitemporal feature [20].…”

Unsupervised Adversarial Domain Adaptation with Error-Correcting Boundaries and Feature Adaption Metric for Remote-Sensing Scene Classification

“…In the early stage of development, traditional machine learning methods have been used for scene classification tasks, such as support vector machine and bag of words [2,3]. Recently, deep learning methods have been proven to be effective for extracting image features [4][5][6][7][8], and many studies have demonstrated effective scene classification performance with the help of deep learning from various novel perspectives including self-supervised learning [9], data augmentation [10], feature fusion [11][12][13][14][15], reconstructing networks [16][17][18][19][20][21][22][23], integration of spectral and spatial information [24], balancing global and local features, refining feature maps through encoding method [25], adding a new mechanism [26,27], as well as introducing a new network [28], open set problem [29], and noisy label distillation [30]. However, a lack of annotated data has restricted the development of deep learning methods in scene classification due to the high cost of annotating data.…”