Semantic Segmentation with Transfer Learning for Off-Road Autonomous Driving

Abstract

Since the state-of-the-art deep learning algorithms demand a large training dataset, which is often unavailable in some domains, the transfer of knowledge from one domain to another has been a trending technique in the computer vision field. However, this method may not be a straight-forward task considering several issues such as original network size or large differences between the source and target domain. In this paper, we perform transfer learning for semantic segmentation of off-road driving environments using a pre-trained segmentation network called DeconvNet. We explore and verify two important aspects regarding transfer learning. First, since the original network size was very large and did not perform well for our application, we proposed a smaller network, which we call the light-weight network. This light-weight network is half the size to the original DeconvNet architecture. We transferred the knowledge from the pre-trained DeconvNet to our light-weight network and fine-tuned it. Second, we used synthetic datasets as the intermediate domain before training with the real-world off-road driving data. Fine-tuning the model trained with the synthetic dataset that simulates the off-road driving environment provides more accurate results for the segmentation of real-world off-road driving environments than transfer learning without using a synthetic dataset does, as long as the synthetic dataset is generated considering real-world variations. We also explore the issue whereby the use of a too simple and/or too random synthetic dataset results in negative transfer. We consider the Freiburg Forest dataset as a real-world off-road driving dataset.

Deep Learning to Unveil Correlations between Urban Landscape and Population Health

“…Rather interestingly, in order to deal with this problem, it is possible to resort to an increasing set of pre-trained deep learning models that can be used for the task of transfer learning [15], i.e., models that are able to represent the input space into a set of latent variables on the basis of a mapping mechanism, usually a deep neural network, learned on a large (external and independent) data set, so that the relationships between such latent variables and the outcomes can be later learned on a specific and smaller data set. A well-known example is Inception-v3, a convolutional neural network trained on more than a million images from the ImageNet database (http://www.image-net.org).…”

Sensors and Sensing for Intelligent Vehicles

“…Concerning the different topics and subtopics, we have identified up to seven main categories, and some sub-categories that are presented in the following list (the number of papers per each category/sub-category is enclosed in parentheses): Object detection and scene understanding (11) Vehicle detection and tracking (4): [ 13 , 14 , 15 , 16 ]. Scene segmentation and interpretation (7) Road segmentation (2): [ 17 , 18 ]. Shadow detection (1): [ 19 ].…”

“…Scene segmentation and interpretation (7) Road segmentation (2): [ 17 , 18 ]. Shadow detection (1): [ 19 ].…”

“…However, this method may not be a straightforward task considering issues such as original network size or large differences between the source and target domains. In [ 17 ], transfer learning is applied for semantic segmentation of off-road driving environments using a lightweight deconvolutional network (depicted in Figure 9 ) which is half the size of the original DeconvNet architecture. Transfer learning and fine-tuning is applied from the original pre-trained DeconvNet to the lightweight version.…”

Transfer Learning for Urban Landscape Clustering and Correlation with Health Indexes

“…Rather interestingly, in order to deal with this problem, it is possible to resort to an increasing set of pre-trained deep learning models that can be used for the task of transfer learning [4], i.e. models that are able to represent the input space into a set of latent variables on the basis of a mapping mechanism, usually a deep neural network, learned on a large (external) data set, so that the relationships between such latent variables and the outcomes can be later learned on a specific and smaller data set.…”