Recursive Inference for Prediction of Objects in Urban Environments

Abstract

Future advancements in robotic navigation and mapping rest to a large extent on robust, efficient and more advanced semantic understanding of the surrounding environment. The existing semantic mapping approaches typically consider small number of semantic categories, require complex inference or large number of training examples to achieve desirable performance. In the proposed work we present an efficient approach for predicting locations of generic objects in urban environments by means of semantic segmentation of a video into object and nonobject categories. We exploit widely available exemplars of non-object categories (such as road, buildings, vegetation) and use geometric cues which are indicative of the presence of object boundaries to gather the evidence about objects regardless of their category. We formulate the object/non-object semantic segmentation problem in the Conditional Random Field framework, where the structure of the graph is induced by a minimum spanning tree computed over a 3D point cloud, yielding an efficient algorithm for an exact inference. The chosen 3D representation naturally lends itself for on-line recursive belief updates with a simple soft data association mechanism. We carry out extensive experiments on videos of urban environments acquired by a moving vehicle and show quantitatively and qualitatively the benefits of our proposal.

Semantic segmentation with heterogeneous sensor coverages

“…-stands for values that are no provided in [21] or are not possible to obtain with the approach of [3]. 2nd: We have already shown very simple and effective features for each sensing modality.…”

“…In this case, we want to use the visual information jointly with the 3D information from a laser range finder or from a depth sensor. The steps described here are a summary of the proposals in [3] for outdoors and [4] for indoors.…”

“…First we show how to solve the problem using only visual information acquired from a monocular camera. Then, we describe the proposal of [3] when we have two sensor modalities with common spatial coverage, in this case visual and 3D information. And finally, we show our strategy for the same two sensor modalities but with different spatial coverage.…”

“…In the outdoors case [3], the 13-D feature vector is composed from the mean and standard deviation of LAB and RGB color spaces of the superpixel, and by the vertical superpixel centroid in the image. In the indoors case [4], the 8-D feature vector is composed from the mean and standard deviation of LAB color space, the vertical superpixel centroid in the image, and the entropy of the probability distribution for the superpixel's boundaries belonging to the dominant vanishing points.…”

“…The use of these modalities creates opportunities for using richer set of features as well as obtaining better coverage. In the common approaches for semantic segmentation proposed [3,20], the features and the graphical model are constrained to the area that is covered simultaneously by both sensors. Areas covered by only one of the sensors are not considered.…”