Attribute-aware Semantic Segmentation of Road Scenes for Understanding Pedestrian Orientations

Abstract

Semantic segmentation is an interesting task to many deep learning researchers for scene understanding. However, recognizing details about object's attributes can be more informative and also helpful for a better scene understanding in intelligent vehicle uses. This paper introduces a method for simultaneous semantic segmentation and pedestrian attributes recognition. A modified dataset built on top of the Cityscapes dataset is created by adding attribute classes corresponding to pedestrian orientation attributes. The proposed method extends the SegNet model and is trained by using both the original and the attribute-enriched datasets. Based on an experiment, the proposed attribute-aware semantic segmentation approach shows the ability to slightly improve the performance on the Cityscapes dataset, which is capable of expanding its classes in this case through additional data training.

Importance-Aware Semantic Segmentation with Efficient Pyramidal Context Network for Navigational Assistant Systems

“…With the development of deep learning, many training methods are advanced to solve somewhat-aware problems like difficult-aware [20] and attribute-aware [21] SS. Li et al [20] considered that different pixels own different ranks of difficulty and propose a difficulty-aware network to pay attention to more difficult pixels.…”

Attribute-Aware Loss Function for Accurate Semantic Segmentation Considering the Pedestrian Orientations

“…For the attribute-aware semantic segmentation task, which can basically be regarded as a multi-task learning goal, we have been conducting a study despite some limitations. A part of work regarding this initial concept has been published in paper [24].…”

“…Since the walking direction is also important for the driver to determine his/her actions, body orientation is designated as an attribute to enrich the class of person. We introduced the initial concept of our work in [24], but it was not effective enough to achieve good accuracy in classifying the orientations. In this paper, we consider four orientation classes as considered in developing an autonomous vehicle [36], including back (0 • ), right (90 • ), front (180 • ), and left (270 • ).…”

“…Moreover, we consider IoU for the person class (IoU p ) to particularly see how the attribute of pedestrian orientation affects the segmentation model in classifying the object class from which those attributes are derived. Detailed definitions on these metrics are provided in our previous paper [24]. In short, gAcc is obtained by the number of correctly-predicted pixels across all classes compared to the number of all pixels, and cAcc is similar to gAcc but calculated for each class, then averaged for all classes.…”

“…In a preliminary work [23], this task was designed in three successive stages, i.e., semantic segmentation, object detection, and attribute recognition, but our current work proposes an approach to train the model in an end-to-end process. The concept was initially introduced in our previous paper [24], while this paper reveals additional aspects along with a newly extended dataset and base models. This paper focuses on pedestrian detection and selects body orientation as its attribute.…”