Object Skeleton Extraction in Natural Images by Fusing Scale-Associated Deep Side Outputs

Abstract

Object skeleton is a useful cue for object detection, complementary to the object contour, as it provides a structural representation to describe the relationship among object parts. While object skeleton extraction in natural images is a very challenging problem, as it requires the extractor to be able to capture both local and global image context to determine the intrinsic scale of each skeleton pixel. Existing methods rely on per-pixel based multi-scale feature computation, which results in difficult modeling and high time consumption. In this paper, we present a fully convolutional network with multiple scale-associated side outputs to address this problem. By observing the relationship between the receptive field sizes of the sequential stages in the network and the skeleton scales they can capture, we introduce a scale-associated side output to each stage. We impose supervision to different stages by guiding the scale-associated side outputs toward groundtruth skeletons of different scales. The responses of the multiple scaleassociated side outputs are then fused in a scale-specific way to localize skeleton pixels with multiple scales effectively. Our method achieves promising results on two skeleton extraction datasets, and significantly outperforms other competitors.

DeepFlux for Skeletons in the Wild

“…We conduct experiments on five well-known, challenging datasets, including three for skeleton detection (SK-LARGE [34], SK506 [35], WH-SYMMAX [33]) and two for local symmetry detection (SYM-PASCAL [17], SYM-MAX300 [45]). We distinguish between the two tasks by associating skeletons with a foreground object, and local symmetry detection with any symmetric structure, be it a foreground object or background clutter.…”

“…Deep learning-based methods: With the popularization of CNNs, deep learning-based methods [35,34,17,24,51,22] igure 2. The DeepFlux pipeline.…”

“…The recent surge of work in convolutional neural networks (CNNs) has lead to vast improvements in the performance of object skeleton detection algorithms [35,34,17,24,51,22]. These existing CNN-based methods usually derive from Holistically-Nested Edge Detection (HED) [47], and frame the problem as binary pixel classification.…”

Recent advances in deep learning for object detection

“…Moreover, a video consists of hundreds of frames and thus con- There are also many other real-world applications based on object detection such as vehicle detection [227,228,229], traffic-sign detection [230,231] and skeleton detection [232,233].…”

SkeletonNet: Shape Pixel to Skeleton Pixel

“…Skeleton extraction is a widely researched area in the last 10 years. However, the most recent works are mainly focused on the extracting skeleton from the RGB images [22] [11], which involves segmentation or detection of the objects and extract the skeleton at the same time. Also, an extensive research is done either on edge detection [8][3][27] [23] or segmentation [27] [10] individually.…”