Estimation of Pedestrian Pose Orientation Using Soft Target Training Based on Teacher–Student Framework

Abstract

Semi-supervised learning is known to achieve better generalisation than a model learned solely from labelled data. Therefore, we propose a new method for estimating a pedestrian pose orientation using a soft-target method, which is a type of semi-supervised learning method. Because a convolutional neural network (CNN) based pose orientation estimation requires large numbers of parameters and operations, we apply the teacher–student algorithm to generate a compressed student model with high accuracy and compactness resembling that of the teacher model by combining a deep network with a random forest. After the teacher model is generated using hard target data, the softened outputs (soft-target data) of the teacher model are used for training the student model. Moreover, the orientation of the pedestrian has specific shape patterns, and a wavelet transform is applied to the input image as a pre-processing step owing to its good spatial frequency localisation property and the ability to preserve both the spatial information and gradient information of an image. For a benchmark dataset considering real driving situations based on a single camera, we used the TUD and KITTI datasets. We applied the proposed algorithm to various driving images in the datasets, and the results indicate that its classification performance with regard to the pose orientation is better than that of other state-of-the-art methods based on a CNN. In addition, the computational speed of the proposed student model is faster than that of other deep CNNs owing to the shorter model structure with a smaller number of parameters.

Computer Vision-Based Assessment of Autistic Children: Analyzing Interactions, Emotions, Human Pose, and Life Skills

“…Full-range yaw estimation is significantly less common than narrowrange estimation, as most known HPE datasets concentrate mainly on frontal to profile views. To determine yaw, recent approaches classify poses into coarse-grained bins/classes [99]- [101]. The limitations of using these existing methods are mentioned as follows: they do not predict pitch and roll; full-range yaw estimation is not robust to occlusions; unreliable pose estimation in heavy noise and jitter environments; low-resolution video; and utilization of multi-camera over monocular images.…”

Monocular Pedestrian Orientation Recognition Based on Capsule Network for a Novel Collision Warning System

“…In this work our intention is not only pedestrian detection but also pedestrian walking direction recognition, exploring whether the pedestrian is moving into one of the eight orientations that are previously defined (front, back, right, left, right-front, right-back, left-front, left-back) in order to anticipate the crossing intention of the pedestrian. Once the pedestrian is detected, the bounding box is used to determine his walking direction, in this context considerable work has been done using Lidar-based approaches [16] (2016) and Image-based approaches [17,18,19]. For this work we will evoke the Image-based one as it is our aim of interest.…”

“…This method has a performance of 49.7% on images that fall in the same orientation as the ground truth and a performance of 81.3% for images that fall in same and adjacent orientation bins as the ground truth. Another approach is proposed by Heo et al [19] (2019) based on a semi-supervised model for estimating pedestrian pose orientation using a teacher-student model where the output of the teacher model is used to train the student model. Enzweiler and Gravila proposed in [18] (2010) a joint method for pedestrian classification and orientation estimation based on a probabilistic framework using 4 pedestrian orientations (Back, Front, Left and Right).…”

Marine Vessel Re-Identification: A Large-Scale Dataset and Global-and-Local Fusion-Based Discriminative Feature Learning

“…Cai et al created an attention network built on multi-scale and multipart masks [5]. Heo et al developed a teacher-student-based semi-supervised framework to estimate a person's attitude and orientation [6]. However, these state-of-the-art part-based and fine-grained methods for person re-identification do not work well on marine vessels well because the attitude of a ship can change greatly from different viewpoints.…”

“…Re-identification of vessels is similar to the reidentification of pedestrians [2]- [6] or vehicles [12], [14]- [17]. It is important for security surveillance, and in the creation of intelligent transportation systems (ITS) at sea [10].…”