Graph convolutional networks (GCNs) achieve promising performance for skeleton-based action recognition. However, restricted by the neighborhood of feature aggregation, the spatial-temporal graph convolution in most GCNs lacks the flexibility of feature extraction. In this paper, we present a novel architecture, named Graph Convolutional skeleton Transformer (GCsT), which can flexibly capture local-global contexts. In GCsT, Transfomer and GCNs play complementary roles for feature representations. Specially, hierarchical GCNs capture local topological information at varying levels. A spatial transformer attention module models the correlations between joint pairs in global topology, which relaxes the constraints of graph convolution. A temporal transformer attention module is designed with long-short term attention, which learns all inter-frame dependencies effectively. GCsT has an effective combination of desirable properties, namely, dynamical attention and global context in Transformer, as well as hierarchy and local topology structure in GCNs. Furthermore, incorporating Transformer allows additional information to be naturally introduced into our model. The final GCsT exhibits a strong expressive capability. To our knowledge, this is the first attempt to design a novel plug-andplay Transformer block integrated into GCNs for skeleton action recognition. We validate the proposed GCsT by conducting extensive experiments, which achieves state-of-theart performance on NTU RGB+D, NTU RGB+D 120 and Northwestern-UCLA datasets.

Object detectors typically use bounding box regressors to improve the accuracy of object localization. Currently, the two types of bounding box regression loss are n-norm-based and intersection over union (IoU)-based. However, we found that these two types of losses have their drawbacks. First, for n-norm-based loss, large-scale objects are more likely to obtain a larger penalty than the smaller ones when calculating localization errors, which will cause regression loss imbalance. Second, n-norm-based loss has symmetry so that when the predicted bounding boxes are in some unique symmetrical relationships (i.e., Symmetric Trap), the regression loss remains unchanged. Third, for IoU-based loss, the overlap area and the union area do not change as the shape or relative position of two bounding boxes changes in some cases(i.e., Area Maze). To address these problems, we propose the scale balanced loss(L SB), which is asymmetric, position-sensitive, and scale-invariant. First, in order to obtain the property of scale invariance, it is designed as a fraction to eliminate the scale information contained in the numerator and denominator. Second, by incorporating the Euclidean distance between different corner points instead of the area, L SB is sensitive to the changes of coordinates of any corner point, so as to solve the area maze problem. Finally, by incorporating the diagonals of the overlap and the smallest enclosing rectangle, this fraction is no longer symmetric, thus solving the symmetry trap problem. To validate the proposed algorithm, we have replaced the n-norm-based loss of YOLOv3 and SSD with L GIoU and L SB and evaluate their performance on Pascal Visual Object Classes and Microsoft Common Objects in Context benchmarks. The final results show that L SB has improved their average precisions at different IoU thresholds and scales. We envision that this regression loss can also improve the performance of other visual tasks.

No abstract