Learning Motion in Feature Space: Locally-Consistent Deformable Convolution Networks for Fine-Grained Action Detection

Mac, Khoi-Nguyen C.; Joshi, Dhiraj; Yeh, Raymond A.; Xiong, Jinjun; Feris, Rogério; Minh, Nguyễn Đức

doi:10.48550/arxiv.1811.08815

Cited by 2 publications

(3 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For instance, the authors in [1] demonstrated that the prediction accuracy of the proposed DCN increases from 70% to 75% on the image semantic segmentation dataset (CityScapes). Significant prediction accuracy improvement is also observed in human motion recognition task [11] [12], action detection task [13] [14] and intelligent medical monitoring and treatment [15] [16].…”

Section: Introductionmentioning

confidence: 89%

“…The unique feature makes it attractive in visual recognition tasks with geometric variations such as lighting and rotation. There have been many deformable convolution architectures proposed recently [1], [14]. They typically include two standard convolution operations.…”

Section: A Deformable Convolutional Networkmentioning

confidence: 99%

“…The structures of DCNs mainly differ on the index reuse and can be roughly divided into two categories. The first category of DCNs has a unique index for each data in the feature plane and the indices are reused across the different channels [14]. The second category of DCNs also has the indices shared across the feature channels, but it has a unique index for each data in each convolution window [1].…”

Section: A Deformable Convolutional Networkmentioning

confidence: 99%

See 2 more Smart Citations

Energy-Efficient Accelerator Design for Deformable Convolution Networks

Xu,

Chu,

Liu

et al. 2021

Preprint

View full text Add to dashboard Cite

Deformable convolution networks (DCNs) proposed to address the image recognition with geometric or photometric variations typically involve deformable convolution that convolves on arbitrary locations of input features. The locations change with different inputs and induce considerable dynamic and irregular memory accesses which cannot be handled by classic neural network accelerators (NNAs). Moreover, bilinear interpolation (BLI) operation that is required to obtain deformed features in DCNs also cannot be deployed on existing NNAs directly. Although a general purposed processor (GPP) seated along with classic NNAs can process the deformable convolution, the processing on GPP can be extremely slow due to the lack of parallel computing capability.To address the problem, we develop a DCN accelerator on existing NNAs to support both the standard convolution and deformable convolution. Specifically, for the dynamic and irregular accesses in DCNs, we have both the input and output features divided into tiles and build a tile dependency table (TDT) to track the irregular tile dependency at runtime. With the TDT, we further develop an on-chip tile scheduler to handle the dynamic and irregular accesses efficiently. In addition, we propose a novel mapping strategy to enable parallel BLI processing on NNAs and apply layer fusion techniques for more energy-efficient DCN processing. According to our experiments, the proposed accelerator achieves orders of magnitude higher performance and energy efficiency compared to the typical computing architectures including ARM, ARM+TPU, and GPU with 6.6% chip area penalty to a classic NNA.

show abstract