Design Light-weight 3D Convolutional Networks for Video Recognition Temporal Residual, Fully Separable Block, and Fast Algorithm

Wang, Haonan; Lin, Jun; Wang, Zhongfeng

doi:10.48550/arxiv.1905.13388

Cited by 1 publication

(2 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Efficient spatiotemporal feature computation: Numerous works have developed approaches to make video processing more efficient, either through using less evidence [59,1,3,39] or through more efficient processing [40,43,35,60,52,50,27,28].…”

Section: Related Workmentioning

confidence: 99%

“…Due to this representational power they are currently the best performing models on action related tasks like action recognition [42,21,14,6], action quality assessment [31,30], skills assessment [5], action detection [12]. This representation power comes at a cost of increased computational complexity [60,50,58,13]. Hadidi et al [13] recently conducted an exhaustive comparison of various CNN's from the perspective of computational cost.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

HalluciNet-ing Spatiotemporal Representations Using a 2D-CNN

Parmar¹,

Morris²

2019

Preprint

View full text Add to dashboard Cite

Spatiotemporal representations learnt using 3D convolutional neural networks (CNN's) are currently the stateof-the-art approaches for action related tasks. However, 3D-CNN's are notoriously known for being memory and compute resource intensive. 2D-CNN's, on the other hand, are much lighter on computing resource requirements, and are faster. However, 2D-CNN's performance on action related tasks is generally inferior to that of 3D-CNN's. Also, where as 3D-CNN's simultaneously attend to appearance and salient motion patterns, 2D-CNN's are known to take shortcuts and recognize actions just from attending to background, which is not very meaningful. Taking inspiration from the fact that we, humans, can intuit how the actors will act and objects will be manipulated through years of experience and general understanding of the "how the world works," we suggest a way to combine the best attributes of 2D-and 3D-CNN's -we propose to hallucinate spatiotemporal representations as computed by 3D-CNN's, using a 2D-CNN. We believe that requiring the 2D-CNN to "see" into the future, would encourage it gain deeper about actions, and how scenes evolve by providing a stronger supervisory signal. Hallucination task is treated rather as an auxiliary task, while the main task is any other action related task such as, action recognition. Thorough experimental evaluation shows that hallucination task indeed helps improve performance on action recognition, action quality assessment, and dynamic scene recognition. From practical standpoint, being able to hallucinate spatiotemporal representations without an actual 3D-CNN, would enable deployment in resource-constrained scenarios such as lowerend phones and edge devices, and/or with lower bandwidth. This translates to pervasion of Video Analytics Software as a Service (VA SaaS), for e.g., automated physiotherapy options for financially challenged demographic.

show abstract