Optimization of Action Recognition Model Based on Multi-Task Learning and Boundary Gradient

Abstract

Recently, people’s demand for action recognition has extended from the initial high classification accuracy to the high accuracy of the temporal action detection. It is challenging to meet the two requirements simultaneously. The key to behavior recognition lies in the quantity and quality of the extracted features. In this paper, a two-stream convolutional network is used. A three-dimensional convolutional neural network (3D-CNN) is used to extract spatiotemporal features from the consecutive frames. A two-dimensional convolutional neural network (2D-CNN) is used to extract spatial features from the key-frames. The integration of the two networks is excellent for improving the model’s accuracy and can complete the task of distinguishing the start–stop frame. In this paper, a multi-scale feature extraction method is presented to extract more abundant feature information. At the same time, a multi-task learning model is introduced. It can further improve the accuracy of classification via sharing the data between multiple tasks. The experimental result shows that the accuracy of the modified model is improved by 10%. Meanwhile, we propose the confidence gradient, which can optimize the distinguishing method of the start–stop frame to improve the temporal action detection accuracy. The experimental result shows that the accuracy has been enhanced by 11%.

Human Action Recognition Using Dense Trajectories

“…They applied a unique-match restriction among the descriptors and eliminated matches that were too far apart from one another to assure correctness. For picture classification, dense sampling has proven to perform better than sparse interest spots (4) . Similarly, dense sampling at predictable geographical and temporal places fared better than cutting-edge space-time interest point detectors in recent assessments of action identification.…”

Temporal Context Modeling Network with Local-Global Complementary Architecture for Temporal Proposal Generation

“…[16] uses human boxes and key points to represent instance-level features, and the action region features of this framework are used as the input of the temporal action head network, which makes the framework more discriminative. The author of [17] proposed a multi-scale feature extraction method used to extract richer feature information. At the same time, a multi-task learning model is introduced.…”

Multi-Task Learning with Task-Specific Feature Filtering in Low-Data Condition

“…Convolutional neural networks (CNNs) are nowadays the state-of-the-art methods for a wide range of computer vision tasks, thanks to the large-scale public datasets [1][2][3] and high performance accelerators like graphical processing units (GPUs). For example, CNNs rank the highest in benchmarks in image classification [4][5][6], object detection [7][8][9][10], semantic segmentation [11], and action recognitions [12][13][14]. The general recipe for a successful CNN model includes training a large-sized model with a large-scale dataset.…”

“…Collecting a large-scale dataset is also very expensive. To solve these issues, previous works have proposed multi-task learning (MTL) [12,[15][16][17]. By definition, multi-task learning trains a model with multiple functionalities.…”