Human action recognition using transfer learning with deep representations

Sargano, Allah Bux; Wang, Xiaofeng; Angelov, Plamen; Habib, Zulfiqar

doi:10.1109/ijcnn.2017.7965890

Cited by 130 publications

(64 citation statements)

References 51 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…In [24], a pretrained deep CNN is used to extract features, followed by the combination of SVM and KNN classifiers for action recognition. A pre-trained CNN on a large-scale annotation dataset can be transmitted for the action recognition with a small training dataset.…”

Section: Related Workmentioning

confidence: 99%

“…The videos are captured in different lighting conditions, gestures, and viewpoint. One of the major challenges in this dataset is the mixture of natural realistic actions and the actions played by many actors while in other datasets the activities and actions are usually performed by one actor only [12], [24], [50], [54]. The UCF101 dataset includes 101 action classes, over 13,000 video clips and 27 hours of video data.…”

Section: Ucf101 Datasetmentioning

confidence: 99%

See 1 more Smart Citation

Complex Human Action Recognition in Live Videos Using Hybrid FR-DL Method

Serpush¹,

Rezaei²

2020

Preprint

View full text Add to dashboard Cite

Automated human action recognition is one of the most attractive and practical research fields in computer vision, in spite of its high computational costs. In such systems, the human action labelling is based on the appearance and patterns of the motions in the video sequences; however, the conventional methodologies and classic neural networks cannot use temporal information for action recognition prediction in the upcoming frames in a video sequence. On the other hand, the computational cost of the preprocessing stage is high. In this paper, we address challenges of the preprocessing phase, by an automated selection of representative frames among the input sequences. Furthermore, we extract the key features of the representative frame rather than the entire features. We propose a hybrid technique using background subtraction and HOG, followed by application of a deep neural network and skeletal modelling method. The combination of a CNN and the LSTM recursive network is considered for feature selection and maintaining the previous information, and finally, a Softmax-KNN classifier is used for labelling human activities. We name our model as "Feature Reduction \& Deep Learning" based action recognition method, or FR-DL in short. To evaluate the proposed method, we use the UCF dataset for the benchmarking which is widely-used among researchers in action recognition research. The dataset includes 101 complicated activities in the wild. Experimental results show a significant improvement in terms of accuracy and speed in comparison with six state-of-the-art articles.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Ucf101 Datasetmentioning

confidence: 99%

Complex Human Action Recognition in Live Videos Using Hybrid FR-DL Method

Serpush¹,

Rezaei²

2020

Preprint

View full text Add to dashboard Cite

show abstract

“…The image classification and video recognition have further application types, image segmentations and clustering. The second application is human activity classification; Sargano et al recognized the human activity on the base of pre-train deep CNN model, feature extraction and representation followed by a hybrid support vector machine (SVM) and K-nearest neighbor (KNN) classifier for activity recognition [19].…”

Section: Heterogeneous Transfer Learning Applicationsmentioning

confidence: 99%

Heterogeneous transfer learning techniques for machine learning

et al. 2018

View full text Add to dashboard Cite

The main assumption in machine learning and data mining is, training the data, and the future data have the same distribution and same features. However, in many applications, in the real world, such assumptions may not be retained. For example, sometimes, we have the task of classification in the one domain of interest, but when the same data is used in another domain, it needed enough training to work in the other domain of interest. In the field of heterogeneous transfer learning, train the data in one domain and test with other domain. In this case, knowledge is transfer; if there is a successful transfer, it can significantly improve performance by avoiding the learning in the label information which is more expensive. Over the past few years, the transfer learning has become a new learning framework to address this issue and heterogeneous transfer learning is the most active research area in the recent years. In this study, we are discussing the relationship between heterogeneous transfer learning and the other machine learning methods, including the field of adaptation, learning and multitasking learning and sample selection bias, as well as the associates of variables. We also reconnoiter some main challenges for the future issue in heterogeneous transfer learning.

show abstract

“…Hence, training of deep learning model from scratch is not an appropriate approach for the domain-specific problems [15], where the size of the dataset is small. On the other hand, some recent studies in image recognition and classification tasks used the concept of transfer learning (domain adaptation), fine-tune the deeply learned models on a specific task to a new task even in a changed domain [15]- [18]. The transfer learning is also favorable for limited size dataset training and can also be used for real-time applications.…”

Section: Introductionmentioning

confidence: 99%

“…There are also other publicly available models i.e., AlexNet [20] and GoogLeNet [21], but due to their high error rate ResNet-50 [19] model is used. The idea has been taken from the approach adopted in human action recognition by using transfer learning with deep representations [15]. Where the author used AlexNet [20] as a feature extractor, afterward, an ensemble classifier is used to recognize the human actions.…”

Section: Introductionmentioning

confidence: 99%

Automatic Depth Estimation from Single 2D Image via Transfer Learning Approach

Shoukat

Sargano

Habib

et al. 2018

2018 2nd European Conference on Electrical Engineering and Computer Science (EECS)

Self Cite

View full text Add to dashboard Cite

Nowadays, depth estimation from a single 2D image is a prominent task due to its numerous applications such as 2D to 3D image/video conversion, robot vision, and self-driving cars. This research proposes an automatic novel technique for the depth estimation of single 2D images via transfer learning of pre-trained deep learning model. This is a challenging problem, as a single 2D image does not carry any cues regarding depth. To tackle this, the pool of available images is exploited for which the depth is known. By following the hypothesis that the color images having similar semantics are most probably to have similar depth. Along these lines, the depth of the input image is predicted through corresponding depth maps of semantically similar images available in the dataset, fetched by high-level features of pre-trained deep learning model followed by a classifier (i.e., K-Nearest Neighbor). Afterward, a Cross Bilateral filter is applied for the removal of fallacious depth variations in the depth map. To prove the quality of the presented approach, different experiments have been conducted on two publicly available benchmark datasets, NYU (v2) and Make3D. The results indicate that the proposed approach outperforms state of the art methods.

show abstract

Human action recognition using transfer learning with deep representations

Cited by 130 publications

References 51 publications

Complex Human Action Recognition in Live Videos Using Hybrid FR-DL Method

Complex Human Action Recognition in Live Videos Using Hybrid FR-DL Method

Heterogeneous transfer learning techniques for machine learning

Automatic Depth Estimation from Single 2D Image via Transfer Learning Approach

Contact Info

Product

Resources

About