Using deep multiple instance learning for action recognition in still images

Bas, Cagdas; Zalluhoglu, Cemil; Ikizler-Cinbis, Nazli

doi:10.1109/siu.2017.7960551

Cited by 3 publications

(2 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The motivation of this paper appears to be different from ours and it does not take advantage of the superior capabilities of long-term memory units such as the LSTM units. Only recently, deep multiple-instance learning has been proposed and used in the context of object detection and annotations [7,30,32]. Our work is an extension to the prior literature by combining multiple-instance learning with deep recurrent neural networks which appears to be novel.…”

Section: Literature Reviewmentioning

confidence: 96%

Explaining Aviation Safety Incidents Using Deep Temporal Multiple Instance Learning

Janakiraman¹

2017

Preprint

View full text Add to dashboard Cite

Although aviation accidents are rare, safety incidents occur more frequently and require a careful analysis to detect and mitigate risks in a timely manner. Analyzing safety incidents using operational data and producing event-based explanations is invaluable to airline companies as well as to governing organizations such as the Federal Aviation Administration (FAA) in the United States. However, this task is challenging because of the complexity involved in mining multi-dimensional heterogeneous time series data, the lack of time-step-wise annotation of events in a flight, and the lack of scalable tools to perform analysis over a large number of events. In this work, we propose a precursor mining algorithm that identifies events in the multidimensional time series that are correlated with the safety incident. Precursors are valuable to systems health and safety monitoring and in explaining and forecasting safety incidents. Current methods suffer from poor scalability to high dimensional time series data and are inefficient in capturing temporal behavior. We propose an approach by combining multiple-instance learning (MIL) and deep recurrent neural networks (DRNN) to take advantage of MIL's ability to learn using weakly supervised data and DRNN's ability to model temporal behavior. We describe the algorithm, the data, the intuition behind taking a MIL approach, and a comparative analysis of the proposed algorithm with baseline models. We also discuss the application to a real-world aviation safety problem using data from a commercial airline company and discuss the model's abilities and shortcomings, with some final remarks about possible deployment directions.

show abstract

Section: Literature Reviewmentioning

confidence: 96%

Explaining Aviation Safety Incidents Using Deep Temporal Multiple Instance Learning

Janakiraman¹

2017

Preprint

View full text Add to dashboard Cite

show abstract

“…Existing methods typically utilize deep convolutional neural networks (CNNs) to classify actions by directly distilling spatial features from images. For example, Bas et al (2017) [1] first extracted the objects from the images using AlexNet [2] and then performed action recognition of the target regions through a multiinstance learning framework. [3] constructed a multibranch attention network based on VGG16 [4] , incorporating multiple regional attention mechanisms for capturing information from regions of different sizes.…”

Section: Introductionmentioning

confidence: 99%

Multi-person Action Recognition Based on the HARNet Network in the Park Environment

Xie

Liu

Chen

et al. 2023

J. Phys.: Conf. Ser.

View full text Add to dashboard Cite

Action recognition is crucial to public security, playing an important role in pedestrian protection. Recently, numerous approaches based on deep learning models have been reported for action recognition, which mainly has problems such as poor versatility, not good robustness, low recognition effects for low resolution, and occlusion in outdoor environments. This paper addresses this issue and aims to develop an approach for multi-person action recognition in the park environment. Frame extraction operation is first used to extract images from surveillance video in real-time, then YOLOv5 is adopted to detect and segment human targets from images. Thirdly, the AlphaPose algorithm is used to estimate the keypoint heatmap of the human body according to human targets. Finally, the heatmaps are preprocessed and then fused and classified by deep learning networks to complete the multi-person action recognition. In addition, an optimization method, i.e., Heatmap-based Action Recognition Network (HARNet), for ResNet is proposed in order to overcome the difficulty of information extraction from heatmaps, where the attention mechanism is considered. The approach presented here was tested on park surveillance video with multiple different scenarios, and the corresponding results verified its performance.

show abstract

An Improved Bilinear Pooling Method for Image-Based Action Recognition

2021

2020 25th International Conference on Pattern Recognition (ICPR)

View full text Add to dashboard Cite

Using deep multiple instance learning for action recognition in still images

Cited by 3 publications

References 14 publications

Explaining Aviation Safety Incidents Using Deep Temporal Multiple Instance Learning

Explaining Aviation Safety Incidents Using Deep Temporal Multiple Instance Learning

Multi-person Action Recognition Based on the HARNet Network in the Park Environment

An Improved Bilinear Pooling Method for Image-Based Action Recognition

Contact Info

Product

Resources

About