Dynamic view selection for multi-camera action recognition

Spurlock, Scott; Souvenir, Richard

doi:10.1007/s00138-015-0715-9

Cited by 5 publications

(6 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In the last decade, conventional CV approaches have dominated the MVHAR field; they represented human body configuration using 2D, 3D, and 4D models. Methods using 2D models extracted silhouettes and optical flow from sequences of images for direct classification [9,15] or transformation to higher-level features [1,18,38,39]. High-level features such as silhouettes contour points and centers of mass [15] showed superiority over other methods employing 2D data [18] to encode movement in human action.…”

Section: Related Workmentioning

confidence: 99%

“…The conventional methods require sophisticated features extraction to identify informative features from raw data [14][15][16]. The features extractor usually works independently from the classifier [17,18]. Studies based on this approach focus either on the classifier or on feature engineering [19,20].…”

Section: Introductionmentioning

confidence: 99%

“…Action recognition was performed either by fusing features from multiple views followed by classifier algorithms [8,9] or by estimating actions in an individual view and then combining or voting for prediction scores [22]. Studies involving classifiers for multi-view techniques intended to improve recognition rates based on the efficient utilization of features from multiple cameras [17,18].…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

A deep neural network model for multi-view human activity recognition

2022

View full text Add to dashboard Cite

Multiple cameras are used to resolve occlusion problem that often occur in single-view human activity recognition. Based on the success of learning representation with deep neural networks (DNNs), recent works have proposed DNNs models to estimate human activity from multi-view inputs. However, currently available datasets are inadequate in training DNNs model to obtain high accuracy rate. Against such an issue, this study presents a DNNs model, trained by employing transfer learning and shared-weight techniques, to classify human activity from multiple cameras. The model comprised pre-trained convolutional neural networks (CNNs), attention layers, long short-term memory networks with residual learning (LSTMRes), and Softmax layers. The experimental results suggested that the proposed model could achieve a promising performance on challenging MVHAR datasets: IXMAS (97.27%) and i3DPost (96.87%). A competitive recognition rate was also observed in online classification.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

A deep neural network model for multi-view human activity recognition

2022

View full text Add to dashboard Cite

show abstract

“…Previous research has investigated both recognition from multiple views and discriminative sensor and view selection. Multi-view recognition methods were developed to recognize objects [4], gait [5], and actions [6] from multiple points of view. The previous approaches are generally based on visual features manually engineered for their specific applications.…”

Section: Introductionmentioning

confidence: 99%

Simultaneous View and Feature Selection for Collaborative Multi-Robot Perception

Reily,

Zhang

2020

Preprint

View full text Add to dashboard Cite

Collaborative multi-robot perception provides multiple views of an environment, offering varying perspectives to collaboratively understand the environment even when individual robots have poor points of view or when occlusions are caused by obstacles. These multiple observations must be intelligently fused for accurate recognition, and relevant observations need to be selected in order to allow unnecessary robots to continue on to observe other targets. This research problem has not been well studied in the literature yet. In this paper, we propose a novel approach to collaborative multi-robot perception that simultaneously integrates view selection, feature selection, and object recognition into a unified regularized optimization formulation, which uses sparsity-inducing norms to identify the robots with the most representative views and the modalities with the most discriminative features. As our optimization formulation is hard to solve due to the introduced non-smooth norms, we implement a new iterative optimization algorithm, which is guaranteed to converge to the optimal solution. We evaluate our approach on multi-view benchmark datasets, a case-study in simulation, and on a physical multi-robot system. Experimental results demonstrate that our approach enables accurate object recognition and effective view selection as defined by mutual information.

show abstract

“…Examples of the 2D methods are layer-based circular representation of human model structure [ 48 ], bag-of-visual-words using spatial-temporal interest points for human modeling and classification [ 49 ], view-invariant action masks and movement representation [ 50 ], R-transform features [ 51 ], silhouette feature space with PCA [ 52 ], low-level characteristics of human features [ 53 ], combination of optical-flow histograms and bag-of-interest-point-words using transition HMMs [ 54 ], contour-based and uniform local binary pattern with SVM [ 55 ], multifeatures with key poses learning [ 56 ], dimension-reduced silhouette contours [ 57 ], action map using linear discriminant analysis on multiview action images [ 58 ], posture prototype map using self-organizing map with voting function and Bayesian framework [ 59 ], multiview action learning using convolutional neural networks with long short term memory [ 60 ], and multiview action recognition with an autoencoder neural network for learning view-invariant features [ 61 ].…”

Section: Introductionmentioning

confidence: 99%

Multiview Layer Fusion Model for Action Recognition Using RGBD Images

Chalearnnetkul

Suvonvorn

2018

Computational Intelligence and Neuroscience

View full text Add to dashboard Cite

Vision-based action recognition encounters different challenges in practice, including recognition of the subject from any viewpoint, processing of data in real time, and offering privacy in a real-world setting. Even recognizing profile-based human actions, a subset of vision-based action recognition, is a considerable challenge in computer vision which forms the basis for an understanding of complex actions, activities, and behaviors, especially in healthcare applications and video surveillance systems. Accordingly, we introduce a novel method to construct a layer feature model for a profile-based solution that allows the fusion of features for multiview depth images. This model enables recognition from several viewpoints with low complexity at a real-time running speed of 63 fps for four profile-based actions: standing/walking, sitting, stooping, and lying. The experiment using the Northwestern-UCLA 3D dataset resulted in an average precision of 86.40%. With the i3DPost dataset, the experiment achieved an average precision of 93.00%. With the PSU multiview profile-based action dataset, a new dataset for multiple viewpoints which provides profile-based action RGBD images built by our group, we achieved an average precision of 99.31%.

show abstract

Dynamic view selection for multi-camera action recognition

Cited by 5 publications

References 35 publications

A deep neural network model for multi-view human activity recognition

A deep neural network model for multi-view human activity recognition

Simultaneous View and Feature Selection for Collaborative Multi-Robot Perception

Multiview Layer Fusion Model for Action Recognition Using RGBD Images

Contact Info

Product

Resources

About