Ego-Motion Classification for Body-Worn Videos

Zhang, Meng; Sánchez, Javier; Morel, Jean‐Michel; Bertozzi, Andrea L.; Brantingham, P. Jeffrey

doi:10.1007/978-3-319-91274-5_10

Cited by 6 publications

(43 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Semi-supervised learning has been studied extensively in the past two decades and has been successfully applied to applications such as hyperspectral images [12] and body-worn videos [11,5]. We refer readers to [22] and the more recent article [1] for a literature review.…”

Section: Related Workmentioning

confidence: 99%

“…Such problems are ripe for the development of semi-supervised learning algorithms, which, by definition, use a small amount of training data. In the last year, progress has been made in applying graph-based semi-supervised learning to bodyworn videos with the goal of recognizing camera-wearers' activities, i.e., ego-motion [11,5]. However, as is often the case with real-world videos, the variability of the data leads to imperfect classification.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Uncertainty quantification for semi-supervised multi-class classification in image processing and ego-motion analysis of body-worn videos

Qiao¹,

Shi²,

Wang³

et al. 2019

View full text Add to dashboard Cite

Semi-supervised learning uses underlying relationships in data with a scarcity of ground-truth labels. In this paper, we introduce an uncertainty quantification (UQ) method for graph-based semi-supervised multi-class classification problems. We not only predict the class label for each data point, but also provide a confidence score for the prediction. We adopt a Bayesian approach and propose a graphical multi-class probit model together with an effective Gibbs sampling procedure. Furthermore, we propose a confidence measure for each data point that correlates with the classification performance. We use the empirical properties of the proposed confidence measure to guide the design of a humanin-the-loop system. The uncertainty quantification algorithm and the human-in-the-loop system are successfully applied to classification problems in image processing and ego-motion analysis of body-worn videos.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Uncertainty quantification for semi-supervised multi-class classification in image processing and ego-motion analysis of body-worn videos

Qiao¹,

Shi²,

Wang³

et al. 2019

View full text Add to dashboard Cite

show abstract

“…the motion of the individual to whom the camera is mounted. In Meng, J. Sanchez, Morel, Bertozzi, and Brantingham [2017], the authors develop an algorithm for the ego-motion classification, combining the MBO scheme for multi-class semi-supervised learning with an inverse compositional algorithm Sánchez [2016] to estimate transformations between successive frames. Thus the video is preprocessed to obtain an eight dimensional feature vector for each frame corresponding to the Left-Right; Up-Down; Rotational; and Forward-Backward motions of the camera wearer along with the frequencies of each of these motions.…”

Section: Volume Penaltiesmentioning

confidence: 99%

Graphical Models in Machine Learning, Networks and Uncertainty Quantification

Bertozzi

2019

Proceedings of the International Congress of Mathematicians (ICM 2018)

Self Cite

View full text Add to dashboard Cite

This paper is a review article on semi-supervised and unsupervised graph models for classification using similarity graphs and for community detection in networks. The paper reviews graph-based variational models built on graph cut metrics. The equivalence between the graph mincut problem and total variation minimization on the graph for an assignment function allows one to cast graph-cut variational problems in the language of total variation minimization, thus creating a parallel between low dimensional data science problems in Euclidean space (e.g. image segmentation) and high dimensional clustering. The connection paves the way for new algorithms for data science that have a similar structure to well-known computational methods for nonlinear partial differential equations. This paper focuses on a class of methods build around diffuse interface models (e.g. the Ginzburg-Landau functional and the Allen-Cahn equation) and threshold dynamics, developed by the Author and collaborators. Semi-supervised learning with a small amount of training data can be carried out in this framework with diverse applications ranging from hyperspectral pixel classification to identifying activity in police body worn video. It can also be extended to the context of uncertainty quantification with Gaussian noise models. The problem of community detection in networks also has a graph-cut structure and algorithms are presented for the use of threshold dynamics for modularity optimization. With efficient methods, this allows for the use of network modularity for unsupervised machine learning problems with unknown number of classes.

show abstract

“…In this chapter, we develop the graph classification method for a real-world application from body worn cameras [108]. Portable cameras record dynamic first-person video footage and these videos contain information on the motion of the individual on whom the camera is mounted, defined as ego.…”

Section: Chaptermentioning

confidence: 99%

“…In Chapter 6, we discuss a direct application of the parallelized classification algorithm -the egomotion classification of body-worn videos [108].…”

Section: Introductionmentioning

confidence: 99%

High Performance Computing and Real Time Software for High Dimensional Data Classification

Zhang¹

2018

Self Cite

View full text Add to dashboard Cite

The classification result of the semi-supervised and unsupervised graph MBO methods on the Urban data for four classes. (a) The ground truth with four classes: asphalt (blue), grass (red), trees (green), roof (yellow). (b) Pixels selected to have known labels (10% of the data) for the semi-supervised algorithm. (c) The classification result of the semi-supervised algorithm with the accuracy of 93.48%. (d) The classification result of the unsupervised algorithm with the accuracy of 92.35%. (e) The result of spectral clustering with K-means with the accuracy of 75.06%.

show abstract

Ego-Motion Classification for Body-Worn Videos

Cited by 6 publications

References 32 publications

Uncertainty quantification for semi-supervised multi-class classification in image processing and ego-motion analysis of body-worn videos

Uncertainty quantification for semi-supervised multi-class classification in image processing and ego-motion analysis of body-worn videos

Graphical Models in Machine Learning, Networks and Uncertainty Quantification

High Performance Computing and Real Time Software for High Dimensional Data Classification

Contact Info

Product

Resources

About