Modeling High-Dimensional Humans for Activity Anticipation using Gaussian Process Latent CRFs

Jiang, Yun; Saxena, Ashutosh

doi:10.15607/rss.2014.x.015

Cited by 30 publications

(30 citation statements)

References 40 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Among the aforementioned approaches, Div-M There are variants of CRFs that rely on sequential models as well such as, Dynamic CRF (dCRF) [40], Infinite Hidden CRF [2], Gaussian Process Latent CRF [17] and Hierarchical Semi-Markov CRF (HSCRF). Although they are applicable to videos, we are not aware of any tractable method to compute a belief over any of the aforementioned graphical model.…”

Section: Background and Related Workmentioning

confidence: 99%

“…It is also common to compute a belief over latent nodes as in the case of infinte hidden CRF [2] and Gaussian Process Latent CRF [17]. However, they are not directly applicable to our problem since they can compute a belief only over the latent node.…”

Section: Background and Related Workmentioning

confidence: 99%

“…It anticipates the future via augmenting set of possible future observations to the CRF. It is also extended with an improved human motion model based on a Gaussian process [17]. However, their accuracy significantly drops for a long anticipation horizon since they fail to represent the uncertainty.…”

Section: Background and Related Workmentioning

confidence: 99%

“…Anticipation is a challenging task, and it requires us to model the relationships between several objects and the human(s) in the scene, as well as their temporal evolution. Although the modelling assumptions and model parametrization varies, the common approach [22,17,23,26] is using Conditional Random Field (CRF) to represent the rich relations in the scene, and anticipating a single or a few most likely future states. Since the future is ambiguous, the most likely state might not be sufficient enough to assess the risk of each action.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

rCRF: Recursive Belief Estimation over CRFs in RGB-D Activity Videos

Şener

Saxena

2015

Robotics: Science and Systems XI

Self Cite

View full text Add to dashboard Cite

Abstract-For assistive robots, anticipating the future actions of humans is an essential task. This requires modelling both the evolution of the activities over time and the rich relationships between humans and the objects. Since the future activities of humans are quite ambiguous, robots need to assess all the future possibilities in order to choose an appropriate action. Therefore, a successful anticipation algorithm needs to compute all plausible future activities and their corresponding probabilities.In this paper, we address the problem of efficiently computing beliefs over future human activities from RGB-D videos. We present a new recursive algorithm that we call Recursive Conditional Random Field (rCRF) which can compute an accurate belief over a temporal CRF model. We use the rich modelling power of CRFs and describe a computationally tractable inference algorithm based on Bayesian filtering and structured diversity. In our experiments, we show that incorporating belief, computed via our approach, significantly outperforms the stateof-the-art methods, in terms of accuracy and computation time.

show abstract

Section: Background and Related Workmentioning

confidence: 99%

Section: Background and Related Workmentioning

confidence: 99%

Section: Background and Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

rCRF: Recursive Belief Estimation over CRFs in RGB-D Activity Videos

Şener

Saxena

2015

Robotics: Science and Systems XI

Self Cite

View full text Add to dashboard Cite

show abstract

“…For example, reasoning about the interactions with objects helps in activity detection [28,43,18], understanding the spatial and structural relationships between objects improves object detection [16] and retrieval [8], and understanding what actions are supported by the objects in an environment is essential for many robotic applications [17,30]. Our goal is to learn a rich functional representation of the environment in terms of object affordances from RGB-D videos of people interacting with their surrounding environment.…”

Section: Introductionmentioning

confidence: 99%

Physically Grounded Spatio-temporal Object Affordances

Koppula

Saxena

2014

Lecture Notes in Computer Science

Self Cite

View full text Add to dashboard Cite

Abstract. Objects in human environments support various functionalities which govern how people interact with their environments in order to perform tasks. In this work, we discuss how to represent and learn a functional understanding of an environment in terms of object affordances. Such an understanding is useful for many applications such as activity detection and assistive robotics. Starting with a semantic notion of affordances, we present a generative model that takes a given environment and human intention into account, and grounds the affordances in the form of spatial locations on the object and temporal trajectories in the 3D environment. The probabilistic model also allows uncertainties and variations in the grounded affordances. We apply our approach on RGB-D videos from Cornell Activity Dataset, where we first show that we can successfully ground the affordances, and we then show that learning such affordances improves performance in the labeling tasks.

show abstract

Holistic 3D Scene Parsing and Reconstruction from a Single RGB Image

Huang

Zhu

et al. 2018

Lecture Notes in Computer Science

129

107

View full text Add to dashboard Cite

We propose a computational framework to jointly parse a single RGB image and reconstruct a holistic 3D configuration composed by a set of CAD models using a stochastic grammar model. Specifically, we introduce a Holistic Scene Grammar (HSG) to represent the 3D scene structure, which characterizes a joint distribution over the functional and geometric space of indoor scenes. The proposed HSG captures three essential and often latent dimensions of the indoor scenes: i) latent human context, describing the affordance and the functionality of a room arrangement, ii) geometric constraints over the scene configurations, and iii) physical constraints that guarantee physically plausible parsing and reconstruction. We solve this joint parsing and reconstruction problem in an analysis-by-synthesis fashion, seeking to minimize the differences between the input image and the rendered images generated by our 3D representation, over the space of depth, surface normal, and object segmentation map. The optimal configuration, represented by a parse graph, is inferred using Markov chain Monte Carlo (MCMC), which efficiently traverses through the non-differentiable solution space, jointly optimizing object localization, 3D layout, and hidden human context. Experimental results demonstrate that the proposed algorithm improves the generalization ability and significantly outperforms prior methods on 3D layout estimation, 3D object detection, and holistic scene understanding.

show abstract

Modeling High-Dimensional Humans for Activity Anticipation using Gaussian Process Latent CRFs

Cited by 30 publications

References 40 publications

rCRF: Recursive Belief Estimation over CRFs in RGB-D Activity Videos

rCRF: Recursive Belief Estimation over CRFs in RGB-D Activity Videos

Physically Grounded Spatio-temporal Object Affordances

Holistic 3D Scene Parsing and Reconstruction from a Single RGB Image

Contact Info

Product

Resources

About