Spatiotemporal Context Integration in Robot Vision

Abstract

Robust vision in dynamic environments using limited processing power is one of the main challenges in robot vision. This is especially true in the case of biped humanoids that use low-end computers. Techniques such as active vision, context-based vision, and multi-resolution are currently in use to deal with these highly demanding requirements. Thus, having as main motivation the development of robust and high performing robot vision systems, which can operate in dynamic environments, with limited computational resources, we propose a spatiotemporal context integration framework that improves the perceptual capabilities of a given robot vision system. Furthermore, we try to link the vision, tracking, and self-localization problems using a context filter to improve the performance of all these parts together more than to improve them separately. This framework computes: (i) an estimation of the poses of visible and nonvisible objects using Kalman filters; (ii) the spatial coherence of each current detection with all other simultaneous detections and with all tracked objects; and (iii) the spatial coherence of each tracked object with all current detections. Using a Bayesian approach, we calculate the a-posteriori probabilities for each detected and tracked object, which is used in a filtering stage. We choose as a first application of this framework, the detection of static objects in the RoboCup Standard Platform League domain, where Nao humanoid robots are employed. The proposed system is validated in simulations and using real video 357 Int. J. Human. Robot. 2010.07:357-377. Downloaded from www.worldscientific.com by UNIVERSITY OF WATERLOO on 03/13/15. For personal use only. 358 R. Palma-Amestoy et al.sequences. In noisy environments, the system is able to decrease largely the number of false detections and to improve effectively the self-localization of the robot.

Object Detection: Current and Future Directions

“…Integrating contextual information (e.g., about the type of scene, or the presence of other objects) can increase speed and robustness, but "when and how" to do this (before, during or after the detection), it is still an open problem. Some proposed solutions include the use of (i) spatio-temporal context [e.g., Palma-Amestoy et al (2010)], (ii) spatial structure among visual words [e.g., Wu et al (2009)], and (iii) semantic information aiming to map semantically related features to visual words [e.g., Wu et al (2010)], among many others [e.g., Torralba and Sinha (2001), Divvala et al (2009), Sun et al (2012), Mottaghi et al (2014), andCadena et al (2015)]. While most methods consider the detection of objects in a single frame, temporal features can be beneficial [e.g., Viola et al (2005) and Dalal et al (2006)].…”