Integrating perception and problem solving to predict complex object behaviours

Abstract

One of the objectives of Cognitive Robotics is to construct robot systems that can be directed to achieve realworld goals by high-level directions rather than complex, low-level robot programming. Such a system must have the ability to represent, problem-solve and learn about its environment as well as communicate with other agents. In previous work, we have proposed ADAPT, a Cognitive Architecture that views perception as top-down and goaloriented and part of the problem solving process. Our approach is linked to a SOAR-based problem-solving and learning framework. In this paper, we present an architecture for the perceptive and world modelling components of ADAPT and report on experimental results using this architecture to predict complex object behaviour.A novel aspect of our approach is a 'mirror system' that ensures that the modelled background and foreground objects are synchronized with observations and task-based expectations. This is based on our prior work on comparing real and synthetic images. We show results for a moving object that collides and rebounds from its environment, hence showing that this perception-based problem solving approach has the potential to be used to predict complex object motions.

A relaxed fusion of information from real and synthetic images to predict complex behavior

“…Figure 4(a) is a camera view with a single foreground object. Figure 4(b) is a synthetic, texture mapped view which works well with the match-mediated difference based system in Lyons et al [13]. However, both We develop a modified fusion operation to handle foreground color as follows.…”

Navigation of uncertain terrain by fusion of information from real and synthetic imagery

“…First, we will assume that the only unmodeled objects are those whose shape and behaviour can be classified in one image; for example, an unstable or collapsing column of masonry. A rolling, bouncing object would not meet this assumption (though we showed in [12] for an otherwise static scene our approach will handle this). Since this assumption is difficult to implement in a physical evaluation framework, we introduce a second simplification in which the robot traverses a (second) simulated scene rather than a real scene.…”

“…The navigation architecture introduced here is heavily based on the ball tracking architecture we introduced in [12]. However, the minimal subscene module in this case includes (see Figure 2):…”

“…ADAPT merges RS [11], a language for specifying and reasoning about sensory-based robot plans with SOAR [9] a widely used cognitive architecture. The visual component of the perceptual apparatus for ADAPT, introduced in [10] and developed in [12] [13], leverages the state of the art in physics-engines for video gaming to provide a fast and effective 3D simulation system. A physics-engine is a module used in video game software to ensure that the virtual game world obeys the laws of physics and looks realistic to the game player.…”

“…In [10] [12] we introduced an approach, called the match-mediated difference (MMD), to combining the information from real and synthetic images for static scenes containing real and/or simulated stationary and moving objects. We leveraged Itti and Arbib's [14] concept of the minimal subscene (developed for discourse analysis) to capture how the robot modelled the scene being viewed, how it deployed the simulation and the MMD operation to determine unexpected scene elements and how it requested and assimilated simulation predictions.…”