A Dual Interaction Perspective for Robot Cognition: Grasping as a “Rosetta Stone”

Abstract

Summary. One of the major milestones to higher cognition may be the ability to shape movements that involve very complex interactions with the environment. Based on this hypothesis we argue that the study and technical replication of manual intelligence may serve as a "Rosetta Stone" for designing cognitive robot architectures. The development of such architectures will strongly benefit if improvements to their interaction capabilities in the task domain become paired with efficient modes of interaction in the domain of configuring and restructuring such systems. We find that this "dual interaction perspective" is closely connected with the challenge of integrating holistic and symbolic aspects of representations. In the case of grasping, this requires a very tight marriage between continuous control and more discrete, symbol-like representations of contact and object states. As a concrete implementation, we propose a two layered architecture, where the lower, subsymbolic layer offers a repository of elementary dynamical processes implemented as specialised controllers for sensori-motor primitives. These controllers are represented and coordinated in the upper, symbolic layer, which employs a hierarchy of state machines. Their states represent semantically related classes of dynamic sensori-motor interaction patterns. We report on the application of the proposed architecture to a robot system comprising a 7-DOF redundant arm and a five-fingered, 20-DOF anthropomorphous manipulator. Applying the dual interaction approach, we have endowed the robot with a variety of grasping behaviours, ranging from simple grasping reflexes over visually instructed "imitation grasping" to grasping actions initiated in response to spoken commands. We conclude with a brief sketch of cognitive abilities that we now feel within close reach for the described architecture.

Flexible and stretchable fabric-based tactile sensor

“…We are striving for a better understanding of human grasping and manipulation skills and are convinced that by observing humans we can gain a deeper insight into the control processes involved in manual intelligence [5,21,22]. Having only postural information about the hand does not reveal the associated forces and we therefore sought an explicit sensing channel for contact forces during manipulation.…”

Approaching Manual Intelligence

“…This work could be seen as a (very coarse) sketch of parts of the first two levels in the cognitively motivated architecture of motor action discussed in the previous section. Additional flexibility was gained by adding the hierarchical state machine layer (loosely amounting to erecting the initial elements of a more abstract "mental representation" layer) and a (here not discussed, but see [22]) XML memory layer for structuring the overall system behavior at a very high level (which might be comparable to the "mental control" layer of the cognitive architecture). Extending the posture representation with a manifold-based representation and integrating machine learning methods for extracting configuration manifolds from motion capture data utilizing a dataglove, we were able to realize on a bi-manual system of anthropomorphic Shadow Hands mounted on a pair of PA-10 arms (totalling to 54 DOF; see Fig.…”

The curious robot - Structuring interactive robot learning

“…Pick-and-Place operations are coordinated using hierarchical state machines, which parameterize appropriate low-level robot controllers [17].…”