Developing Intelligent Robots that Grasp Affordance

Abstract

Humans and robots operating in unstructured environments both need to classify objects through haptic exploration and use them in various tasks, but currently they differ greatly in their strategies for acquiring such capabilities. This review explores nascent technologies that promise more convergence. A novel form of artificial intelligence classifies objects according to sensory percepts during active exploration and decides on efficient sequences of exploratory actions to identify objects. Representing objects according to the collective experience of manipulating them provides a substrate for discovering causality and affordances. Such concepts that generalize beyond explicit training experiences are an important aspect of human intelligence that has eluded robots. For robots to acquire such knowledge, they will need an extended period of active exploration and manipulation similar to that employed by infants. The efficacy, efficiency and safety of such behaviors depends on achieving smooth transitions between movements that change quickly from exploratory to executive to reflexive. Animals achieve such smoothness by using a hierarchical control scheme that is fundamentally different from those of conventional robotics. The lowest level of that hierarchy, the spinal cord, starts to self-organize during spontaneous movements in the fetus. This allows its connectivity to reflect the mechanics of the musculoskeletal plant, a bio-inspired process that could be used to adapt spinal-like middleware for robots. Implementation of these extended and essential stages of fetal and infant development is impractical, however, for mechatronic hardware that does not heal and replace itself like biological tissues. Instead such development can now be accomplished in silico and then cloned into physical robots, a strategy that could transcend human performance.

NSF DARE—Transforming modeling in neurorehabilitation: Four threads for catalyzing progress

“…Nature, in contrast, has evolved hierarchical distributed sensorimotor neural architectures, where computation happens throughout (centrally, in middleware, and 'the edge'). This form of biological edge computing happens at subcortical, spinal, and even anatomical levels [97][98][99]. Therefore, successful smart neuro-assistive or neuro-rehabilitation devices (which are, in fact, a hybrid human+robot system engaged in a game-theoretic dance) would, like robots in general, do well to learn from such forms of biological edge computing for physical action.…”

Remembrance of things perceived: Adding thalamocortical function to artificial neural networks

“…Much of what neuroscientists now hope to learn about brain function comes from building computational models and examining their emergent behaviors. Perhaps we can learn from the disabilities of these deficient “experiments of technology.” Furthermore, normal humans exhibit behaviors such as illusions that an AI engineer might consider to be defects to be overcome by inventing a better machine A scientist seeking to discover the substrate for human intelligence could use such quirks to differentiate among models that otherwise have similar capabilities (Loeb, 2022 ).…”

“…In the 73 years since Donald Hebb’s classic proposal regarding The Organization of Behavior (Hebb, 1949 ), increasingly ambitious implementations of neuromorphic machines such as deep learning ANNs have achieved remarkable success at specific tasks that were once thought to be hallmarks of uniquely human intelligence, e.g., identifying objects in complex scenes, interpreting running speech, playing strategic games. At the same time, we have come to realize that such capabilities do not generalize well to the robust interactions with unstructured physical environments that can be demonstrated by a 5-year-old child (Loeb, 2022 ). The engineers who built the machines that now outperform humans in tasks like playing chess may resent this “moving of the goalposts” for AI, but the need to do so is another aspect of the lack of a theory of computation.…”

“…This is a much more robust form of generalization than has been demonstrated in machine vision. It may be the product of representing classes of objects as clustered patterns of associations between exploratory actions and multimodal percepts (Loeb, 2022 ).…”

“…The curse of dimensionality appears to be overcome by iterative hypothesis testing and decision making (Fishel and Loeb, 2012 ; Loeb, 2022 ), which may be an emergent property of the recurrent thalamocortical circuitry. Bayesian inference uses the data from one of many possible observations to adjust the probabilities of various possible identities of the unknown entity.…”