Efficient online bootstrapping of sensory representations

Abstract

This is a simulation-based contribution exploring a novel approach to the open-ended formation of multimodal representations in autonomous agents. In particular, we address the issue of transferring ("bootstrapping") feature selectivities between two modalities, from a previously learned or innate reference representation to a new induced representation. We demonstrate the potential of this algorithm by several experiments with synthetic inputs modeled after a robotics scenario where multimodal object representations are "bootstrapped" from a (reference) representation of object affordances. We focus on typical challenges in autonomous agents: absence of human supervision, changing environment statistics and limited computing power. We propose an autonomous and local neural learning algorithm termed PRO-PRE (projection-prediction) that updates induced representations based on predictability: competitive advantages are given to those feature-sensitive elements that are inferable from activities in the reference representation. PRO-PRE implements a bi-directional interaction of clustering ("projection") and inference ("prediction"), the key ingredient being an efficient online measure of predictability controlling learning in the projection step. We show that the proposed method is computationally efficient and stable, and that the multimodal transfer of feature selectivity is successful and robust under resource constraints. Furthermore, we successfully demonstrate robustness to noisy reference representations, non-stationary input statistics and uninformative inputs. to be an open-ended and largely unsupervised process, leading to internal representations of ever increasing specialization and usefulness to the agent. The labeled boxes indicate the work covered in this article: I 1 , I 2 , I 3 stand for input representations derived from sensors, N indicates the newly formed induced representation, and R stands for the reference representation that is derived from another sensory modality, controlling the bootstrapping process.

PROPRE: PROjection and PREdiction for multimodal correlations learning. An application to pedestrians visual data discrimination

“…In previous articles [18], [19], PROPRE was already applied on artificial data representative of a robotic task. In this article, targeting the use of PROPRE on real robotic platform, we illustrate multiple of its functional properties when applied to real visual pedestrian data on a challenging supervised classification task.…”

“…We previously studied the PROPRE paradigm with artificial multimodal data related to basic robotic behaviors [18], [19]. Targeting its use on a real developmental robotic platform, in this article we apply it to a challenging visual discrimination task of namely real-world pedestrian pose classification [20], [21] (see figure 1).…”

Bioethanol Combustion in an Industrial Gas Turbine Combustor: Simulations and Experiments

“…The first of the aforementioned explanations is however unlikely, because the correlation of El-Shanawany yields relatively small droplet sizes compared to those obtained using other empirical correlations for prefilming airblast atomizers [25,26]. The droplet life time is highly sensitive to the initial droplet size, so the use of a different SMD correlation will generally increase the impingement of droplets significantly.…”

“…Regarding the second explanation, Gepperth et al [26] have shown that the droplet sizes predicted by Eq. (3) can only be representative after secondary breakup.…”

Simultaneous concept formation driven by predictability

“…Such concepts must be statistically related across sensory modalities while being non-redundant within their own modality. For achieving this in an online learning process, we propose a variant of the PROPRE (projection-prediction) algorithm [11]: PROPRE is a neural learning method which uses projection to map input stimuli to a two-dimensional neural representation ("induced representation") while using prediction from another neural representation ("reference representation") to modulate the adaptation of the projection step 1 . This results in selectivity to distinct patterns whose presence can be inferred from activity in the reference representation.…”