Reward Maximization Justifies the Transition from Sensory Selection at Childhood to Sensory Integration at Adulthood

Abstract

In a multisensory task, human adults integrate information from different sensory modalities -behaviorally in an optimal Bayesian fashion- while children mostly rely on a single sensor modality for decision making. The reason behind this change of behavior over age and the process behind learning the required statistics for optimal integration are still unclear and have not been justified by the conventional Bayesian modeling. We propose an interactive multisensory learning framework without making any prior assumptions about the sensory models. In this framework, learning in every modality and in their joint space is done in parallel using a single-step reinforcement learning method. A simple statistical test on confidence intervals on the mean of reward distributions is used to select the most informative source of information among the individual modalities and the joint space. Analyses of the method and the simulation results on a multimodal localization task show that the learning system autonomously starts with sensory selection and gradually switches to sensory integration. This is because, relying more on modalities -i.e. selection- at early learning steps (childhood) is more rewarding than favoring decisions learned in the joint space since, smaller state-space in modalities results in faster learning in every individual modality. In contrast, after gaining sufficient experiences (adulthood), the quality of learning in the joint space matures while learning in modalities suffers from insufficient accuracy due to perceptual aliasing. It results in tighter confidence interval for the joint space and consequently causes a smooth shift from selection to integration. It suggests that sensory selection and integration are emergent behavior and both are outputs of a single reward maximization process; i.e. the transition is not a preprogrammed phenomenon.

Bayes-Like Integration of a New Sensory Skill with Vision

“…Bayes-like manner (Bejjanki, Knill, & Aslin, 2016;Körding & Wolpert, 2004). Thus, there might perhaps be more flexibility than suggested by the models reviewed above (Daee et al, 2014;Weisswange et al, 2011). In other words, an alternative hypothesis is that humans can actually learn to combine novel sensory cues, in line with the other kinds of adaptation that have already been documented (Knill & Pouget, 2004;Pouget et al, 2013).…”

“…that learning to combine an augmented sense would be a truly extraordinary effortessentially impossible given the limited resources of a routine laboratory experiment or a routine therapy method. Specifically, they suggest that it would take a full decade of daily cue-specific experience for Bayes-like combination to emerge (Daee, Mirian, Ahmadabadi, Brenner, & Tenenbaum, 2014;Weisswange, Rothkopf, Rodemann, & Triesch, 2011). This view may explain why children under 10-12 years don't yet combine sensory cues in this way (Gori, Del Viva, Sandini, & Burr, 2008;Gori, Sandini, & Burr, 2012;Nardini, Jones, Bedford, & Braddick, 2008;Petrini, Remark, Smith, & Nardini, 2014), even within the same modality (Dekker et al, 2015;Nardini, Bedford, & Mareschal, 2010), despite having a number of other skills in the domain of cross-modal sensory perception (Bahrick & Lickliter, 2000;Gottfried, Rose, & Bridger, 1977;Lewkowicz, 2000;Lewkowicz & Turkewitz, 1980;Spelke, 1979).…”

“…If naïve adults show both markers, we will have evidence for the acquisition of Bayes-like cue combination with an augmented sense. This would lead to revising current models of multisensory learning (Daee et al, 2014;Weisswange et al, 2011) and would have important potential applications to augmenting the human senses.…”

Sensory Cue Combination in Children Under 10 Years of Age

“…has the same reliability as all further possible audio stimuli in the same place (Daee et al, 2014;Weisswange et al, 2011). The estimated reliability parameter for each modality can adapt over long times (thousands of trials), but there are not, for example, separate reliability estimates for a dark room versus a brightly lit room.…”

Dad2K: An Adaptation of SafeCare to Enhance Positive Parenting Skills With At-Risk Fathers

“…The computer technology allows the father to connect with a software program specifically designed for Dad2K, which presents SafeCare PCI information using a sports-theme approach (see Appendix A Screen Shots). The benefit of the technology-based resource is that it allows for standardization and adherence to the evidence-based SafeCare protocol and offers an engaging, multimodal learning experience for fathers to get initial access to the SafeCare information (Daee, Mirian, & Ahmadabadi, 2014;Mitchel, Christiansen, & Weiss, 2014). The Dad2K software provides motivational content (engaging dialogues and directed assessment questions with tailored feedback based on Motivational Interviewing Principles; Miller & Rollnick, 1991), as well as psychoeducation and video modeling of the PAT skills at each session.…”