Biological organisms learn from interactions with their environment throughout their lifetime. For artificial systems to successfully act and adapt in the real world, it is desirable to similarly be able to learn on a continual basis. This challenge is known as lifelong learning, and remains to a large extent unsolved. In this perspective article, we identify a set of key capabilities that artificial systems will need to achieve lifelong learning. We describe a number of biological mechanisms, both neuronal and non-neuronal, that help explain how organisms solve these challenges, and present examples of biologically inspired models and biologically plausible mechanisms that have been applied to artificial intelligence systems in the quest towards development of lifelong learning machines. We discuss opportunities to further our understanding and advance the state of the art in lifelong learning, aiming to bridge the gap between natural and artificial intelligence.

In uncertain domains, the goals are often unknown and need to be predicted by the organism or system. In this paper, contrastive excitation backprop (c-EB) was used in a goal-driven perception task with pairs of noisy MNIST digits, where the system had to increase attention to one of the two digits corresponding to a goal (i.e., even, odd, low value, or high value) and decrease attention to the distractor digit or noisy background pixels. Because the valid goal was unknown, an online learning model based on the cholinergic and noradrenergic neuromodulatory systems was used to predict a noisy goal (expected uncertainty) and re-adapt when the goal changed (unexpected uncertainty). This neurobiologically plausible model demonstrates how neuromodulatory systems can predict goals in uncertain domains and how attentional mechanisms can enhance the perception of that goal.Artificial top-down attentional systems tend to respond to sensory inputs similarly regardless of context and goals. However, biological systems select relevant information to guide behavior in the face of noisy and unreliable signals, as well as rapidly adapt to unforeseen situations. Goal-driven perception treats the same situation differently based on context and causes attention to be directed to goal-relevant inputs. Oftentimes, these goals are unknown and must be learned through experience. Moreover, these goals or contexts can shift without warning. Goal-driven perception helps prevent over-emphasizing less relevant stimuli and focus instead on critical stimuli that require an immediate response.Neuromodulators are important contributors for attention and goal-driven perception. In particular, the cholinergic system drives bottom-up, stimulus-driven attention, as well as top-down, goal-directed attention [1]. Furthermore, the cholinergic system increases attention to task-relevant stimuli, while decreasing attention to the distractions [2,3]. This is a similar idea to contrastive Excitatory Backpropagation (c-EB) where a top-down excitation mask increments attention to the target features and an inhibitory mask decrements attention to distractors [4]. The noradrenergic system responds to surprises or large violations of priors. When the noradrenergic system responds phasically, where the neural activity rapidly and transiently increases, it causes a network reset (e.g., re-initializing weights or connections) that allows rapid adaptation and relearning under novel conditions [5,6].We modified a c-EB network for use in a goal-driven perception task, where the system had to increase attention to the intended goal object and decrease attention to the distractor. Specifically, we presented to the network pairs of noisy MNIST digits. One goal class was to attend to the digit based on its parity (i.e., even or odd goal), and the other goal class was to attend based on the magnitude of the digit (i.e., low-value or high-value goal). In addition, we added a neuromodulation model to the head of the network architecture that regulated goal selection. Simi...

Nasopharyngeal carcinoma (NPC) is one of the most common head and neck malignancies, and the primary treatment methods are radiotherapy and chemotherapy. Radiotherapy alone, concurrent chemoradiotherapy, and induction chemotherapy combined with concurrent chemoradiotherapy can be used according to different grades. Treatment options and prognoses vary greatly depending on the grade of disease in the patients. Accurate grading and risk assessment are required. Recently, radiomics has combined a large amount of invisible high-dimensional information extracted from computed tomography, magnetic resonance imaging, or positron emission tomography with powerful computing capabilities of machine-learning algorithms, providing the possibility to achieve an accurate diagnosis and individualized treatment for cancer patients. As an effective tumor biomarker of NPC, the radiomic signature has been widely used in grading, differential diagnosis, prediction of prognosis, evaluation of treatment response, and early identification of therapeutic complications. The process of radiomic research includes image segmentation, feature extraction, feature selection, model establishment, and evaluation. Many open-source or commercial tools can be used to achieve these procedures. The development of machine-learning algorithms provides more possibilities for radiomics research. This review aimed to summarize the application of radiomics in NPC and introduce the basic process of radiomics research.

Understanding why deep neural networks and machine learning algorithms act as they do is a difficult endeavor. Neuroscientists are faced with similar problems. One way biologists address this issue is by closely observing behavior while recording neurons or manipulating brain circuits. This has been called neuroethology. In a similar way, neurorobotics can be used to explain how neural network activity leads to behavior. In real world settings, neurorobots have been shown to perform behaviors analogous to animals. Moreover, a neuroroboticist has total control over the network, and by analyzing different neural groups or studying the effect of network perturbations (e.g., simulated lesions), they may be able to explain how the robot's behavior arises from artificial brain activity. In this paper, we review neurorobot experiments by focusing on how the robot's behavior leads to a qualitative and quantitative explanation of neural activity, and vice versa, that is, how neural activity leads to behavior. We suggest that using neurorobots as a form of computational neuroethology can be a powerful methodology for understanding neuroscience, as well as for artificial intelligence and machine learning.

Despite the recent success of deep reinforcement learning (RL), domain adaptation remains an open problem. Although the generalization ability of RL agents is critical for the real-world applicability of Deep RL, zero-shot policy transfer is still a challenging problem since even minor visual changes could make the trained agent completely fail in the new task. To address this issue, we propose a two-stage RL agent that first learns a latent unified state representation (LUSR) which is consistent across multiple domains in the first stage, and then do RL training in one source domain based on LUSR in the second stage. The cross-domain consistency of LUSR allows the policy acquired from the source domain to generalize to other target domains without extra training. We first demonstrate our approach in variants of CarRacing games with customized manipulations, and then verify it in CARLA, an autonomous driving simulator with more complex and realistic visual observations. Our results show that this approach can achieve state-of-the-art domain adaptation performance in related RL tasks and outperforms prior approaches based on latent-representation based RL and image-to-image translation.