Evolutionary vs imitation learning for neuromorphic control at the edge*

Abstract

Neuromorphic computing offers the opportunity to implement extremely low power artificial intelligence at the edge. Control applications, such as autonomous vehicles and robotics, are also of great interest for neuromorphic systems at the edge. It is not clear, however, what the best neuromorphic training approaches are for control applications at the edge. In this work, we implement and compare the performance of evolutionary optimization and imitation learning approaches on an autonomous race car control task using an edge neuromorphic implementation. We show that the evolutionary approaches tend to achieve better performing smaller network sizes that are well-suited to edge deployment, but they also take significantly longer to train. We also describe a workflow to allow for future algorithmic comparisons for neuromorphic hardware on control applications at the edge.

Avoiding excess computation in asynchronous evolutionary algorithms

“…Our example of a problem with a positive correlation between evaluation time and fitness is a spiking neural network (SNN) learning problem: we aim to learn an SNN controller to drive a simulated vehicle that corresponds to the F1TENTH autonomous racing platform (O'Kelly et al, 2020). 3 The fitness evaluation for the F1TENTH example computes the mean of how far (in units of distance) the car is able to drive, using the simulated SNN as a controller, before crashing across five training tracks (Schuman et al, 2022; Patton et al, 2021). We use the EA to determine the weights, thresholds, and synaptic delays of an SNN with 20 input neurons (for 10 LIDAR inputs that use a spike‐encoding scheme that requires 2 neurons per input), 10 hidden neurons, and 40 output neurons.…”

“…The fitness evaluation for the F1TENTH example computes the mean of how far (in units of distance) the car is able to drive, using the simulated SNN as a controller, before crashing across five training tracks (Schuman et al, 2022;Patton et al, 2021). We use the EA to determine the weights, thresholds, and synaptic delays of an SNN with 20 input neurons (for 10 LIDAR inputs that use a spike-encoding scheme that requires 2 neurons per input), 10 hidden neurons, and 40 output neurons.…”

Hardware software co-design for leveraging STDP in a memristive neuroprocessor

“…EONS optimizes parameters of the network like number of neurons, number of synapses, connection between neurons and parameters of the neurons and synapses over time. Schuman et al [34] EONS uses the traditional evolutionary approach, it randomly initializes a parent network of a given population size, and then evaluates that network using a fitness score. It uses tournament selection and mutation to produce a child population.…”

Disclosure of a Neuromorphic Starter Kit

“…Then it must be stored as JSON in the format specified by the TENNLab software framework. TENNLab networks have been trained using genetic algorithms [14], [21], [26], deep learning [27], decision trees [23], and they have been hand-designed for various tasks [9], [16], [18]. It is anticipated that in the nearterm, network training will require support from TENNLab software; however, that is not a strict requirement.…”