Magnetic domain walls are information tokens in both logic and memory devices, and hold particular interest in applications such as neuromorphic accelerators that combine logic in memory. Here, we show that devices based on the electrical manipulation of magnetic domain walls are capable of implementing linear, as well as programmable nonlinear, functions. Unlike other approaches, domain-wall-based devices are ideal for application to both synaptic weight generators and thresholding in deep neural networks. Prototype micrometer-size devices operate with 8 ns current pulses and the energy consumption required for weight modulation is ≤ 16 pJ. Both speed and energy consumption compare favorably to other synaptic nonvolatile devices, with the expected energy dissipation for scaled 20-nm-devices close to that of biological neurons. Deep neural networks 1, 2 mimic the synaptic and activation functionality of human neurons using repeated applications of linear filters interspersed by nonlinear decision functions. Among other applications, deep neural networks offer promising solutions to image 3 , speech 4 and video 5recognition. Training is performed using a backpropagation learning procedure 6 , with the filter weights updated continuously like synapses, until the calculated output matches the desired output.