This article provides a review of current development and challenges in brain-inspired computing with memristors. We review the mechanisms of various memristive devices that can mimic synaptic and neuronal functionalities and survey the progress of memristive spiking and artificial neural networks. Different architectures are compared, including spiking neural networks, fully connected artificial neural networks, convolutional neural networks, and Hopfield recurrent neural networks. Challenges and strategies for nanoelectronic brain-inspired computing systems, including device variations, training, and testing algorithms, are also discussed.