The exceptional capabilities of the human brain provide inspiration for artificially intelligent hardware that mimics both the function and the structure of neurobiology. In particular, the recent development of nanodevices with biomimetic characteristics promises to enable the development of neuromorphic architectures with exceptional computational efficiency. In this work, we propose biomimetic neurons comprised of domain wall-magnetic tunnel junctions that can be integrated into the first trainable CMOS-free recurrent neural network with biomimetic components. This paper demonstrates the computational effectiveness of this system for benchmark tasks and its superior computational efficiency relative to alternative approaches for recurrent neural networks.