Artificial spin ices are arrays of coupled single domain nanomagnets that have mainly been explored in two dimensions. They display a number of intriguing phenomena arising from the collective behavior of the magnets including vertex frustration, emergent magnetic monopoles, and phase transitions. Escaping this flat paradigm into the third dimension is now possible, thanks to advances in fabrication and characterization of three-dimensional mesoscopic magnetic systems, which have magnetic elements with dimensions between a few 10's and a few 100's nanometers. By exploiting the extra degrees of freedom inherent to fully three-dimensional structures, it will be possible to harness the dipolar and other interactions between magnetic elements in a way that cannot be achieved in planar systems. This will offer an unparalleled opportunity to produce three-dimensional mesoscopic magnetic structures exhibiting true spin ice physics and also, more broadly, to engineer exotic magnetic states and cooperative phenomena in a range of three-dimensional artificial spin ices that may have no direct analog in natural materials. In this perspective, we review the development of research into three-dimensional artificial spin ice, highlighting the main routes by which such structures can be created and measured. We discuss some new frontiers for the field, both in terms of realizing 3D model systems, and exciting opportunities for applications, such as sensing and computing.