The problem of two-dimensional, independent electrons subject to a periodic potential and a uniform perpendicular magnetic field unveils surprisingly rich physics, as epitomized by the fractal energy spectrum known as Hofstadter's butterfly. It has hitherto been addressed using various approximations rooted in either the strong potential or the strong field limiting cases. Here, we report calculations of the full spectrum of the single-particle Schrödinger equation without further approximations. Our method is exact, up to numerical precision, for any combination of potential and uniform field strength. We first study a situation that corresponds to the strong potential limit, and compare the exact results to the predictions of a Hofstadter-like model. We then go on to analyze the evolution of the fractal spectrum from a Landau-like nearly free electron system to the Hofstadter tight-binding limit by tuning the amplitude of the modulation potential.