We investigate the surface of the chiral phase transition in the threedimensional parameter space of temperature, baryon chemical potential and magnetic field in two different approaches, the field-theoretical Nambu-Jona-Lasinio (NJL) model and the holographic Sakai-Sugimoto model. The latter is a top-down approach to a gravity dual of QCD with an asymptotically large number of colors and becomes, in a certain limit, dual to an NJL-like model. Our main observation is that, at nonzero chemical potential, a magnetic field can restore chiral symmetry, in apparent contrast to the phenomenon of magnetic catalysis. This "inverse magnetic catalysis" occurs in the Sakai-Sugimoto model and, for sufficiently large coupling, in the NJL model and is related to the physics of the lowest Landau level. While in most parts our discussion is a pedagogical review of previously published results, we include new analytical results for the NJL approach and a thorough comparison of inverse magnetic catalysis in the two approaches.