In light of the conference Quantum Mathematical Physics held in Regensburg in 2014, we give our perspective on the external field problem in quantum electrodynamics (QED), i.e., QED without photons in which the sole interaction stems from an external, time-dependent, four-vector potential. Among others, this model was considered by Dirac, Schwinger, Feynman, and Dyson as a model to describe the phenomenon of electron-positron pair creation in regimes in which the interaction between electrons can be neglected and a mean field description of the photon degrees of freedom is valid (e.g., static field of heavy nuclei or lasers fields). Although it may appear as second easiest model to study, it already bares a severe divergence in its equations of motion preventing any straight-forward construction of the corresponding evolution operator. In informal computations of the vacuum polarization current this divergence leads to the need of the so-called charge renormalization. In an attempt to provide a bridge between physics and mathematics, this work gives a review ranging from the heuristic picture to our rigorous results in a way that is hopefully also accessible to non-experts and students. We discuss how the evolution operator can be constructed, how this construction yields well-defined and unique transition probabilities, and how it provides a family of candidates for charge current operators without the need of removing ill-defined quantities. We conclude with an outlook of what needs to be done to identify the physical charge current among this family.