The photon polarization tensor is the central building block of an effective theory description of photon propagation in the quantum vacuum. It accounts for the vacuum fluctuations of the underlying theory, and in the presence of external electromagnetic fields, gives rise to such striking phenomena as vacuum birefringence and dichroism. Standard approximations of the polarization tensor are often restricted to on-the-light-cone dynamics in homogeneous electromagnetic fields, and are limited to certain momentum regimes only. We devise two different strategies to go beyond these limitations: First, we aim at obtaining novel analytical insights into the photon polarization tensor for homogeneous fields, while retaining its full momentum dependence. Second, we employ wordline numerical methods to surpass the constant-field limit.