Interferometric scattering (iSCAT) microscopy is an emerging label-free technique optimized for the sensitive detection of nano-matter. Previous iSCAT studies have approximated the point spread function in iSCAT by a Gaussian intensity distribution. However, recent efforts to track the mobility of nanoparticles in challenging speckle environments and over extended axial ranges has necessitated a quantitative description of the interferometric point spread function (iPSF). We present a robust vectorial diffraction model for the iPSF in tandem with experimental measurements and rigorous FDTD simulations. We examine the iPSF under various imaging scenarios to understand how aberrations due to the experimental configuration encode information about the nanoparticle. We show that the lateral shape of the iPSF can be used to achieve nanometric three-dimensional localization over an extended axial range on the order of 10 µm either by means of a fit to an analytical model or calibration-free unsupervised machine learning. Our results have immediate implications for three-dimensional single particle tracking in complex scattering media.