Using state-of-the-art ab initio GW many-body perturbation theory calculations, we show that monolayer MoS2 on Au is a p-type contact, in contrast to the vast majority of theoretical predictions using density functional theory. The predominantly n-type behaviour observed experimentally for MoS2/Au junctions can be attributed to the presence of sulfur vacancies, which pin the Fermi level. GW calculations on WSe2/Au junctions likewise predict p-type contacts for pristine WSe2 and n-type contacts for junctions with selenium vacancies. Experimentally, WSe2/metal junctions are predominantly p-type or ambipolar, with p-type junctions being observed for selenium-deficient WSe2, suggesting that selenium vacancies are not effective in pinning the Fermi level for WSe2/metal junctions. We rationalize these apparently contradictory results by noting that selenium vacancies in WSe2 are readily passivated by oxygen atoms. Taken together, our state-of-the-art calculations clearly elucidate the relation between contact polarity and atomic structure. We show that non-local exchange and correlation effects are critical for determining the energy level alignment and even the contact polarity (in the case of MoS2 on Au). We further reconcile a large body of experimental literature on TMDC/metal contact polarities by consideration of the defect chemistry.