We present an analysis of the two-point spatial correlation functions of high-mass X-ray binary (HMXB) and young star cluster (YSC) populations in M31 and M33. We find evidence that HMXBs are spatially correlated with YSCs to a higher degree than would be expected from random chance in both galaxies. When supplemented with similar studies in the Milky Way, Small Magellanic Cloud, and NGC 4449, we find that the peak value of the spatial correlation function correlates strongly with the specific star formation rate of the host galaxy. We additionally perform an X-ray stacking analysis of 211 non-X-ray detected YSCs in M31 and 463 YSCs in M33. We do not detect excess X-ray emission at the stacked cluster locations down to 3σ upper limits of ∼1033 erg s−1 (0.35-8 keV) in both galaxies, which strongly suggests that dynamical formation within YSCs is not a major HMXB formation channel. We interpret our results in the context of (1) the recent star formation histories of the galaxies, which may produce differences in the demographics of compact objects powering the HMXBs, and (2) the differences in natal kicks experienced by compact objects during formation, which can eject newly-formed HMXB from their birth clusters.