It is well known that gas-phase experiments and computational methods point to the dominance of dispersion forces in the molecular association of hydrocarbons. Estimates or even quantification of these weak forces are complicated due to solvent effects in solution. The dissection of interaction energies and quantification of dispersion interactions is particularly challenging for polar systems such as ionic liquids (ILs) which are characterized by a subtle balance between Coulomb interactions, hydrogen bonding, and dispersion forces. Here, we have used vaporization enthalpies, far-infrared spectroscopy, and dispersion-corrected calculations to dissect the interaction energies between cations and anions in aprotic (AILs), and protic (PILs) ionic liquids. It was found that the higher total interaction energy in PILs results from the strong and directional hydrogen bonds between cation and anion, whereas the larger vaporization enthalpies of AILs clearly arise from increasing dispersion forces between ion pairs.