Semiempirical (SM2, SM5.4A, MST‐AM1, COSMO‐AM1) and ab inito (HF/PCM‐vdW, MP2//PCM‐vdW, COSMO‐DFT) dielectric continuum‐solvation models as well as the surface‐tension model SM5.0R are analyzed with respect to predicting Henry's law constant at 25°C using a compound set of benzene and 39 benzene derivatives. Both hydrophilic and hydrophobic compounds are covered with a total variation in Henry's law constant of almost eight orders of magnitude corresponding to 44 kJ/mol, and the data set is selected such that there are cases where subtle changes in the molecular structure result in substantial changes of the free energy of solvation. The calculations with SM2, COSMO‐AM1, and COSMO‐DFT include solution‐phase geometry optimization, and the ab initio results refer to polarized basis sets of double‐zeta quality, with two gradient‐corrected functionals (BPW and BLYP) being used for the DFT‐based models. The results show considerable differences in performance between the different continuum‐solvation models, and among the methods yielding solvation free energies the systematic error ranges from −0.9 kJ/mol (SM5.0R) to 12.1 kJ/mol (MP2//PCM‐vdW). In particular, the nonelectrostatic solvation energy contributions of SM2, SM5.4A, MST‐AM1, and PCM‐vdW do not correlate with each other, and with PCM‐vdW omission of the nonelectrostatic component significantly improves the relative trend. The best statistics after scaling through linear regression are achieved with the electrostatic component of MP2//PCM‐vdW (r 2adj=0.94) and with COSMO‐DFT (r 2adj=0.93). The discussion includes detailed analyses of pecularities associated with certain functional groups, deviations from the expected relationship between dipole moment and solvation energy, and a simple approach to model dispersion interaction and cavitation energy by surface area terms that differentiate between individual atom types. © 2000 John Wiley & Sons, Inc. J Comput Chem 21: 17–34, 2000