A halogen bond is a highly directional, non-covalent interaction between a halogen atom and another electronegative atom. It arises due to the formation of a small region of positive electrostatic potential opposite the covalent bond to the halogen, called the “sigma hole.” Empirical force fields in which the electrostatic interactions are represented by atom-centered point charges cannot capture this effect because halogen atoms usually carry a negative charge and therefore interact unfavorably with other electronegative atoms. A strategy to overcome this problem is to attach a positively charged virtual particle to the halogen. In this work, we extend the additive CHARMM General Force Field (CGenFF) to include such interactions in model systems of phenyl-X, with X being Cl, Br or I including di- and trihalogenated species. The charges, Lennard-Jones parameters, and halogen-virtual particle distances were optimized to reproduce the orientation dependence of quantum mechanical interaction energies with water, acetone, and N-methylacetamide as well as experimental pure liquid properties and relative hydration free energies with respect to benzene. The resulting parameters were validated in molecular dynamics simulations on small-molecule crystals and on solvated protein-ligand complexes containing halogenated compounds. The inclusion of positive virtual sites leads to better agreement across experimental observables, including preservation of ligand binding poses as a direct result of the improved representation of halogen bonding.