The optical properties of perylene-based pigments are arising from the interplay between neutral molecular excitations and charge transfer between adjacent molecules. In the crystalline phase, these excitations are coupled via electron and hole transfer, two quantities relating directly to the width of the conduction and valence band in the crystalline phase. Based on the crystal structure determined by x-ray diffraction, densityfunctional theory ͑DFT͒ and Hartree-Fock are used for the calculation of the electronic states of a dimer of stacked molecules. The resulting transfer parameters for electron and hole are used in an exciton model for the coupling between Frenkel excitons and charge-transfer states. The deformation of the positively or negatively charged molecular ions with respect to the neutral ground state is calculated with DFT and the geometry in the optically excited state is deduced from time-dependent DFT and constrained DFT. All of these deformations are interpreted in terms of the elongation of an effective internal vibration which is used subsequently in the exciton model for the crystalline phase. A comparison between the calculated dielectric function and the observed optical spectra allows to deduce the relative energetic position of Frenkel excitons and the chargetransfer state involving stack neighbors, a key parameter for various electronic and optoelectronic device applications. For five out of six perylene pigments studied in the present work, this exciton model results in excellent agreement between calculated and observed optical properties.