Quantum dots (QDs) with optically active Cu impurities have been proposed as heavy-metal free alternatives to Cd and Pb chalcogenides. However, the origin of their unusual optical properties is not well understood. In particular, spectral broadening is an issue for their use in high color purity light-emitting diodes, and reabsorption-free solar windows. Here, we show with density functional theory calculations that chemical bonding variations have a major effect on the optical properties of Cu doped ZnSe QDs. The Cu-Se coordination sphere is highly covalent, and therefore sensitive to local variations in electrostatics and bond geometry. Correspondingly, changes in the Cu impurity environment lead to large shifts in their ground state energy, which causes spectral broadening when multiple Cu impurity bonding environments coexist as subensembles with distinct absorption and emission energies. We conclude that while electronphonon coupling is stronger for these systems than for typical II-VI QDs, spectral broadening predominantly occurs due to the inhomogeneous spatial distribution of Cu impurities. This is in agreement with studies that have shown narrow (~60 meV) single-particle emission linewidths in related Cu x In 2-x Se y S 2-y , or "CIS" QDs, which also emit through Cu impurities. Hence, we predict that narrow ensemble emission in photonic devices can be achieved if heterogeneity is controlled.