The influence of core-hole delocalization for X-ray photoelectron, X-ray absorption, and X-ray emission spectrum calculations is investigated in detail, using approaches including response theory, transition-potential methods, and ground state schemes. The question of a localized/delocalized vacancy is relevant for systems with symmetrically equivalent atoms, as well as near-degeneracies which can distribute the core-orbitals over several atoms. We show that issues relating to core-hole delocalization are present for calculations considering explicit core-hole states, e.g. when using a core-excited or core-ionized reference state, or for fractional occupation numbers. Including electron correlation eventually alleviates the issues, but even using CCSD(T) there is a noticable discrepancy between core-ionization energies obtained with a localized and delocalized core-hole (0.5 eV for the carbon K-edge). Within density functional theory, the discrepancy is associated with the exchange interaction involving the core orbitals of the same spin symmetry as the delocalized core-hole, and the size of the error is thus directly related to the amount of HF exchange included in the functional. For linear response methods, we further show that if X-ray absorption spectra are modelled by considering symmetry-unique sets of atoms, care has to be taken such that there are no delocalizations of the core orbitals, which would otherwise introduce shifts in absolute energies and relative features.