Radio interferometer arrays such as HERA consist of many close-packed dishes arranged in a regular pattern, giving rise to a large number of ‘redundant’ baselines with the same length and orientation. Since identical baselines should see an identical sky signal, this provides a way of finding a relative gain/bandpass calibration without needing an explicit sky model. In reality, there are many reasons why baselines will not be exactly identical, giving rise to a host of effects that spoil the redundancy of the array and induce spurious structure in the calibration solutions if not accounted for. In this paper, we seek to build an understanding of how differences in the primary beam response between antennas affect redundantly-calibrated interferometric visibilities and their resulting frequency (delay-space) power spectra. We use simulations to study several generic types of primary beam variation, including differences in the width of the main lobe, the angular and frequency structure of the sidelobes, and the beam ellipticity and orientation. For all of these types, we find that additional temporal structure is induced in the gain solutions, particularly when bright point sources pass through the beam. In comparison, only a low level of additional spectral structure is induced. The temporal structure modulates the cosmological 21cm power spectrum, but only at the level of a few percent in our simulations. We also investigate the possibility of signal loss due to decoherence effects when non-redundant visibilities are averaged together, finding that the decoherence is worst when bright point sources pass through the beam, and that its magnitude varies significantly between baseline groups and types of primary beam variation. Redundant calibration absorbs some of the decoherence effect however, reducing its impact compared to if the visibilities were perfectly calibrated.