The creation of delocalized coherent superpositions of quantum systems experiencing different relativistic effects is an important milestone in future research at the interface of gravity and quantum mechanics. This could be achieved by generating a superposition of quantum clocks that follow paths with different gravitational time dilation and investigating the consequences on the interference signal when they are eventually recombined. Light-pulse atom interferometry with elements employed in optical atomic clocks is a promising candidate for that purpose, but suffers from major challenges including its insensitivity to the gravitational redshift in a uniform field. All these difficulties can be overcome with a novel scheme presented here which is based on initializing the clock when the spatially separate superposition has already been generated and performing a doubly differential measurement where the differential phase shift between the two internal states is compared for different initialization times. This can be exploited to test the universality of the gravitational redshift (UGR) with delocalized coherent superpositions of quantum clocks and it is argued that its experimental implementation should be feasible with a new generation of 10-meter atomic fountains that will soon become available. Interestingly, the approach also offers significant advantages for more compact set-ups based on guided interferometry or hybrid configurations. Furthermore, in order to provide a solid foundation for the analysis of the various interferometry schemes and the effects that can be measured with them, a general formalism for a relativistic description of atom interferometry in curved spacetime is developed. It can deal with freely falling atoms, but also include the effects of external forces and guiding potentials, and can be applied to a very wide range of situations. As an important ingredient for quantum-clock interferometry, suitable diffraction mechanisms for atoms in internal-state superpositions are investigated too. Finally, the relation of the proposed doubly-differential measurement scheme to other experimental approaches and to tests of the universality of free fall (UFF) is discussed in detail. arXiv:1810.06744v1 [physics.atom-ph]