The Mach-Zehnder interferometer is a powerful device for detecting small phase shifts between two light beams. Simple input states-such as coherent states or single photons-can reach the standard quantum limit of phase estimation while more complicated states can be used to reach Heisenberg scaling; the latter, however, require complex states at the input of the interferometer which are difficult to prepare. The quest for highly sensitive phase estimation therefore calls for interferometers with nonlinear devices which would make the preparation of these complex states more efficient.Here, we show that the Heisenberg scaling can be recovered with simple input states (including Fock and coherent states) when the linear mirrors in the interferometer are replaced with controlled-swap gates and measurements on ancilla qubits. These swap tests project the input Fock and coherent states onto NOON and entangled coherent states, respectively, leading to improved sensitivity to small phase shifts in one of the interferometer arms. We perform detailed analysis of ancilla errors, showing that biasing the ancilla towards phase flips offers a great advantage, and perform thorough numerical simulations of a possible implementation in circuit quantum electrodynamics. Our results thus present a viable approach to phase estimation approaching Heisenberg-limited sensitivity.