Unlike classical systems, a measurement performed on a quantum system always alters its state. In this work, the impacts of two diagnostic schemes to determine the performance of quantum Otto heat engines are compared: In one scheme the energy of the engine's working substance is measured after each stroke (repeated measurements) and in the other one the energies after each stroke are recorded in one or two pointer states and measured only after the completion of a prescribed number of cycles (repeated contacts). A single pointer state suffices if one is only interested in either work or heat. For a joint work and heat diagnostics two pointers are needed. These schemes are applied to Otto engines, whose working substance consists of a two-level system. Depending on the engine protocol, the duration of a single cycle may be infinite or finite. Because in the repeated contact scheme the number of measurements is drastically reduced compared to the repeated measurement scheme, the quantum coherence after, but also during the contact diagnostics is much better maintained as compared to repeated measurements which destroy any coherence at the end of each stroke. We demonstrate that maximum power, reliability, and efficiency of the engine in presence of repeated contacts typically outperform these figures of merit of repeated measurements. Due to the improved coherence persistence, heat engines with a finite cycle duration require a larger number of cycles to reach a periodically asymptotic state. Overall, our results document the importance of taking into account the particular nature of diagnostic tools for monitoring and testing purposes but also for feedback control, both in theory and experiment.