The manner in which costs associated with antibiotic resistance mutations are alleviated through the acquisition of compensatory mutations has bearing on the dynamics of resistance accumulation and spread as well as on our more general understanding of the persistence of adaptive genotypes. Here, we combine evolutionary experiments, fitness analyses, and whole genome sequencing to examine the way compensation to alleviate costs associated with rifampicin resistance is achieved, both when resistance is fixed and when initial majorities of resistant cells are grown alongside susceptible cells. We found compensatory mutations to occur across all examined populations. However, compensation never fully alleviates the initial costs of resistance. In contrast to prevalent assumptions, compensatory mutations mostly occurred outside of the RNA polymerase core enzyme, which serves as the target of rifampicin. The target size for compensatory mutations appears to be high, as great variation exists in compensatory mutations, even within a single population. At the same time, the pathways of compensation are quite convergent, meaning that, across independent populations, we often observe compensatory mutations in the same loci and sometimes even observe the precise same compensatory mutations across populations.