The mutation rate is an important determinant of evolutionary dynamics. Because the mutation rate determines the rate of appearance of beneficial and deleterious mutations, it is subject to second-order selection. The mutation rate varies between and within species and populations, increases under stress, and is genetically controlled by mutator alleles. The mutation rate may also vary among genetically identical individuals: empirical evidence from bacteria suggests that the mutation rate may be affected by translation errors and expression noise in various proteins (1). Importantly, this non-genetic variation may be heritable via transgenerational epigenetic inheritance. Here we investigate how the inheritance mode of the mutation rate affects the rate of adaptive evolution on rugged fitness landscapes. We model an asexual population with two mutation rate phenotypes, non-mutator and mutator. An offspring may switch from its parental phenotype to the other phenotype. The rate of switching between the mutation rate phenotypes is allowed to span a range of values. Thus, the mutation rate can be interpreted as a genetically inherited trait when the switching rate is low, as an epigenetically inherited trait when the switching rate is intermediate, or as a randomly determined trait when the switching rate is high. We find that epigenetically inherited mutation rates result in the highest rates of adaptation on rugged fitness landscapes for most realistic parameter sets. This is because an intermediate switching rate can maintain the association between a mutator phenotype and pre-existing mutations, which facilitates the crossing of fitness valleys. Our results provide a rationale for the evolution of epigenetic inheritance of the mutation rate, suggesting that it could have been selected because it facilitates adaptive evolution.