The evolution of antimicrobial resistance (AMR) in bacteria is a major public health concern. When resistant bacteria are highly prevalent in microbial populations, antibiotic restriction protocols are often implemented to reduce their spread. These measures rely on the existence of deleterious fitness effects (i.e., costs) imposed by AMR mutations during growth in the absence of antibiotics. According to this assumption, resistant strains will be outcompeted by susceptible strains that do not pay the cost during the period of restriction. Hence, the success of a given intervention depends on the magnitude and direction of fitness effects of mutations, which can vary depending on the genetic and environmental context. However, the fitness effects of AMR mutations are generally studied in laboratory reference strains and estimated in a limited number of environments, usually a standard laboratory growth medium. In this study, we systematically measure how three sources of variation impact the fitness effects of AMR mutations: the type of resistance mutation, the genetic background of the host, and the growth environment. We demonstrate that while AMR mutations are generally costly in antibiotic-free environments, their fitness effects vary widely and depend on complex interactions between the AMR mutation, genetic background, and environment. We test the ability of the Rough Mount Fuji genotype-fitness model to reproduce the empirical data in simulation. We identify model parameters that reasonably capture the variation in fitness effects due to genetic variation. However, the model fails to accommodate variation when considering multiple growth environments. Overall, this study reveals a wealth of variation in the fitness effects of resistance mutations owing to genetic background and environmental conditions, that will ultimately impact their persistence in natural populations.