Plant tolerance of herbivory, i.e., the ability to recover after damage, is an important component of how plants cope with herbivores. Tolerance has long been hypothesized to be constrained evolutionarily by plant resistance to herbivores, traits that allow plants to cope with stressful growing conditions, and traits that influence the timing of damage in relation to reproduction. Variation in tolerance and resistance can be caused by differences in the identity of the plant (e.g., genotype, species, clade) and by the context of the herbivore threat (e.g., identity of the herbivore, type of damage it causes, abiotic conditions in which the plant is growing). To date, the vast majority of studies have explored trade-offs with tolerance within species. Here, we test hypotheses of constraints on tolerance using comparative approaches in a clade of mustards, emphasizing the variety of contexts in which damage is realistically tolerated. We estimated tolerance to leaf damage, tolerance to apical clipping at the bolting stage - simulating browsing -, and resistance to a specialist and generalist lepidopteran herbivore for a group of native mustards, grown in field soils unique to each population and in a common potting soil. Resistance to herbivores was soil dependent, while surprisingly, tolerance was not. Phylogenetic signal in resistance to specialist and generalist lepidopteran herbivores was present, but only when plants were grown in field soils. Tolerance had low phylogenetic signal. Tolerance to leaf damage was unrelated to tolerance to simulated browse. We found no evidence for a resistance-tolerance trade-off, and some evidence for a soil-dependent positive correlation between tolerance and resistance to both herbivores. Drought-tolerant species had poorer ability to tolerate browse damage, and earlier flowering species tended to be less tolerant to leaf damage. Our results suggest that tolerance trades off with traits that allow mostly annual, monocarpic Streptanthus (s.l.) to persist in drought-prone conditions but is largely unrelated to resistance to herbivores. Our study highlights a need for a new framework for tolerance to herbivory that explicitly acknowledges that the relationship among tolerance, resistance, and traits that ameliorate abiotic stress.