Allocation of resources to growth and defense against herbivores crucially affects plant competitiveness and survival, resulting in a specific distribution of assimilates and defense compounds within plant individuals. Additionally, plants rarely experience stable environmental conditions, and adaptations to abiotic and biotic stresses may involve shifts in resistance to herbivores. We studied the allocation of phytochemicals in Brassica oleracea (Brussels sprouts) due to leaf age, drought stress and herbivore damage and assessed effects on two lepidopteran herbivores differing in diet breadth: the generalist Spodoptera littoralis and the specialist Pieris brassicae. Glucosinolates as secondary defense compounds and total nitrogen and carbon were quantified and linked to plant palatability, i.e., herbivore feeding preference. Herbivore responses were highly species-specific and partially related to changes in phytochemicals. Spodoptera littoralis preferred middle-aged leaves with intermediate levels of glucosinolates and nitrogen over young, glucosinolate and nitrogen rich leaves, as well as over old leaves, poor in glucosinolates and nitrogen. In contrast, P. brassicae preferred young leaves. Both species preferred severely drought-stressed plants to the well-watered control, although analyzed glucosinolate concentrations did not differ. Both S. littoralis and P. brassicae feeding induced an increase of indole glucosinolate levels, which may explain a reduced consumption of damaged plants detected for S. littoralis but not for P. brassicae. By revealing distinct, sometimes contrasting responses of two insect herbivores to within-plant and stress-mediated intraspecific variation in phytochemistry of B. oleracea, this study emphasizes the need to consider specific herbivore responses to understand and predict the interactions between herbivores and variable plants.