Ecological pest management seeks to improve pest control through the manipulation of ecological processes that promote natural enemies and suppress pests. These approaches can involve cultural practices such as reduced tillage, increased use of non-crop plants that provide food and shelter for natural enemies, and intercropping to enhance the abundance and diversity of natural enemies. A major assumption of ecological pest management is that these activities can be equally effective for all insect herbivores. Here, I propose that these strategies may only be effective for a subset of pests and geographic regions because most insect pests have complex evolutionary histories that make them difficult to manage. I discuss how crop domestication and human-mediated migration are major evolutionary events that shape the geography of interactions between plants, herbivores, and natural enemies. Insect herbivores can evolve to be pests through three major modes: 1) herbivores associated with the crop wild ancestor may shift onto the domesticated crop, 2) herbivores may host-shift from native host plants onto an introduced crop, or 3) human-mediated migration can introduce insect pests into new cropping regions. The resulting geographic structure can influence the success of pest management by altering ecological factors such as: species distributions, patterns of biodiversity, community structure, and natural enemy attack rates. I discuss how the different modes of insect pest evolution structure a set of relevant questions and approaches for ecological pest management. By acknowledging how agricultural history and geography shape the ecology and evolution of insect pests, we may collectively develop a better capacity to identify where and how ecological pest management approaches can be most broadly effective.