Insect species richness is estimated to exceed three million species, of which roughly half is herbivorous. Despite the vast number of species and varied life histories, the proportion of herbivorous species among plant-consuming organisms is lower than it could be due to constraints that impose limits to their diversification. These include ecological factors, such as vague interspecific competition; anatomical and physiological limits, such as neural limits and inability of handling a wide range of plant allelochemicals; phylogenetic constraints, like niche conservatism; and most importantly, a low level of concerted genetic variation necessary to a phyletic conversion. It is suggested that diversification ultimately depends on what we call the intrinsic trend of diversification of the insect genome. In support of the above, we survey the major types of host-specificity, the mechanisms and constraints of host specialization, possible pathways of speciation, and hypotheses concerning insect diversification.