The possibility of intrinsic barriers to gene flow is often neglected in empirical research on local adaptation and speciation with gene flow, for example when interpreting patterns observed in genome scans. However, we draw attention to the fact that, even with gene flow, divergent ecological selection may generate intrinsic barriers involving both ecologically selected and other interacting loci. Mechanistically, the link between the two types of barriers may be generated by genes that have multiple functions (i.e., pleiotropy), and/or by gene interaction networks. Because most genes function in complex networks, and their evolution is not independent of other genes, changes evolving in response to ecological selection can generate intrinsic barriers as a by-product. A crucial question is to what extent such by-product barriers contribute to divergence and speciation-that is whether they stably reduce gene flow. We discuss under which conditions by-product barriers may increase isolation. However, we also highlight that, depending on the conditions (e.g., the amount of gene flow and the strength of selection acting on the intrinsic vs. the ecological barrier component), the intrinsic incompatibility may actually destabilize barriers to gene flow. In practice, intrinsic barriers generated as a by-product of divergent ecological selection may generate peaks in genome scans that cannot easily be interpreted. We argue that empirical studies on divergence with gene flow should consider the possibility of both ecological and intrinsic barriers. Future progress will likely come from work combining population genomic studies, experiments quantifying fitness and molecular studies on protein function and interactions.