A substantial amount of phenotypic diversity results from changes in gene regulation. Understanding how regulatory diversity evolves is therefore a key priority in identifying mechanisms of adaptive change. However, in contrast to powerful models of sequence evolution, we lack a consensus model of regulatory evolution. Furthermore, recent work has shown that many of the comparative approaches used to study gene regulation are subject to biases that can lead to false signatures of selection. In this review, we first outline the main approaches for describing regulatory evolution and their inherent biases. Next, we bridge the gap between the fields of comparative phylogenetic methods and transcriptomics to reinforce the main pitfalls of inferring regulatory selection and use simulation studies to show that shifts in tissue composition can heavily bias inferences of selection. We close by highlighting the multi-dimensional nature of regulatory variation and identifying major, unanswered questions in disentangling how selection acts on the transcriptome.