Social gaze has received much attention in social cognition research in both human and non-human animals. Gaze following appears to be a central skill for acquiring social information, such as the location of food and predators, but can also draw attention to important social interactions, which in turn promotes the evolution of more complex socio-cognitive processes such as theory of mind and social learning. In the past decades, a large number of studies has been conducted in this field introducing differing methodologies. Thereby, various factors influencing the results of gaze following experiments have been identified. This review provides an overview of the advances in the study of gaze following, but also highlights some limitations within the research area. The majority of gaze following studies on animals have focused on primates and canids, which limits evolutionary interpretations to only a few and closely related evolutionary lineages. This review incorporates new insights gained from previously understudied taxa, such as fishes, reptiles, and birds, but it will also provide a brief outline of mammal studies. We propose that the foundations of gaze following emerged early in evolutionary history. Basic, reflexive co-orienting responses might have already evolved in fishes, which would explain the ubiquity of gaze following seen in the amniotes. More complex skills, such as geometrical gaze following and the ability to form social predictions based on gaze, seem to have evolved separately at least two times and appear to be correlated with growing complexity in brain anatomy such as increased numbers of brain neurons. However, more studies on different taxa in key phylogenetic positions are needed to better understand the evolutionary history of this fundamental socio-cognitive skill.