Mutualistic symbioses are common throughout the animal kingdom. Rather unusual is a form of symbiosis, photosymbiosis, where animals are symbiotic with photoautotrophic organisms. Photosymbiosis is found among sponges, cnidarians, flatworms, molluscs, ascidians and even some amphibians. Generally the animal host harbours a phototrophic partner, usually a cyanobacteria or a unicellular alga. An exception to this rule is found in some sea slugs, which only retain the chloroplasts of the algal food source and maintain them photosynthetically active in their own cytosol - a phenomenon called 'functional kleptoplasty'. Research has focused largely on the biodiversity of photosymbiotic species across a range of taxa. However, many questions with regard to the evolution of the ability to establish and maintain a photosymbiosis are still unanswered. To date, attempts to understand genome adaptations which could potentially lead to the evolution of photosymbioses have only been performed in cnidarians. This knowledge gap for other systems is mainly due to a lack of genetic information, both for non-symbiotic and symbiotic species. Considering non-photosymbiotic species is, however, important to understand the factors that make symbiotic species so unique. Herein we provide an overview of the diversity of photosymbioses across the animal kingdom and discuss potential scenarios for the evolution of this association in different lineages. We stress that the evolution of photosymbiosis is probably based on genome adaptations, which (i) lead to recognition of the symbiont to establish the symbiosis, and (ii) are needed to maintain the symbiosis. We hope to stimulate research involving sequencing the genomes of various key taxa to increase the genomic resources needed to understand the most fundamental question: how have animals evolved the ability to establish and maintain a photosymbiosis?