Enzymes of the α-carbonic anhydrase gene family (CAs) are essential for the deposition of calcium carbonate biominerals. In calcareous sponges (phylum Porifera, class Calcarea), specific CAs are involved in the formation of calcite spicules, a unique trait and synapomorphy of this class. However, detailed studies on the CA repertoire of calcareous sponges exist for only two species of one of the two Calcarea subclasses, the Calcaronea. The CA repertoire of the second subclass, the Calcinea, has not been investigated so far, leaving a considerable gap in our knowledge about this gene family in Calcarea. Here, using transcriptomic analysis, phylogenetics, and in situ hybridization, we study the CA repertoire of four additional species of calcareous sponges, including three from the previously unsampled subclass Calcinea. Our data indicate that the last common ancestor of Calcarea had four ancestral CAs with defined subcellular localizations and functions (mitochondrial/cytosolic, membrane-bound, and secreted non-catalytic). The evolution of membrane-bound and secreted CAs involved gene duplications and losses, whereas mitochondrial/cytosolic and non-catalytic CAs are evidently orthologous genes. Mitochondrial/cytosolic CAs are biomineralization-specific genes recruited for biomineralization in the last common ancestor of calcareous sponges. The spatial–temporal expression of these CAs differs between species, which may reflect differences between subclasses or be related to the secondary thickening of spicules during biomineralization that does not occur in all species. With this study, we extend the understanding of the role and the evolution of a key biomineralization gene in calcareous sponges.