Seedlessness is one of the most important agronomic traits in mandarins on the fresh fruit market. Creation of triploid plants is an important breeding strategy for development of new commercial varieties of seedless citrus. To this end, one strategy is to perform sexual hybridizations, with tetraploid genotypes as male parents. However, while seed development has been widely studied in citrus, knowledge of key steps such as microsporogenesis and microgametogenesis, is scarce, especially in polyploids. Therefore, we performed a study on the effect of ploidy level on pollen development by including diploid and tetraploid (double diploid) genotypes with different degrees of pollen performance. A comprehensive study on the pollen ontogeny of diploid and doubled diploid “Sanguinelli” blood orange and “Clemenules” clementine was performed, with focus on pollen grain germination in vitro and in planta, morphology of mature pollen grains by scanning electron microscopy (SEM), cytochemical characterization of carbohydrates by periodic acid–Shiff staining, and specific cell wall components revealed by immunolocalization. During microsporogenesis, the main difference between diploid and doubled diploid genotypes was cell area, which was larger in doubled diploid genotypes. However, after increase in size and vacuolization of microspores, but before mitosis I, doubled diploid “Clemenules” clementine showed drastic differences in shape, cell area, and starch hydrolysis, which resulted in shrinkage of pollen grains. The loss of fertility in doubled diploid “Clemenules” clementine is mainly due to lack of carbohydrate accumulation in pollen during microgametogenesis, especially starch content, which led to pollen grain abortion. All these changes make the pollen of this genotype unviable and very difficult to use as a male parent in sexual hybridization with the objective of recovering large progenies of triploid hybrids.