A-kinase anchoring protein 12 (AKAP12) plays key roles in male germ cells and female ovarian granulosa cells, whereas its influence on livestock litter size remains unclear. Herein we detected the genetic variants of AKAP12 gene and their effects on litter size as well as alternative splicing variants expression in Shaanbei white cashmere (SBWC) goats, aiming at exploring theoretical basis for goat molecular breeding. We identified two Insertion/deletions (Indels) (7- and 13-bp) within the AKAP12 gene. Statistical analyses demonstrated that the 13-bp indel mutation in the 3′ UTR was significantly associated with litter size (n = 1,019), and the carriers with DD genotypes presented lower litter sizes compared with other carriers (P < 0.01). Bioinformatics analysis predicted that this 13-bp deletion sequence could bind to the seed region of miR-181, which has been documented to suppress porcine reproductive and respiratory syndrome virus (PRRSV) infection by targeting PRRSV receptor CD163 and affect the pig litter size. Therefore, luciferase assay for this 13-bp indel binding with miRNA-181 was performed, and the luciferase activity of pcDNA-miR-181-13bp-Deletion-allele vector was significantly lower than that of the pcDNA-miR-181-13bp-Insertion-allele vector (P < 0.05), suggesting the reduced binding capability with miR-181 in DD genotype. Given that alternative spliced variants and their expression considerably account for the Indel genetic effects on phenotypic traits, we therefore detected the expression of the alternative spliced variants in different tissues and identified that AKAP12-AS2 exhibited the highest expression levels in testis tissues. Interestingly, the AKAP12-AS2 expression levels of homozygote DD carriers were significantly lower than that of individuals with heterozygote ID, in both testis and ovarian tissues (P < 0.05), which is consistent with the effect of the 13-bp deletion on the reduced litter size. Taken together, our results here suggest that this 13-bp indel mutation within goat AKAP12 might be utilized as a novel molecular marker for improving litter size in goat breeding.