Improvement in acid phosphatase (APase) activity is considered as an important approach to enhance phosphorus (P) utilization in crops. Here, GmPAP14 was significantly induced by low P (LP), and its transcription level in ZH15 (P efficient soybean) was higher than in NMH (P inefficient soybean) under LP conditions. Further analyses demonstrated that there were several variations in gDNA (G-GmPAP14Z and G-GmPAP14N) and the promoters (P-GmPAP14Z and P-GmPAP14N) of GmPAP14, which might bring about differential transcriptional levels of GmPAP14 in ZH15 and NMH. Histochemical staining measurements revealed that a stronger GUS signal was present in transgenic Arabidopsis with P-GmPAP14Z under LP and normal P (NP) conditions compared with the P-GmPAP14N plant. Functional research demonstrated that transgenic Arabidopsis with G-GmPAP14Z had a higher level of GmPAP14 expression than the G-GmPAP14N plant. Meanwhile, higher APase activity was also observed in the G-GmPAP14Z plant, which led to increases in shoot weight and P content. Additionally, validation of variation in 68 soybean accessions showed that varieties with Del36 displayed higher APase activities than the del36 plant. Thus, these results uncovered that allelic variation in GmPAP14 predominantly altered gene expression to influence APase activity, which provided a possible direction for research of this gene in plants.