Nodulation process in legume plants is essential for biological nitrogen fixation during which process a large amount of phosphorus (P) is required. Under P deficiency, nodule formation is greatly affected, and induction of purple acid phosphatases (PAPs) is an adaptive strategy for nodules to acquire more P. However, regulation roles of PAPs in nodules remain largely understood. In this study, by transcriptome sequencing technology, five PAP genes were found to be differentially expressed, which led to the greatly increased acid phosphatase (APase) and phytase activities in soybean mature nodules under P starvation conditions; and among the five PAP genes, GmPAP12 had the highest transcript level, and RT-PCR indicated expression of GmPAP12 was gradually increasing during nodule development. GUS activity driven by GmPAP12 promoter was also significantly induced in low phosphorus conditions. Further functional analysis showed that under low phosphorus stress, overexpression of GmPAP12 resulted in higher nodule number, fresh weight, and nitrogenase activity as well as the APase activity than those of control plant nodules, whereas the growth performance and APase activity of nodules on hairy roots were greatly lower when GmPAP12 was suppressed, indicating that GmPAP12 may promote P utilization in soybean nodules under low P stress, which thus played an important role in nodulation and biological nitrogen fixation. Moreover, P1BS elements were found in the promoter of GmPAP12, and yeast one-hybrid experiment further proved the binding of P1BS by transcription factor GmPHR1 in the promoter of GmPAP12. At last, overexpression and suppression of GmPHR1 in nodules indeed caused highly increased and decreased expression of GmPAP12, respectively, indicating that GmPAP12 is regulated by GmPHR1 in soybean nodules. Taken together, these data suggested that GmPAP12 was a novel soybean PAP involved in the P utilization and metabolism in soybean root nodules and played an important role in the growth and development of root nodules and biological nitrogen fixation.