Yellow phosphorus slag (YPS) is a byproduct from the production of yellow phosphorus. It has potential pozzolanic activity and can be used as a supplementary cementitious material. However, the early strength of cement mortar decreases significantly with increasing YPS dosage, which restricts the utilization of YPS in cement and concrete. This study aimed to increase the pozzolanic activity of YPS ash by thermal activation. The strength method, alkali dissolution method and polymerization degree method were used to evaluate the effect of thermal activation at different temperatures on the pozzolanic activity of YPS ash. The results showed that YPS ash calcined at 800 °C helps to enhance the early strength because the fluorine in cuspidine (Ca4Si2O7F2) is insoluble, reducing the retarding effect on the mortar. The higher late strength of YPS ash calcined at 100 °C was due to the low polymerization degree of [SiO4]. The pozzolanic activity of YPS ash is positively correlated with the dissolution concentration of (Si + Al) and the compressive strength and negatively associated with the polymerization degree. This paper shows a possibility for the large-scale utilization of YPS.