A novel phosphorus-nitrogen-containing flame retardant (DOPT) has been successfully synthesized via the substitution reaction of cyanuric chloride, pentaerythritol phosphate and 9, 10-dihydro-9-oxa-10-phosphaphenanthrene-10-oxide. The chemical structure of DOPT was confirmed by 1H, 31P and 13C nuclear magnetic resonance, Fourier transform infrared spectroscopy and elemental analysis. Then, flame retardants were added to epoxy resin to prepare epoxy resin composites by pouring method. Thermal properties, flame retardancy, and combustion behavior of epoxy resin composites were evaluated by thermogravimetric analysis, vertical burning, limiting oxygen index and cone calorimeter test. Thermogravimetric analysis test showed that the carbon residue rate of DOPT at 800°C reached 52.53%, which indicated that the introduction of high-efficiency char-forming agent triazine and pentaerythritol phosphate groups could significantly improve its char-forming performance and thermal stability. The epoxy resin composite achieved vertical burning V-0 grade and the limiting oxygen index value reached 35.5% when 7 wt% DOPT was incorporated. Furthermore, the cone calorimeter test results manifested that the addition of DOPT stimulated degradation of the epoxy resin matrix during the combustion process and accelerated the formation of an expanded and dense carbon layer. Additionally, the incombustible gas produced during the decomposition of DOPT had played a flame-retardant effect in the gas phase. Hence, compared with neat epoxy resin, the total heat release and total smoke production of the EP-7 wt% DOPT composite decreased by 14.0% and 25.3%, respectively. Moreover, owing to the excellent compatibility and the strong interface effect between DOPT and epoxy resin, the addition of DOPT also enhanced the mechanical and fire resistance properties of the epoxy resin composite. Therefore, it is proposed that DOPT could be exploited as an economical and high-efficiency flame retardant, and it has considerable prospects in flame retardant epoxy resin composites.