Phosphorus, nitrogen-containing monomer, acryloxyethyl phenoxy phosphorodiethyl amidate (AEPPA), was synthesized and copolymerized with styrene (St). Nanocomposites of polystyrene and poly(St-co-AEPPA) with various amounts of Mg-Al layered double hydroxide (LDH) were then prepared by in situ bulk polymerization. Structure and morphology of the nanocomposites were investigated by Fourier transform infrared (FTIR), X-ray diffraction (XRD), and transmission electron microscopy (TEM). The nanocomposites were also examined by differential scanning calorimetry (DSC), thermogravimetric analysis (TGA), and microscale combustion calorimeter (MCC) to evaluate the thermal properties and flammability. Intercalated or exfoliated structures were obtained for all the nanocomposites. The results from XRD and TEM showed the LDH layers dispersed better in poly(St-co-AEPPA) than those in PS matrix. Decrease in thermal stability and enhancement in char residues were observed for poly(St-co-AEPPA) nanocomposites compared with PS nanocomposites at the same LDH loading. The addition of LDH can obviously reduce the heat release capacity (HRC) and total heat release (THR) of PS. Moreover, further reductions in HRC and THR were found in poly(St-co-AEPPA) nanocomposites. The reduction in flammability was attributed to the lower maximum mass loss rate (MMLR) and higher char residues of the nanocomposites.