Zinc phosphinate (Exolit OP950), nanosilica particles and polyethylene terephthalate (PET) have been blended and thereafter melt‐spinned to develop a new flame retardant (FR) system for PET textiles. The effects of the two types of nanosilica fillers on the wettability, dispersibility and thermal properties were studied to determine how the degree of hydrophilicity affects the PET matrix. The influence of the blends on thermal transitions has been investigated by differential scanning calorimetry (DSC), the thermal stability of the polymer/FR blend composites has been assessed by thermogravimetric analysis (TGA) and cone calorimetry has been used to study the fire reaction. It was noticed that the nanoparticles have a limited influence on the thermal transitions of the PET matrix, but zinc phosphinate acted as a plasticizer and a compatibilizer for the more hydrophobic particles. Thermogravimetric analysis results showed that the addition of silica particles and FR compound improves slightly the thermal stability of the PET systems under nitrogen and air atmospheres. Furthermore, it was noticed that the incorporation of nanoparticles gives almost no improvement in the PET fire reaction from cone calorimeter experiments. However, in the presence of Exolit OP950, the systems acted as FR in PET films and knitted structures. The heat release rate during the combustion decreased, and the thermal behaviors of these structures were closed to those with 10% wt of Exolit OP950. Copyright © 2017 John Wiley & Sons, Ltd.