The effect of graphene oxide (GO) functionalized with 3‐amino‐propyl‐triethoxy‐silane(APTS), organo‐vermiculite(OVMT), and magnesium hydroxide (MH) combinations on mechanical, thermal, and flame retardant (FR) properties of polypropylene (PP) was studied. GO was obtained via a slight modification of the Hummers method and then chemically surface functionalized with APTS. VMT clay was modified with maleic anhydride (MA) via a chemical reaction with acetic acid to increase its inter‐laminar spacing. The results of Fourier‐transform infrared analysis, X‐ray diffraction, and transmission electron microscopy demonstrated that APTS had been successfully attached to the GO and that VMT was modified with MA. Subsequently, each functionalized filler was incorporated in combination with MH to the flame‐retardant‐polypropylene system. The performance of PP composites with each filler as well as with their combinations, including a reduced “30 wt% of MH” were compared with the reference PP composite with 55 wt% of MH as the only FR additive. PP grafted with MA (PP‐gMA) and PP grafted with amine‐alcohol(PP‐gDMAE) were used as compatibilizer agents between each filler and the polymer matrix. The results obtained confirmed a better mechanical and FR performance when using PP‐gDMAE. Composites with 30 wt% MH combined with very low contents of GO (0.5 and 1.0 wt%) showed improved FR properties, similar to the reference sample, with an evident reduction in peak of the heat release rate (pHRR) and total heat release and increased limiting oxygen index (LOI) values. The combination of MH and GO showed the best FR and mechanical properties: LOI of 22.5% and pHRR of 540 kW/m2 which were very similar to the reference sample. In addition, PP‐gDMAE improved the OVMT exfoliation with a slight increase in the intergallery spacing but OVMT and modified GO combinations did not significantly improve the FR and mechanical properties compared with the effect of MH and modified GO combinations. The combination of MH and modified GO makes it possible to reduce the total MH filler content from 55 to 30 wt% to accomplish the FR requirements and with enhanced mechanical properties. This filler combination promoted the formation of a continuous, intact residual char layer on the PP surface, which acts as an insulating barrier to protect the base material. These filler combinations offer an option to meet the FR properties using halogen‐free FR with better mechanical properties.