For the energy‐saving and safe utilization of industrial furnaces, it is essential to develop refractories with low thermal conductivity and high strength. In this study, low‐grade bauxite and coal gangue were used as the main raw materials to prepare microporous mullite refractories with low thermal conductivity and high strength by the in situ decomposition method. The effect of coal gangue content and sintering temperature on the structure and properties of mullite refractories was also investigated. The results show that the average pore size of mullite refractories containing coal gangue is between 1.3 and 4.4 μm; with the addition of coal gangue, the apparent porosity of mullite refractories increases and their thermal conductivity decreases. Coal gangue reduces the number of pores larger than 14 μm, improves the microstructure uniformity of mullite refractories, and increases their cold compressive strength. The strength of microporous refractory treated at 1500°C containing 36.2% coal gangue as raw material is up to (105.6 ± 1.7 MPa); and its thermal conductivity tested at 1000°C is only .442 W/(m K)).