An aluminum silicate fiber/alumina aerogel (ASF/AA) composite was prepared via the sol-gel method and atmospheric drying (APD) method using coal gangue (CG) solid waste from Xingxian county, Shanxi Province, as the aluminum source. Utilizing N2 adsorption, scanning electron microscopy, Fourier transform infrared spectroscopy, X-ray diffraction, and thermal conductivity meters, researchers examined the microstructure, composition, pore structure, and thermal insulation performance of ASF/AA composites. The thermal conductivity mechanism of the composite was analyzed. The experimental results show that most of the aluminum in CG is used. More importantly, in the process of aerogel synthesis, the atmospheric pressure drying method is used to obtain similar properties to supercritical drying. The composite material exhibits a low thermal conductivity of 0.047 W/(m·K), a high specific surface area of 416 m2/g, and a low density of 0.26 g/cm3 at room temperature. After heating at 1200 °C for 2 h, the thermal conductivity was as low as 0.071 W/(m·K). This strategy can not only effectively achieve a reduction in the harmfulness of solid waste coal gangue, but also alleviate the shortage of related energy and resources in our country.