The aim of this work was to investigate the dispersibility of kaolinite-rich coal gangue in rubber matrix, the mechanical properties and thermal stability of coal gangue/styrene butadiene rubber (SBR) composites, and to compare these properties to those of the same coal gangue but had undergone thermal activation and modification. Several experimental techniques, such as X-ray diffraction (XRD), X-ray fluorescence (XRF), scanning electron microscopy (SEM), Fourier-transform infrared spectroscopy (FTIR), thermogravimetric-differential scanning calorimetry (TG-DSC), laser-scattering particle analyzer were adopted to characterize the coal gangue particles and then the obtained composites. The results demonstrated the raw coal gangue (RCG) was mainly composed of kaolinite. Calcination led to amorphization of thermal activated coal gangue (ACG), increased hydrophilicity and void volume, and decreased pH. The grain size of ACG became coarser than RCG, but ACG turned loose confirmed by higher degree of refinement after grinding. Modification enhanced the hydrophobicity of the coal gangue and improved its dispersibility than fillers without modification. Calcined samples had better dispersibility than uncalcined fillers. Additionally, the coal gangue treated by calcinating, grinding and modifying (MGA) had the best dispersion in rubber matrix. Either calcination or modification could improve the mechanical properties and thermal stability of coal gangue filled rubber, while the performance of MGA reinforced SBR (MGA-SBR) was the best. The enhanced performance of the MGA-SBR was owed to better dispersion of particles as well as stronger interactions between particles and rubber macromolecules.