Coal gangue is the main solid waste in coal mining areas, and its annual emissions account for about 10% of coal production. The composition information of coal gangue is the basis of reasonable utilization of coal gangue, and according to the composition information of coal gangue, one can choose the appropriate application scene. The reasonable utilization of coal gangue can not only effectively alleviate the environmental problems in mining areas but also produce significant economic and social benefits. Chemical analysis techniques are the principal ones used in traditional coal gangue analysis; however, they are slow and expensive. Many researchers have used machine learning techniques to analyze the spectral data of coal gangue, primarily random forests (RFs), extreme learning machines (ELMs), and two-hidden-layer extreme learning machines (TELMs). However, these techniques are heavily reliant on the preprocessing of the spectral data. This research suggests a quick analysis approach for coal gangue based on thermal infrared spectroscopy and deep learning in light of the drawbacks of the aforementioned methodologies. The proposed deep learning model is named SR-TELM, which extracts spectral features using a convolutional neural network (CNN) consisting of a spatial attention mechanism and residual connections and implements content prediction with TELM as a regressor, which can effectively overcome the dependence on preprocessing. The usefulness and speed of SR-TELM in coal gangue analysis were demonstrated by comparing several models in order to verify the proposed coal gangue analysis model. The experimental findings show that, for the prediction tasks of moisture, ash, volatile matter, and fixed carbon content, respectively, the SR-TELM model attained an R2 of 0.947, 0.972, 0.967, and 0.981 and an RMSE of 0.274, 4.040, 1.567, and 2.557 with a test time of just 0.03 s. It offers a method for the analysis of coal gangue that is low cost, highly effective, and highly reliable.