As the conventional disposal method for industrial by-products and wastes, landfills can cause environmental pollution and huge economic costs. However, some secondary materials can be effectively used to develop novel underground filling materials. Controlled low-strength material (CLSM) is a highly flowable, controllable, and low-strength filling material. The rational use of coal industry by-products to prepare CLSM is significant in reducing environmental pollution and value-added disposal of solid waste. In this work, five different by-products of the coal industry (bottom ash (BA), fly ash, desulfurized gypsum, gasification slag, and coal gangue) and cement were used as mixtures to prepare multi-component coal industry solid waste-based CLSM. The microstructure and phase composition of the obtained samples were analyzed by scanning electron microscopy and X-ray diffraction. In addition, the particle size/fineness of samples was also measured. The changes in fresh and hardened properties of CLSM were studied using BA after ball milling for 20 min (BAI group) and 45 min (BAII group) that replaced fly ash with four mass ratios (10 wt%, 30 wt%, 50 wt%, and 70 wt%). The results showed that the CLSM mixtures satisfied the limits and requirements of the American Concrete Institute Committee 229 for CLSM. Improving the mass ratio of BA to fly ash and the ball-milling time of the BA significantly reduced the flowability and the bleeding of the CLSM; the flowability was still in the high flowability category, the lowest bleeding BAI70 (i.e., the content of BA in the BAI group was 70 wt%) and BAII70 (i.e., the content of BA in the BAII group was 70 wt%) decreased by 48% and 64%, respectively. Furthermore, the 3 d compressive strengths of BAI70 and BAII70 were increased by 48% and 93%, respectively, compared with the group without BA, which was significantly favorable, whereas the 28 d compressive strength did not change significantly. Moreover, the removability modulus of CLSM was calculated, which was greater than 1, indicating that CLSM was suitable for structural backfilling that requires a certain strength. This study provides a basis for the large-scale utilization of coal industry solid waste in the construction industry and underground coal mine filling.