Backfill mining is an effective way to solve environment pollute, surface subsidence, and groundwater system damage which caused by coal mining. However, the complex underground environment may change the physical and mechanical properties of the backfill body, which poses a risk of strength failure. This paper analyzed the failure of gangue-based cemented backfill body which made up of gangue and fly ash. The results show that physicochemical reactions will generate quartz, kaolinite, and other high-strength substances; hydration reaction between the fine particles will generate hydrocalcium silicate and other C-S-H gels, they wrapped gangues as a whole, which provide a high strength of the cemented backfill body. Several experiments were carried out in order to find the reason for failure in samples under loads. The conclusion drawn as following: (1) When the load is large, the cracks extend from the surface of the samples to the interior, at the same time, the length and width of the cracks increasing obviously and connecting as net. Especially the external load exceeds the peak intensity. (2) The relationship between sample failure and pores is weak, but obvious with crack development, especially the cracks connected as a net. (3) The interface structure formed by gangue is an important source of crack development and, thus, will stimulate the development of cracks.