Brownmillerite-belite-sulfoaluminate clinker with different contents of brownmillerite were designed and successfully prepared by using limestone (LS), aluminum tailings (AT), aluminum mine (AM), and anhydrite (AH) calcined at 1330 °C for 30 min. Then, three kinds of brownmillerite-belite-sulfoaluminate cement (BBSC) were obtained by grinding mixtures of the clinker and AH. Hydration and mechanical performances of the prepared BBSC were thus intensively studied. The increase in brownmillerite in BBSC decreased the hydration exothermic rate and delayed the renewed rapid formation of AFt at early hydration stages. However, the formation of C2AS·8H2O would be promoted, where the higher the brownmillerite content in BBSC, the earlier the C2AS·8H2O formed. The increase in brownmillerite might change the morphologies of the formed AFt, grass-shaped AFt enriched in iron would be the main hydration products in BBSC with a higher content of brownmillerite. The increase in brownmillerite content contributed to the stability improvement in flexural strength and the stable growth in the compressive strength of BBSC. The appropriate content of brownmillerite (20 wt%) can balance the whole hydration reaction process, which was conducive to the development of BBSC mechanical strength, the decrease in the hydration heat release, and the volume stability of hardened pastes.