To improve the mechanical performance and lower the production cost of magnesium oxysulfate cement (MOSC), this article investigates the effects of single and compounded addition of metakaolin (MK) and/or fly ash (FA) on the setting time, mechanical strength, water resistance, hydration product, composition, and microstructure of the resulting cement. MOSC samples with different proportions, ranging from 0 to 30 wt.%, of FA and/or MK substituting magnesium oxide (MgO) were prepared. The microstructure was explored by scanning electron microscopy, X-ray diffraction, and mercury intrusion porosimetry. The findings suggest that adding FA can delay the setting of MOSC; however, the effect of adding MK to MOSC was reversed. Furthermore, the phase composition of the MOSC hydration products was unaltered upon adding FA and/or MK, but thicker and longer 517 phase crystals were observed. FA and MK can effectively fill the large pores of MOSC through filling and nucleation effects, reduce the pore size, and form a denser microstructure, thereby improving its mechanical properties. The optimal MOSC sample was found by substituting 10 wt.% of both FA and MK, resulting in a cement that exhibited a short setting time and an incredibly high mechanical strength and density. These findings will further the development of stronger, more cost-efficient, and more water-resistant MOSC products.