Drought, excessive use of nitrogen fertilizer, and decline in soil organic matter threaten corn production. This study investigated the potential of water retaining agents, inhibitors, and corn stalks in enhancing soil physicochemical properties to bolster corn yield in semi-arid farmlands. In our study, polyacrylamide addition increased the content of ammonium nitrogen (NH4+-N) and nitrate nitrogen (NO3–-N) in the seedling stage, exchangeable potassium (K) in the mature stage but decreased the content of available phosphorus (P) in the seedling stage. Potassium polyacrylate addition increased the content of NH4+-N and decreased the content of available P in the seedling stage. The addition of inhibitors decreased the content of NH4+-N and available P in the seedling stage, NO3–-N and available P in the jointing stage, and NH4+-N in the mature stage, respectively. Corn stalks returning could maintain soil moisture, decrease the content of NH4+-N in the seedling stage and exchangeable K in the mature stage, and increase the content of available P and exchangeable K in the seedling stage. Combined application of inhibitors and corn stalks could increase soil organic carbon (SOC) and ensure corn yield, which was the best fertilization mode in semi-arid cropland.