Organic dryland wheat (Triticum aestivum L.) is an important crop in Utah and the intermountain western United States; however, growers face numerous challenges maintaining yield, grain quality, soil fertility, and health. Wheat yields in the West are severely constrained by lack of rainfall, so dryland organic wheat farmers generally do not grow cover crops due to the risk of subsequent crop failure or apply soil amendments due to the inability to recoup application costs in the short term. Compost may have significant non‐nutritive as well as nutritive effects on soil processes, however, potentially changing calculations of perceived costs vs. benefits. The goal of this study was to quantify the effects of a one‐time compost application on soil health, yield, and crop quality in an organic dryland wheat–fallow system in northern Utah. Four rates of compost, applied once, (0, 12.5, 25, and 50 Mg ha−1 dry weight, DW) were compared to anhydrous ammonia (50 kg N ha−1) applied at planting. Mineralizable C, microbial biomass, and phosphatase enzyme activity increased significantly at all compost rates, while available soil P increased at the 25 and 50 Mg ha−1 rates and total soil N at the 50 Mg ha rate−1. Soil moisture was generally no different between treatments. Wheat yield was significantly greater 3 yr after application at the 50 Mg ha−1 compost rate only, with no response to conventional fertilizer. A lack of yield response to conventional fertilizer suggests that non‐nutritive benefits of improved soil health were responsible for improved yields at the high compost rate.