Phosphorus (P), a plant macronutrient, must be adequately supplied for crop growth. In Germany, many soils are high in plant-available P; specifically in arable farming, P fertilizer application has been reduced or even omitted in the last decade. Therefore, it is important to understand how long these soils can support sustainable crop production, and what concentrations of soil P are required for it. We analyzed a 36-year long-term field experiment regarding the effects of different P application and liming rates on plant growth and soil P concentrations with a crop rotation of sugar beet, wheat, and barley. Sugar beet reacted to low soil P and low soil pH levels more sensitively than wheat, which was not significantly affected by the long-term omitted P application. All three crop species showed adequate growth at soil P levels lower than the currently recommended levels, if low soil pH was optimized by liming. The increase in efficacy of soil and fertilizer P by reduced P application rates therefore requires the adaptation of the soil pH to a soil type-specific optimal level.