Crop removal of cations and anions is one of the main factors causing soil acidification. Mollisols, also known as black soils, which are inherently neutral soils, have been acidified due to irrational land use. However, the magnitude of acidification of black soils in relation to crop removal under crop rotation and field management remains unclear. In this study, we collect soil samples from 9- and 10-year long-term trials with seven treatments of soybean and corn monoculture, and corn–soybean rotation under different fertilization or straw return in the Mollisol region of Northeast China. Total removal of cations and anions is 371–508 and 92–125 kg ha−1 in corn but 166–250 and 56–82 kg ha−1 in soybean, which leads to the production of 7.06–10.2 kmol ha−1 H+ for corn and 4.43–5.77 kmol ha−1 H+ for soybean. Among the four cations (K+, Ca2+, Na+, and Mg2+) analyzed in different tissues of crops, K+ contributes more to the total H+ production, where K+ in corn stem contributes more than 55% H+, while K+ in soybean seed contributes more than 64% H+. The sum of Ca2+ and Mg2+ removal for soybean is 37.6–66.9 kg ha−1, 1.26–2.07 times (p < 0.05) greater than for corn. Net H+ produced by corn and soybean removal may potentially lead to a soil acidification of 0.053–0.074 and 0.032–0.045 year−1 units pH, respectively. The greater the corn or soybean biomass, the higher the net H+ production. Continuous soybean and combination of chemical fertilizer for corn but no chemical fertilizer for soybean reduce the soil pH buffering capacity (pHBC), while other treatments increase the soil pHBC, but there is no effect for continuous corn. Compared to normal corn–soybean rotation, soil pH in 9-year continuous corn declines from 5.76 to 5.63, while 10-year continuous soybean declines from 5.68 to 5.43 (p < 0.05). However, soil pH with chemical fertilizer for corn and dairy manure for soybean increases by 0.60 and 0.89 units (p < 0.05). Changes in pH might not be fully related to crop ion removal. Manure application with crop rotation is an effective approach to alleviate soil acidification.