Oil palm (Elaeis guineensis Jacq.), an industrial oilseed crop, is the most prominent oil-producing crop in the world and one of the highest dry matter producers among C3 plants. It removes large amounts of soil nutrients to support its substantial biomass and oil production. Thus, maintaining adequate levels of limiting nutrients at favorable ratios in the soil is crucial to sustain the required palm nutritional status and oil production over extended periods. We evaluated the levels and ratios of the available soil and leaf nutrients in high-yielding (HYOPP) and low-yielding (LYOPP) oil palm plantations from two locations in southern India and assessed their relations to develop effective nutrient management for sustaining fresh fruit bunch (FFB) production. The FFB yield varied from 2.00 to 31.0 t ha−1 (location 1) and 6.84 to 30.9 t ha−1 (location 2), and the FFB yields of the high- and low-yield population at each location differed significantly. Wide ranges of soil properties, available nutrients, and leaf nutrients concentrations were observed at 0–20- and 20–40-cm soil depths at both locations. The soil nutrient ratios at both sampling depths of the HYOPP were significantly different than the LYOPP at location 1. The mean leaf N (2.09%) and K (0.68%) concentrations at location 2 were significantly higher than those in location 1 (1.85% and 0.54, respectively). The mean leaf concentrations at both locations followed the order N (1.85%, 2.09%) > Ca (1.65%, 1.36%) > K (0.54%, 0.68%) > Mg (0.49%, 0.47%) > S (0.16%, 0.18) > P (0.11%, 0.12%) > B (35.8 mg kg−1, 27.5 mg kg−1). We observed positive and significant correlations between the available N and P at 0–20 cm and between the available N and K as well as Ca and S at both soil depths for HYOPPs. In LYOPPs, the available N was significantly correlated with the available P, K, Ca, and Mg at both soil depths and with the available S at 0–20 cm. The FFB yield of LYOPPs was positively and significantly correlated with the available P and S at the 0–20-cm soil depth and with the available N and P at the 20–40-cm soil depth. Except for a negative correlation (p < 0.05) with exchangeable Ca at the 20–40-cm soil depth, the FFB yield in HYOPPs had no significant correlation with the available soil nutrients at both depths. The FFB yield of LYOPPs had a positive significant (p < 0.01) correlation with the leaf P and K concentrations, while the yield of HYOPPs had a negative correlation (p < 0.01) with the leaf P. Fertilizer accounts for a large part of the on-farm cost in oil palm plantations, and an evidenced approach for fertilizer management based on nutrient contents and ratios in the soil and in the plant is useful for sustainable and profitable production of palm oil.