The soil–vegetation relationships are reciprocal and fundamental for terrestrial ecosystem integrity. However, the long-term degradation of vegetation cover alters edaphic conditions, which can lead to degradation of habitats, and obstructs proper ecosystem functioning. This study aimed to assess the effects of the degradation of perennial steppe vegetation due to overgrazing (i.e., continuous and free grazing) on the physicochemical characteristics and soil fertility in the steppe rangelands of Halfa (Macrochloa tenacissima syn. Stipa tenacissima) of semi-arid areas in Algeria. The edaphic parameters of the superficial horizons of degraded steppes and other undegraded steppes were compared in order to suggest adequate strategies for rangeland management and remedy the degradation of vegetation and thereby ensure the sustainability of these agro-pastoral systems. The soil, collected from the surface horizons (A1 and A2) of pedological profiles, was analyzed according to standard methods of soil physicochemical analyses to determine the particle size fractions (clay, silt, and sands), pH, electrical conductivity (EC), total and active CaCO3, organic matter (SOM), organic carbon, total nitrogen, and the C:N ratio. The distribution and variation of soil parameters between horizons and the two types of steppes were examined using generalized linear mixed models and redundancy analysis. Findings of this study revealed that the degradation of steppe vegetation cover by short-term overgrazing did not cause a significant variability in soil physicochemical parameters between degraded and undegraded steppes and the horizons A1 and A2. The sites studied have clayey–silty textures in non-degraded steppes and coarse textures dominated by sands in degraded steppes. The edaphic environment of the Halfa grass steppes was characterized by a calcareous substratum with high carbonate content (total CaCO3 = 36.8–41%, active CaCO3 = 17.5–18.5%), with an alkaline pH (8.09–8.19) and EC averaging 0.99 ± 1.24 dS/m. Soils had low SOM contents (1.42–2.93%), organic carbon (0.82–1.64%), and nitrogen (0.1–0.15%). The strong positive correlations recorded between the fine-size fractions, SOM, total CaCO3, nitrogen, and EC indicated good soil structural stability in undegraded steppes, while the negative correlations between these same variables with sand and pH substantiate the structural degradation of the soil. The negative correlation between C:N ratio and SOM indicated that the rate of SOM mineralization was rapid in semiarid steppes, with a higher fertility state in the undegraded steppe. Our findings suggest and urge to implement a restoration plan against the degradation of semi-arid Halfa steppes in order to avoid the irreversible state of soil and habitat destruction in the future because even the degradation of rangeland plant cover due to livestock overgrazing—in the short term—has not affected the edaphic characteristics of steppe rangelands.