Soil heavy metal pollution has become one of the major environmental issues of global concern and solving this problem is a major scientific and technological need for today’s socio-economic development. Environmentally friendly bioremediation methods are currently the most commonly used for soil heavy metal pollution remediation. Via controlled experiments, the removal characteristics of chromium from contaminated soil were studied using earthworms (Eisenia fetida and Pheretima guillelmi) and plants (ryegrass and maize) at different chromium concentrations (15 mg/kg and 50 mg/kg) in acidic and alkaline soils. The effects of chromium contamination on biomass, chromium bioaccumulation, and earthworm gut microbial communities were also analyzed. The results showed that E. fetida had a relatively stronger ability to remove chromium from acidic and alkaline soil than P. guillelmi, and ryegrass had a significantly better ability to remove chromium from acidic and alkaline soil than maize. The combined use of E. fetida and ryegrass showed the best effect of removing chromium from contaminated soils, wih the highest removal rate (63.23%) in acidic soil at low Cr concentrations. After soil ingestion by earthworms, the content of stable chromium (residual and oxidizable forms) in the soil decreased significantly, while the content of active chromium (acid-extractable and reducible forms) increased significantly, thus promoting the enrichment of chromium in plants. The diversity in gut bacterial communities in earthworms decreased significantly following the ingestion of chromium-polluted soil, and their composition differences were significantly correlated with soil acidity and alkalinity. Bacillales, Chryseobacterium, and Citrobacter may have strong abilities to resist chromium and enhance chromium activity in acidic and alkaline soils. There was also a significant correlation between changes in enzyme activity in earthworms and their gut bacterial communities. The bacterial communities, including Pseudomonas and Verminephrobacter, were closely related to the bioavailability of chromium in soil and the degree of chromium stress in earthworms. This study provides insights into the differences in bioremediation for chromium-contaminated soils with different properties and its biological responses.