The long-term use of chemical N fertilization may have a negative impact upon soil fertility and quality. On the contrary, organic fertilization is considered a sustainable development agricultural strategy. However, the remediation effect of organic fertilization on agroecosystems remains unclear. This study was conducted in a long-term (1979–2020) field experiment to investigate the influence of organic and chemical fertilizers on the soil microbiome assembly processes. The experiment consisted of six treatments: chemical N fertilization (N), double N fertilization (N2), organic fertilization (M), organic and N fertilization (MN), double organic and N fertilization (M2N2), and unfertilized control. The chemical N fertilization (N and N2) treatments significantly decreased soil microbial diversity, as well as soil pH, compared to the Control treatments (p < 0.05). MN and M2N2 treatments increased microbial diversity compared to that of N and N2 treatments. The combination of organic and chemical N fertilizer recovered the decreased microbial diversity to the level of the Control and M treatments, but the application of double organic fertilizer (M2N2) still showed a significantly lower microbial diversity than that of the Control and M treatments. From the results of the microbial community assembly processes, it was found that environmental filtering was induced by N fertilization, while organic fertilization developed a stochastic process and mitigated the role of environmental filtering in the community assembly process. An ecological network analysis showed that the decrease in Acidobacteria in organic fertilization treatments played a key role in mitigating the soil acidification induced by 40-year chemical N fertilization. It indicated that organic fertilizer could mitigate the decrease in soil fertility induced by chemical N fertilization. Higher environmental filtering effects in M2N2 than those in MN treatments suggested that the mitigation effect of organic fertilizer was weakened when double chemical N fertilization was applied in black soils. These results are helpful for a unified understanding of the ecological processes for microbial communities in the development of sustainable agriculture.