Soil microbial communities are essential in the cycling of nutrients that affect crop production. Our goal was to characterize the microbial community structure following 34 years of nitrogen (N) fertilization treatments in continuous maize production in highly fertile soils. Using 16S rRNA gene-based analysis of the V4 region via Illumina HiSeq2500 technology with downstream bioinformatics processing and analysis with QIIME 2.0, we aimed to characterize the prokaryotic communities under three increasing N fertilization rates. Factor analyses indicated that a high N level decreased the diversity of soil bacterial and archaeal communities and altered the relative abundance (RA) of the dominant (>1% RA) and minor (<1% RA) phyla. Among the 12 major phyla, we determined increases in Gemmatimonadetes, Proteobacteria, and Euryarchaeota, accompanied by reductions in Cyanobacteria, Chloroflexi, Firmicutes, and Planctomycetes with increasing N. Within the 29 minor phyla, N fertilization led to increases in Aquificae, WPS2, Parvarchaeota, AD3, FCPU426, Armatimonadetes, TM7, Chlamydiae, and OD1, along with reductions of Nitrospirae, WS3, Tenericutes, Lentisphaerae, OP3, Synergistetes, Thermotogae, and prokaryotes that could not be reliably assigned to a phylum (classified as Other).