Cassava (Manihot esculenta Crantz) is mainly cultivated in marginal land in the south of China where seasonal drought stress occurs frequently and the soil becomes more compact year by year. The study aimed to explore the effect of Fenlong tillage (FLT) combined with nitrogen applications on cassava rhizosphere soil particle composition and fungal community diversity. Conventional tillage (CT) was set as the control. The results indicated that the contents of clay and silt of the cassava rhizosphere soil were influenced by the tillage method, nitrogen (N), and their interaction. There was no difference in the richness and diversity of rhizosphere soil fungal communities among all treatments in 2019, while the richness of FLT was lower than that of CT in 2020. FLT caused a stronger influence on the community structure of rhizosphere fungi than N applications in the first year. The differences in the community structure of all treatments were reduced by continuous cropping of cassava in the second year. The top 10 dominant rhizosphere fungi at the class level of cassava found in 2019 and 2020 were Sordariomycetes, Dothideomycetes, Eurotiomycetes, Agaricomycetes, Intramacronucleata, norank_p__Mucoromycota, unclassified_p__Ascomycota, unclassified_k__Fungi, Pezizomycetes, and Glomeromycetes, which had an important relationship with soil pH, activity of urease, available nitrogen, available phosphorus, organic matter, and clay. These results indicated that FLT created a better soil environment for cassava growth than CT, thus promoting the formation of more stable rhizosphere fungal community structures.