Lycium barbarum L., goji berry, is a precious traditional Chinese medicine and it is homology of medicine and food. Its growth is heavily dependent on nitrogen. The use of chemical fertilizers has significantly promoted the yield of goji berry and the development of the L. barbarum L. industry. However, crop plants are inefficient in the acquisition and utilization of applied nitrogen, it often leads to excessive application of nitrogen fertilizers by producers, which cause negatively impact to the environment ultimately. The exploration of an interaction model which deals with crops, chemical fertilizers, and rhizosphere microbes to improve nitrogen use efficiency, is, therefore, an important research objective to achieve sustainable development of agriculture greatly. In our study, we explored the effects of nitrogen input on soil microbial community structure, soil nitrogen cycling, and the contents of nutrients in L. barbarum fruits. The structure and composition of the bacterial community in the rhizosphere soil of L. barbarum were significantly different under different nitrogen supply conditions, and high nitrogen addition inhibited the diversity and stability of bacterial communities. Low nitrogen input stimulated the relative abundance of ammonia-oxidizing bacteria (AOB), such as Nitrosospira, catalyzing the first step of the ammonia oxidation process. The results of the GLMM model showed that the level of nitrogen fertilizer (urea) input, the relative abundance of AOB, the relative abundance of Bradyrhizobium, and their combinations had significant effects on the soil nitrogen cycling and contents of nutrients in L. barbarum fruits. Therefore, we believe that moderately reducing the use of urea and other nitrogen fertilizers is more conducive to improving soil nitrogen use efficiency and Goji berry fruit quality by increasing the nitrogen cycling potential of soil microorganisms.