As the population ages globally, there seem to be more people with Alzheimer’s disease. Unfortunately, there is currently no specific treatment for the disease. At present, Huperzine A (HupA) is one of the best drugs used for the treatment of Alzheimer’s disease and has been used in clinical trials for several years in China. HupA was first separated from Huperzia serrata, a traditional medicinal herb that is used to cure fever, contusions, strains, hematuria, schizophrenia, and snakebite for several hundreds of years in China, and has been confirmed to have acetylcholinesterase inhibitory activity. With the very slow growth of H. serrata, resources are becoming too scarce to meet the need for clinical treatment. Some endophytic fungal strains that produce HupA were isolated from H. serrate in previous studies. In this article, the diversity of the endophytic fungal community within H. serrata was observed and the relevance to the production of HupA by the host plant was further analyzed. A total of 1167 strains were obtained from the leaves of H. serrata followed by the stems (1045) and roots (824). The richness as well as diversity of endophytic fungi within the leaf and stem were higher than in the root. The endophytic fungal community was similar within stems as well as in leaves at all taxonomic levels. The 11 genera (Derxomyces, Lophiostoma, Cyphellophora, Devriesia, Serendipita, Kurtzmanomyces, Mycosphaerella, Conoideocrella, Brevicellicium, Piskurozyma, and Trichomerium) were positively correlated with HupA content. The correlation index of Derxomyces with HupA contents displayed the highest value (CI = 0.92), whereas Trichomerium showed the lowest value (CI = 0.02). Through electrospray ionization mass spectrometry (ESI-MS), it was confirmed that the HS7-1 strain could produce HupA and the total alkaloid concentration was 3.7 ug/g. This study will enable us to screen and isolate the strain that can produce HupA and to figure out the correlation between endophytic fungal diversity with HupA content in different plant organs. This can provide new insights into the screening of strains that can produce HupA more effectively.