Rhizosphere and root-associated microbial communities are known to be related to soil-borne disease and plant health. In the present study, the microbial communities in rhizosphere soils and roots of both healthy and diseased Panax notoginseng were analyzed by high-throughput sequencing of 16S rRNA for bacteria and 18S rRNA internal transcribed spacer for fungi, to reveal the relationship of microbial community structure with plant health status. In total, 5593 bacterial operational taxonomic units (OTUs) and 963 fungal OTUs were identified in rhizosphere soils, while 1794 bacterial and 314 fungal OTUs were identified from root samples respectively. Principal coordinate analysis separated the microbial communities both in the rhizosphere soils and roots of diseased P. notoginseng from healthy plants. Compared to those of healthy P. notoginseng, microbial communities in rhizosphere soils and roots of diseased plants showed a decrease in alpha diversity. By contrast, bacterial community dissimilarity increased and fungal community dissimilarity decreased in rhizosphere soils of diseased plants, while both bacterial and fungal community dissimilarity in roots showed no significant difference between healthy and diseased plants. Redundancy analysis at the phylum level showed that mycorrhizal colonization and soil texture significantly affected microbial community composition in rhizosphere soils, whereas shoot nutrition status had a significant effect on microbial community composition in root samples. Our study provided strong evidence for the hypothesis that microbial diversity could potentially serve as an indicator for disease outbreak of medicinal plants, and supported the ecological significance of microbial communities in maintaining plant healthy and soil fertility.