The mucosa of vertebrates is a particularly complex but dynamic environment in which the host constantly interacts with trillions of commensal microorganisms and pathogens. Although the internal and external mucosal microbiomes with immune defense of mammals have been well investigated, the relationship between mucosal microbes and their host’s immune responses has not been systematically understood in the early vertebrates. In this study, we compared the composition and distribution of mucosal microbiota in common carp (Cyprinus carpio), and found that there were significant differences of microbiota between in the internal (gut) and external mucosal (buccal mucosa, gills and skin) tissues. Next, we successfully constructed an infection model with spring viremia of carp virus (SVCV). Specifically, following viral infection, the immune and antiviral related genes showed different up-regulation in all selected mucosal tissues while significant morphological changes were only found in external tissues including buccal mucosa, gills and skin. Using 16S rRNA gene sequence, we revealed that the abundance of Proteobacteria in mucosal tissues including buccal mucosa, gills and gut showed increased trend after viral infection, whereas the abundance of Fusobacteria significantly decreased in gut. In addition, the loss of dominant commensal microorganisms and increased colonization of opportunistic bacteria were discovered in the mucosal surfaces indicating that a secondary bacterial infection might occur in these mucosal tissues after viral infection. Overall, our results firstly point out the distribution of internal and external mucosal microbiota and analyze the changes of mucosal microbiota in common carp after SVCV infection, which may indicated that the potential role of mucosal microbiota in the antiviral process in early vertebrates.