The gallbladder (GB) microbiota plays critical roles in mammalian metabolism and immune homeostasis, and its relationship with human disease has been extensively studied over the past decade. However, very little is known about the interplay between GB microbiota and the immune functions of teleost fish, the earliest bony vertebrate with a GB. Therefore, this study sought to investigate the composition of the teleost GB microbiota and the potential mechanisms through which it affects mucosal immunity. In our results, we found that the GB mucosa (GM) and bile bacterial community shared a similar microbiological composition with that of the gut mucosa in naïve individuals. IHNV infection induced a profound GB inflammation and disrupted their microbial homeostasis followed by a strong anti-bacterial response. Interestingly, beneficial bacteria from the Lactobacillales order showed a significant increase in the abundance of the bile microbial community, whereas the structure of the Mycoplasmatales order in the gut microbial community was markedly changed. All in all, our study characterized the structure of the GB microbial ecosystem in teleost fish, and the fish GB microbiome shared a high similarity with the gut microbiota. More importantly, our findings offer solid evidence that the teleost GB evolved immune functions to preserve its mucosal microbial homeostasis, suggesting that both the microbiota and mucosal immunity of the GB might have co-evolved in early vertebrates.