In the financial market, information and investment behaviors disseminate in investor social networks, and different contagion patterns may cause diverse investment trends. Prior studies have investigated the impact of investor social networks, but few have considered community structure. In this paper, we study the impact of the community structure of investor social networks on the diffusion of internet investment products. A two-stage diffusion model is proposed, and the clustering coefficient and modularity of an investor social network are considered. The results show that both modularity and the clustering coefficient have an impact on the diffusion velocity and scale and that the impact is most evident at the stage of explosive growth. The negative influence of a large modularity can be hardly mitigated by adjusting other factors. Furthermore, a decrease in modularity and an increase in the clustering coefficient can better facilitate diffusion when the temporary investment rate is high and can partly offset the negative impact of information discarding and divestment.