Mass mortality of fagaceous trees caused by Japanese oak wilt has occurred widely in Japan. Although virulence of the causal fungus, Raffaelea quercivora, appeared to differ among isolates, its relation to the fungal growth within trees was unknown. To clarify the differences in fungal virulence against susceptible Quercus crispula, we examined fungal growth of four R. quercivora isolates within trees and the resulting virulence. In our study, the isolates were multiple-inoculated in seedlings and single-inoculated in twigs of mature trees. In the multiple-inoculation test, mortality rates were examined by the observation of external symptoms. In the single-inoculation test, water conductance and hyphal growth within the trees were examined by applying aqueous dyes and fluorescence microscopy, respectively. Mortality rates, the proportion of the cross-sectional area comprising non-conductive sapwood and horizontal hyphal growth differed significantly among the isolates. Univariate logistic regression analyses showed that both the proportion of non-conductive sapwood and hyphal growth were significantly positively related to mortality rates. For three isolates, hyphal growth was significantly positively correlated with the proportion of non-conductive sapwood. These results suggested that the virulence against Q. crispula varies among R. quercivora isolates and that the extent of fungal colonization of the tree determines fungal virulence.