We examined the population dynamics of three broad-leaved tree species with different susceptibilities to deer predation. Simulation analysis was conducted using a size-structured matrix model for a primary forest plot (PP) and a secondary forest plot (SP) with 56% and 12% evergreen conifer composition in the canopy, respectively. In both plots, populations of Neolitsea sericea, a species that is susceptible to deer predation, initially declined significantly but eventually leveled off. The number of small stems decreased, while that of larger stems increased, indicating that the population dynamics of N. sericea are strongly affected by browsing pressure and that the number of large trees is important for population maintenance. When we examined two deer-resistant species, Pieris japonica and Illicium anisatum, the population of P. japonica increased in the SP and decreased in the PP, whereas that of I. anisatum increased in both plots, likely because mortality tends to increase in persistently dark environments. No significant difference was observed between the present and predicted size distributions of resistant species in the PP. Competition for resources is expected to intensify in the SP as a result of the predicted increase in large stems of the resistant species I. anisatum. Therefore, a specific conservation and management strategy for tree species should be considered for each forest type under the influence of Sika deer.