This study analyzes the impacts of natural hazards on neighborhoods, focusing on their age and housing type diversity. It estimates how the diversity of neighborhoods having experienced large natural hazards since 2005 changed between 1995 and 2015, as compared to neighborhoods without such experiences. "Neighborhood" was defined as a census tract of the National Statistical Office, and longitudinal data analysis was used to clarify the differences in natural hazards' impacts according to the characteristics (damage intensity and financial independence) of the neighborhoods. The results of the analyses are as follows: First, age and housing type diversity decrease immediately in the aftermath of large natural hazards but tend to recover quickly. Second, the impacts differ in accordance with the neighborhood's characteristics. Age diversity in neighborhoods with severe damage tends to decrease sharply but increases rapidly during recovery. In neighborhoods with high levels of financial independence, age diversity tends to increase, while housing type diversity tends to decrease, and post-disaster growth rates tend to be reversed.