The management of seismic risk is an important aspect of social development. However, urbanization has led to an increase in disaster-bearing bodies, making it more difficult to reduce seismic risk. To understand the changes in seismic risk associated with urbanization and then adjust the risk management strategy, remote-sensing technology is necessary. By identifying the types of earthquake-bearing bodies, it is possible to estimate the seismic risk and then determine the changes. For this purpose, this study proposes a set of algorithms that combine deep-learning models with object-oriented image classification and extract building information using multisource remote sensing data. Following this, the area of the building is estimated, the vulnerability is determined, and, lastly, the economic and social impacts of an earthquake are determined based on the corresponding ground motion level and fragility function. Our study contributes to the understanding of changes in seismic risk caused by urbanization processes and offers a practical reference for updating seismic risk management, as well as a methodological framework to evaluate the effectiveness of seismic policies. Experimental results indicate that the proposed model is capable of effectively capturing buildings’ information. Through verification, the overall accuracy of the classification of vulnerability types reaches 86.77%. Furthermore, this study calculates social and economic losses of the core area of Tianjin Baodi District in 2011, 2012, 2014, 2016, 2018, 2020, and 2021, obtaining changes in seismic risk in the study area. The result shows that for rare earthquakes at night, although the death rate decreased from 2.29% to 0.66%, the possible death toll seems unchanged, due to the increase in population.