Battery safety is a critical factor in the design of electrified vehicles. As such, understanding the battery responses under extreme conditions have gained a lot of interest. Previously, abuse tolerance tests were applied to measure the safety metrics of different types of batteries. Nevertheless, conducting these tests in various conditions is usually expensive and time consuming. Computational modeling, on the other hand, provides an efficient and cost-effective tool to evaluate battery performance during abuse, and therefore has been widely used in optimizing the battery system design. In this Perspective, we discuss the main progresses and challenges in battery safety modeling. In particular, we divide the battery safety models into two groups according to the stage in a typical battery failure process. The first group focuses on predicting the failure conditions of batteries in different scenarios, while the second one aims to evaluate the hazard after the onset of battery failure like thermal runaway. Although the models in these groups serve different purposes, they are intercorrelated and their combination provides a better understanding of the failure process of a battery system. The framework, capabilities, and limitations of typical models in each group are presented here. The main challenges in building battery safety models and their future development and applications are also discussed.