Numerous research efforts have been conducted to simulate crowd movements, while relatively few of them are specifically focused on multi-hazard situations. In this paper, we propose a novel crowd simulation method by modeling the generation and contagion of panic emotion under multi-hazard circumstances. In order to depict the effect from hazards and other agents to crowd movement, we first classify hazards into different types (transient and persistent, concurrent and nonconcurrent, static and dynamic) based on their inherent characteristics. Second, we introduce the concept of perilous field for each hazard and further transform the critical level of the field to its invoked-panic emotion. After that, we propose an emotional contagion model to simulate the evolving process of panic emotion caused by multiple hazards. Finally, we introduce an Emotional Reciprocal Velocity Obstacles (ERVO) model to simulate the crowd behaviors by augmenting the traditional RVO model with emotional contagion, which for the first time combines the emotional impact and local avoidance together. Our experiment results demonstrate that the overall approach is robust, can better generate realistic crowds and the panic emotion dynamics in a crowd. Furthermore, it is recommended that our method can be applied to various complex multi-hazard environments.