Urban infrastructure systems play essential roles in the smooth functioning of modern society but are also threatened by seismic hazards in the earthquake‐prone areas. Retrofitting critical components of those systems has been considered as the most frequently used mitigation strategy in both the literature and practice. The seismic retrofit budget is usually limited, then it needs to identify a set of critical components to be retrofitted, which is generally formulated as a seismic retrofit optimization problem. This article proposes a multi‐perspective modeling and solution framework for the seismic retrofit optimization of urban infrastructure systems, which allows choosing different performance measures including vulnerability, resilience loss and economic loss as the objective function. The proposed framework can be used to explore how different performance measures and the infrastructure interdependencies affect the seismic retrofit decision. Taking the interdependent Shelby power and gas systems as an example, results show that if considering single systems, the optimal economic loss‐based performance improvement ratio (PIR) is larger than the best resilience loss‐based PIR, which is larger than the vulnerability‐based PIR; if considering interdependent systems, the interdependency intensity is indeed a key factor affecting the retrofit decision.