Various intrusion-tolerant certification authority (CA) systems have been proposed to provide attack resilient certificate signing (or update) services. However, it is difficult to compare them against each other directly, due to the diversity in system organizations, threshold signature schemes, protocols and usage scenarios. We present a framework for intrusion-tolerant CA system evaluation, which consists of three components, namely, an intrusion-tolerant CA model, a threat model and a metric for comparative evaluation. The evaluation framework covers system organizations, protocols, usage scenarios, the period of certificate validity, the revocation rate and the mean time to recovery. Based on the framework, four representative systems are evaluated and compared in three typical usage scenarios, producing reasonable and insightful results. The interdependence between usage scenarios and system characteristics is investigated, providing a guideline to design better systems for different usage scenarios. The proposed framework provides an effective and practicable method to evaluate intrusion-tolerant CA systems quantitatively, and helps customers to choose and configure an intrusion-tolerant CA system. Moreover, the comparison results offer valuable insights to further improve the attack resilience of intrusion-tolerant CA systems.To validate the framework, an in-depth comparative evaluation of four representative intrusion-tolerant CA systems (ITTC, 53 ARECA, 25 COCA 57 and MOCA 56 ), is conducted. After analyzing these systems, we evaluate them in three typical usage scenarios (LANs, the Internet and ad hoc networks), and study the evaluation results with small variations of the usage scenario parameters. The interdependence between usage scenarios and system characteristics is also studied based on the evaluation results. Then, several conclusions and suggestions for designing better systems are presented. The analysis can be applied to facilitate designing a new system to obtain optimal performance in a certain usage scenario. Note that in this paper, performance means attack resilience.Although COCA provides two types of certificate services, that is, update and query, only the certificate update service is compared with other CA systems in our evaluation framework because a commodity CA typically does not support certificate query. To support attack resilient certificate query, the COCA design is not solely driven by maximizing the resilience of certificate update services. As a result, the comparisons are not meant to judge the overall design of COCA. Instead, the framework compares only the certificate update service of COCA with other genuine certificate update solutions. To make this point clear, in the rest of this paper we denote COCA as COCA CUS (certificate update service). of the Internet P C , P A and P D are estimated based on statistics on the Internet. In 2003, 202,369,889 hosts (denoted as N h ) were advertised in Internet domain survey (DNS) averagely, 22 and 137,529 incidents (denoted as ...