Tall buildings are an important part of a city, and their earthquake-induced damage or collapse will lead to heavy losses, extended repair time, and casualties. Therefore, it is essential to quantify and improve the resilience of tall buildings. To this end, this paper develops a component damage-based metric to characterize tall buildings' functionality loss and then proposes a general quantitative evaluation process to evaluate tall buildings' resilience. Next, the evaluation process is applied to a 42-story reinforced concrete frame shear wall building to demonstrate its applicability. Finally, retrofit strategies on nonstructural components are discussed to enhance the building's resilience. It can be concluded that the proposed metric can be effectively used to evaluate tall buildings' functionality loss. The building being studied has great seismic resilience, with resilience values of 99.95%, 98.68%, and 88.69% at service level earthquake (SLE), design level earthquake (DBE), and maximum considered earthquake (MCE), respectively. The influence of nonstructural components on seismic resilience is greater than that of structural components at SLE and DBE levels. It is an effective alternative to enhance the seismic resilience of tall buildings under SLE and DBE by improving the performance of partition walls, ceilings, and equipment.