This paper presents the seismic fragility assessment of the Smithsonian Institute Museum Support Center (MSC), which sustained appreciable damage during the 2011 Virginia Earthquake. A three-dimensional (3D) finite element model (FEM) for the building was created and validated using measured dynamic characteristics determined from field vibration test data. Two suites of bi-directional ground motions at different hazard levels were applied to the FEM to generate fragility curves for structural as well as nonstructural (storage cabinets) damage. The effect of brace yielding strength on structural and nonstructural damage is also investigated to provide recommendations for future retrofit. The fragility curves show that the spectral acceleration to cause structural damage to the building is not high. Due to low seismicity, however, the probability for the structure to be damaged at the design basis earthquake is small. Nevertheless, the probability for non-structural damage is considerable, which is an important issue related to the seismic performance of the building.