Bridges are critical transportation infrastructure in Taiwan that are at high risk of structural damage during major earthquake incidents. The structural health monitoring method currently used to assess bridge safety nationwide is labor-intensive and expensive to implement. In this study, a novel automated bridge safety assessment system was developed to facilitate post-earthquake bridge inspections using symbiotic organism search-transfer learning-PointRend (STF-PointRend) for component and damage type detection, EfficientNetB0 for damage level detection, and earthquake resistance index for safety assessment. In the case studies, the STF-PointRend model obtained good testing results, with global mean Intersection over Union values of 76.31 and 75.47% for bridge-component and damage-type detections. Furthermore, the EfficientNetB0 model obtained an average F1 score of 0.86 for damage-level detection in the testing results. The developed model was used to conduct a safety evaluation of two bridges (Lempenge I and Luk I) that had suffered damage in a 2018 earthquake in Palu, Indonesia. The earthquake-resistance scores of 56.15 and 53.21 respectively earned by the two bridges indicate that both require immediate maintenance.