Understanding the relationship between the built environment and metro ridership has become essential for advancing sustainable transportation development. Limited research has been given to how built environment factors influence metro ridership at a station-to-station level. Moreover, most studies focus on the general population, overlooking the special groups. This study examines the influence of the built environment on metro origin–destination (OD) ridership for older adults. Specially, we employ the CatBoost model, along with SHAP interpretation, to assess feature importance and capture nonlinear effects. Taking Xi’an as a case study, the results show that: (1) The CatBoost model demonstrates superior fitting and predictive performance, outperforming both the XGBoost and Logistic Regression models. (2) There are distinct variations in the influence of built environment factors at origin and destination stations. Traffic-related variables have a stronger effect at origin stations, while land-use variables exert a more significant influence at destination stations. (3) The built environment’s impact on older adults’ metro ridership exhibits a clear nonlinear relationship. Notably, an optimal land-use mix of 1.8–1.9 and a job density of 5000–7000 significantly enhance ridership. These findings provide valuable insights into how the built environment shapes older adults’ metro travel, contributing to the sustainable urban development.