The study aims to examine the impact of the built environment and weather conditions on travel time for bus passengers in Weinan, China. Various sources of data, including smart card data, bus GPS data, bus station data, road information data, and smart card swiping time, were integrated and analyzed. The study employed the light gradient boosting machine (LightGBM) model and SHapley Additive exPlanations (SHAP) value to assess the feature importance and nonlinear effects of different types of POI density, weather conditions, and time series on bus passengers’ travel time. The study findings indicate that several factors are associated with bus passengers’ travel time, including destination residential density, destination diversity, destination life service density, origin science and education density, origin residential density, origin diversity, humidity, visibility, boarding time between 7 and 8 a.m., and precipitation. This study also reveals nonlinear threshold effects. The study findings provide valuable insights that can be utilized to optimize the bus network and develop low-carbon-oriented land-use planning.