Examining the transferability of habitat selection models is vital when they are used to forecast predictions in new times or places, but this issue is too often neglected. Nest boxes are often installed in agricultural landscapes to attract barn owls (Tyto spp.) and the ecosystem services they provide. For this practice to be effective, farmers need actionable guidelines for nest box design and placement to optimize nest box use. We addressed 3 primary objectives: 1) develop a nest box selection model in the well‐studied region of Napa Valley, California, USA, 2) evaluate this model's predictive performance in other regions of California, and 3) use data from all regions to build a more generalizable model. Based on data from 6 years of monitoring used and unused American barn owl (Tyto furcata) nest boxes in Napa Valley, we found that nest box selection was best predicted by nest box attributes (e.g., pole height, box height, and entrance orientation), local land cover (e.g., grassland within 75 m), and landscape‐scale metrics (e.g., grassland within 2.81 km). This model's predictions were strongly correlated with observed nest box use in Napa, but the model performed poorly when used to predict nest box use in other regions that are ecologically similar (Sonoma County) or dissimilar (Fresno, Merced, and Madera counties). A model pooling data from all regions fit the data well and again showed effects of box, local, and landscape predictors. It was more generalizable than the Napa‐only model and lost little precision when applied with forecasting predictions to Napa in particular. Taken together, our results indicate that local data should be used to make the most reliable predictions of nest box use. Until those data are available, general recommendations should be made from models that pool data from as many regions as feasible and should provide appropriate caveats. Results of this work can inform nest box design and placement for the benefit of farmers and owls in California, and future research should examine nest box selection by barn owls in other areas of the world with different climates and local habitats.