This study aimed to test a model of factors associated with resilience and physical activity post-hip fracture and compare model fit between men and women. We used data from the seventh Baltimore Hip Study that included 339 participants. Model testing indicated that health status (men: β = .237, p = .002; women: β = .265, p = <.001), depression (men: β = −.245, p = .001; women: β = −.241, p = <.001), and optimism (men: β = .320, p = <.001; women: β = .282, p = <.001) were associated with resilience in men and women, but resilience was only associated with physical activity in men (β = .203, p = .038) and not in women. Social interaction was related to physical activity only among women (β = .206, p = .044). This study provides support for the relationship between resilience and physical activity at least among men.