With the help of spatial calculation and numerical analysis, this paper reveals the spatial distribution of rehabilitation infrastructures and the accessibility difference in the city Xiamen. The calculation of Moran’s I, Z value, and P value are, respectively, −0.229787, −0.122751, and 0.902304 for health facilities and −0.159235, 0.186166, and 0.852315 for fitness facilities. Such calculation results indicate an uneven pattern and varied accessibility numerical values. The reasons for this influence are diverse, but it is worth noting that the two-step floating catchment area (2SFCA) calculation shows that compared to fitness facilities, the standard deviation of accessibility numerical value of health facilities decreased significantly, which indicates that the accessibility of health service facilities is becoming better, while the change of accessibility of fitness facilities is not obvious. It is pointed out this is due to society’s insufficient attention and education on daily health. For better rehabilitation under the pandemic, the importance of fitness facilities should be noticed.