The role of senior citizen centers is becoming more important, with a greater emphasis placed on ensuring that these establishments facilitate leisure and communication for older adults. These developments are taking place as population aging has now become an irreversible global trend. However, there is still a lack of systematic research on predicting the distribution density of senior citizen centers based on physical factors in urban planning. Therefore, this study set each administrative district in Busan as a unit and investigated physical factors affecting the distribution density of senior citizen centers around small apartment complexes to validate their effectiveness. First, the spatial hierarchy of each administrative district in Busan was examined. The city was divided into administrative districts while focusing on the distribution density of senior citizen centers (the dependent variable) around small apartment complexes where older adults live (within a 500 m radius). The spatial accessibility of senior citizen centers and the number of apartments in each administrative district were set as independent variables. This selection was made to verify the effectiveness of the physical factors by conducting an independent sample t-test, normality test, Friedman test, and two-way ANOVA. The chief findings of this study are as follows. (1) The spatial awareness of each administrative district was low, and there were large disparities in the land development density relative to the spatial scale of the administrative districts. (2) Regarding the physical factors affecting the distribution density of senior citizen centers, the spatial accessibility of senior residences was more significant than the number of small apartment complexes nearby. (3) Personal and social factors may have indirectly influenced the distribution density of senior citizen centers more than physical factors, depending on the type of house in which the older adults live. The findings will provide a theoretical basis for determining the location and distribution density of senior citizen centers in urban planning considering physical factors, as well as serve as a reference for public policy decisions related to the allocation of such centers in the future.