The interpretation of settlement behavior in shrinking areas can provide insights into sustainability strategies in shrinking communities. However, the choice to settle in areas undergoing shrinkage is hard to interpret when considering residents as rational decisionmakers. To attain a deeper understanding of this decision-making process, a framework of residential decision making (RDM) considering a subjective environmental evaluation dimension, psychological dimension, and cognitive dimension is proposed. This process was further validated by conducting a questionnaire survey in Japanese communities. The results of the structural equation modeling reveal that the RDM framework proposed is applicable to RDM in shrinking communities. By considering geographical differences, we further found that residents in suburban communities tend to consider overall satisfaction with their location when deciding whether to stay, whereas residents in mountainous communities value emotional satisfaction factors such as place attachment when considering continuous residence. Different residential preferences contributing to the formation of RDM factors were also revealed between communities. The results of this study imply that sustainable development strategies to assist shrinking communities should be tailored to their geographical characteristics. Further, a regional design that can enrich residential experiences and neighborhood communication is important for promoting population settlement.