The uneven distribution of medical and health resources leads to changes in the choice of patients for medical treatment, which is the key to restrict the reform of medical services in China currently. Taking service accessibility and residents' cognition as the starting point, this study utilized the data from the questionnaire and applied logistic regression and mediation test. By taking service accessibility as an explanatory variable and residents' cognition as an intermediary variable, the study examined the differences between residents' choice of medical treatment at the primary and non-primary levels. Thus, the influencing factors of residents' choice of medical treatment at the primary level were explored. The research statistics came from questionnaires of 1,589 residents in Nanjing, Jiangsu Province, China. The results showed that service accessibility and residents' cognition were significantly correlated with the residents' choice of primary medical treatment. Household registration, age, the signing situation with family doctors, hospital service fees, and distance to the hospital were positively related to residents' choice of primary medical treatment; while the reputation, scale, residents' income, and the reimbursement ratio of residents' medical insurance were negatively correlated with the choice. In addition, residents' cognition played an intermediary effect between service accessibility and the residents' choice of primary medical treatment. The signing situation with family doctors indirectly affected the choice of primary medical treatment through residents' cognition, and residents' cognition masked some negative influence of the reimbursement ratio of residents' medical insurance on the choice of primary medical treatment.