Objective: To explore the correlation between different types of vegetable and fruit intake and cognitive function among the older adults in Chongqing, China, and to provide a scientific basis for developing efficient lifestyle interventions for the prevention of Mild Cognitive Impairment (MCI). Method: Approximately 728 older adults in urban and rural areas of Chongqing were surveyed using face-to-face questionnaires. Cognitive function was assessed with the Montreal Cognitive Assessment-Basic (MoCA-B) scale, and the vegetable and fruit intake groups were investigated with the Simple Food Frequency Counting Survey Scale. Binary logistic regression was used to explore the effect of the vegetable and fruit intake group on cognitive function. Subgroup analysis was used to demonstrate the robustness of the results. Result: Of the 728 participants in the study, 36.40% were likely to have MCI, which is higher than the national average for this condition. After adjusting for confounders, compared to the Q1 group, fruit and root vegetable intake was a protective factor for MCI, showing a dose–response relationship (p < 0.05). Only lower intake (Q2) of total vegetables, medium intake (Q2, Q3) of solanaceous vegetables, and medium–high intake (Q2, Q4) of fungi and algae was protective against MCI, whereas the leafy vegetables showed no relation to MCI. Apart from this, participants who were older, female, unmarried, non-smoking, and engaged in physical labor, and who had an average monthly income of less than 3000 RMB were more likely to suffer from cognitive impairment. Conclusion: This suggested that the fruit-intake groups and some vegetable-intake groups showed a protective effect on cognitive function, and might behave differently depending on their different intake and demographic characteristics. A sensible, healthy diet can help prevent MCI.