Despite the high prevalence of cognitive impairment among older adults, little is known about the association of the selection of dexmedetomidine and propofol on cognitive functions of patients after a critical illness. Patients aged ≥70 years who received intensive care unit (ICU) care from Cangzhou Central Hospital between 2013 and 2016 were enrolled and randomized into a dexmedetomidine group and a propofol group with matched demographic and clinical characteristics. At discharge from the ICU and 4 weeks later, the cognitive status of patients was assessed and compared using the Montreal Cognitive Assessment system. There were 164 patients included in the dexmedetomidine group and 159 patients in the propofol group. No significant difference was observed between the 2 groups in terms of age, female sex, body weight, educational level, ICU and hospital stay, comorbidities, and medications. Further, patients from the 2 groups at ICU discharge did not demonstrate significant difference on the Montreal Cognitive Assessment component scores, which showed significant differences between the 2 groups 4 weeks later (P < .05). Moreover, dexmedetomidine and propofol showed different levels of impacts on the cognitive function of patients discharged from the postanesthesia care unit, neurological ICU, and medical ICU. This study demonstrated that patients discharged from the ICU who received propofol for sedation showed less impairment on the cognitive functions when compared with patients who received dexmedetomidine during ICU care 4 weeks after discharge. Despite some limitations, this study provides insights to the decision‐making process in the selection of appropriate sedation strategy, especially for the elderly patients.