Cognitive frailty, defined as the coexistence of mild cognitive impairment symptoms and physical frailty phenotype in older persons, is increasingly considered the main geriatric condition predisposing to dementia. Recent studies have demonstrated that gut microbiota may be involved in frailty physiopathology by promoting chronic inflammation and anabolic resistance. The contribution of gut microbiota to the development of cognitive impairment and dementia is less defined, even though the concept of “gut–brain axis” has been well demonstrated for other neuropsychiatric disorders. The aim of this systematic review was to summarize the current state-of-the-art literature on the gut microbiota alterations associated with cognitive frailty, mild cognitive impairment and dementia and elucidate the effects of pre- or probiotic administration on cognitive symptom modulation in animal models of aging and human beings. We identified 47 papers with original data (31 from animal studies and 16 from human studies) suitable for inclusion according to our aims. We concluded that several observational and intervention studies performed in animal models of dementia (mainly Alzheimer’s disease) support the concept of a gut–brain regulation of cognitive symptoms. Modulation of vagal activity and bacterial synthesis of substances active on host neural metabolism, inflammation and amyloid deposition are the main mechanisms involved in this physiopathologic link. Conversely, there is a substantial lack of human data, both from observational and intervention studies, preventing to formulate any clinical recommendation on this topic. Gut microbiota modulation of cognitive function represents, however, a promising area of research for identifying novel preventive and treatment strategies against dementia.