Visual hallucinations (VHs) are among the most striking features of dementia with Lewy bodies (DLB). Given that Lewy body pathology is frequently observed in the brains of patients with AD, we aimed to study factors associated with VHs in AD and examine their association with DLB features. This cross-sectional study enrolled a consecutive series of AD patients who visited the dementia clinic of a regional hospital. Clinically diagnosed possible or probable DLB cases were excluded. VH frequency and associated factors including age, sex, education, disease severity, DLB features, vascular risk factors, cognitive function, and neuropsychiatric symptoms were compared between AD patients with VHs (VH+) and those without VHs (VH−). Among a total of 295 patients analyzed, 42 (14.2%) had VHs. After adjusting for age, sex, and disease severity, DLB features including fluctuations in cognition scores, rapid-eye-movement behavioral disorder (RBD), and severe neuroleptic sensitivity were more frequent in the VH+ group. Furthermore, depression score, total Neuropsychiatric Inventory (NPI) score, and total caregiver burden score as assessed by the NPI were higher in the VH+ group. Among neuropsychiatric symptoms, delusions, hallucinations in the non-visual domains, anxiety, and disinhibition were more frequent in the VH+ group. Conversely, none of the vascular risk factors (VRFs) or cognitive domains of the Cognitive Abilities Screening Instrument (CASI) was associated with VHs in AD. In summary, VHs, albeit occurring at a low rate, had a high impact on AD. Diagnostic features of DLB, including fluctuations, RBD, and severe neuroleptic sensitivity were significantly associated with VHs in AD. AD patients with VHs tended to have more severe neuropsychiatric symptoms and greater caregiver burden.