Long-term continuous exposure to potentially inappropriate medications (PIMs) can lead to adverse events in the elderly. However, the effects of long-term exposure of the elderly to PIM and the relationship between PIM and chronic diseases remain unclear. The objective of this study was to investigate the continuous use of PIMs in a community-dwelling elderly population. A cross-sectional population-based study was conducted using community pharmacy–filed dispensing records from the Hcare system. Twenty-three community pharmacies were sampled from 2013 to 2015 to obtain records of patients above 65 years-old with continuous prescriptions. PIM were identified according to the 2015 Beers Criteria. The prevalence of patients using PIM was highest in patients with co-morbid mental disorders (40.05%), followed by neurological system disorders (28.91%). Patients who were prescribed a PIM were more than three times as likely to have a mental disorder as those (odds ratio 3.16, 95% confidence interval: 3.06–3.28) with non-chronic diseases. The most prescribed PIM agents were central nervous system drugs (53.16%), and benzodiazepines (35.15%). Patients with mental disorders had the highest rate of long-term persistent PIM exposure, with benzodiazepines being the most frequently dispensed. Drug safety concerns should be closely monitored in elderly patients with the abovementioned conditions.