The ongoing COVID-19 pandemic is believed to have caused a sharp increase in the incidence of elder abuse (EA), including as a result of isolation, social distance combined with increased interpersonal stressors. Thus, the aim of this study is to determine the impact of the COVID-19 pandemic on the elder abuse rates and the characteristics of risk factors. A total of 347 patients hospitalized in the Department of Neurology and Department of Geriatrics at University Hospital No. 1 in Bydgoszcz were selected as subjects for the analysis. The tools used in the study are: Authors-Designed Questionnaire, the Geriatric Depression Scale and the Activities of Daily Living Scale. Descriptive statistics, chi-squared tests, and logistic regression analyses were used. In the studied population, nearly 45% of the elderly were victims of violence. This represents an increase of more than 6 percent compared to the pre-pandemic. The most common type of EA was psychological abuse (72.3%). In the final models, the risk factors include, among others, low income (OR = 3.60, 95% CI = 1.93–6.72), chronic diseases (OR = 2.06, 95% CI = 1.28–3.31), poor relationship with the family (OR = 3.26, 95% CI = 1.96–5.43), and moderate and severe depression (OR = 18.29, 95% CI = 10.24–32.69; OR = 18.49, 95% CI = 3.91–87.30, respectively). Moreover, moderate functional impairment 5.52 times more often and severe functional impairment 21.07 times more likely to predispose to EA. People who suffered from COVID-19 are 1.59 times more likely to be victims of EA (95% CI = 1.03–2.46). In this study, we saw significant increases in EA rates during the COVID-19 pandemic.