Background The elderly constitute a large fraction of patients with type 2 diabetes worldwide. It has been well documented that the elderly’s adherence to disease control is not adequate. The present study aimed to evaluate the impact of a mobile-based educational intervention on self-care behaviors and glycemic control among elderly with type 2 diabetes. Methods The present study was conducted on 118 older people (59 in the intervention group and 59 in the control group) with type 2 diabetes who referred to Golestan Hospital in Ahvaz, southwest of Iran in 2020. Participants were randomly divided into experimental and control groups. Data were collected at baseline and after a 3-month follow-up. At baseline, the participants completed a valid and reliable multi-section questionnaire including items on attitude, the multidimensional scale of perceived social support (MSPSS), the Coping Self-Efficacy Scale (CSES), self-care constructs, and HBA1C. After analyzing the pre-test data, we designed a training program which was offered to the intervention group online via mobile phone in three online sessions. The control group, however, received no intervention except diabetes routine care. Data were analyzed using SPSS-15 at a significance level of 0.05. Results Before the intervention, the mean scores of CSES, attitudes towards self-care, MSPSS, and self-care were not statistically significant between study groups (P > 0.05), but after intervention, the study found significant differences between the groups in terms of CSES, attitude, MSPSS, and self-care (P = 0.001). Furthermore, after implementation of the intervention, the mean value of HbA1C in the intervention group was significantly less than that of the control group (7.00 vs. 7.32%) (P = 0.001). Conclusion The present results indicated that implementing an educational intervention via mobile phone can improve self-care practice and reduce HbA1C in the elderly with type 2 diabetes. The study also showed a moderate to large effect on the outcome variables. However, further studies with longer follow-up periods are recommended to confirm the results.