Melodic intonation therapy (MIT) positively impacts the speech function of patients suffering from aphasia and strokes. Fixed-pitch melodies and phrases formulated in MIT provide the key to the target language to open the language pathway. This randomized controlled trial compared the effects of music therapy-based MIT and speech therapy on patients with non-fluent aphasia. The former is more effective in the recovery of language function in patients with aphasia. Forty-two participants were enrolled in the study, and 40 patients were registered. The participants were randomly assigned to two groups: the intervention group (n = 20; 16 males, 4 females; 52.90 ± 9.08 years), which received MIT, and the control group (n = 20; 15 males, 5 females; 54.05 ± 10.81 years), which received speech therapy. The intervention group received MIT treatment for 30 min/day, five times a week for 8 weeks, and the control group received identical sessions of speech therapy for 30 min/day, five times a week for 8 weeks. Each participant of the group was assessed by a Boston Diagnostic Aphasia Examination (BDAE) at the baseline (t1, before the start of the experiment), and after 8 weeks (t2, the experiment was finished). The Hamilton Anxiety Scale (HAMA) and Hamilton Depression Scale (HAMD) were also measured on the time points. The best medical care of the two groups is the same. Two-way ANOVA analysis of variance was used only for data detection. In the spontaneous speech (information), the listening comprehension (right or wrong, word recognition, and sequential order) and repetitions of the intervention group were significantly higher than the control group in terms of the cumulative effect of time and the difference between groups after 8 weeks. The intervention group has a significant time effect in fluency, but the results after 8 weeks were not significantly different from those in the control group. In terms of naming, the intervention group was much better than the control group in spontaneous naming. Regarding object naming, reaction naming, and sentence completing, the intervention group showed a strong time accumulation effect. Still, the results after 8 weeks were not significantly different from those in the control group. These results indicate that, compared with speech therapy, MIT based on music therapy is a more effective musical activity and is effective and valuable for the recovery of speech function in patients with non-fluent aphasia. As a more professional non-traumatic treatment method, MIT conducted by qualified music therapists requires deeper cooperation between doctors and music therapists to improve rehabilitating patients with aphasia. The Ethics Committee of the China Rehabilitation Research Center approved this study (Approval No. 2020-013-1 on April 1, 2020) and was registered with the Chinese Clinical Trial Registry (Registration number: Clinical Trials ChiCTR2000037871) on September 3, 2020.