Five-phase permanent magnet synchronous motors (PMSM) have special applications in which highly accurate speed and torque control of the motor are a strong requirement. Direct Torque Control (DTC) is a suitable method for the driver structure of these motors. If in this method, instead of using a common five-phase voltage source inverter, a three-phase to five-phase matrix converter is used, the low-frequency current harmonics and the high torque ripple are limited, and an improved input power factor is obtained. Because the input voltages of such converters are directly supplied by input three-phase supply voltages, an imbalance in the voltages will cause problems such as unbalanced stator currents and electromagnetic torque fluctuations. In this paper, a new method is introduced to remove speed and torque oscillator factors. For this purpose, motor torque equations were developed and the oscillation components created by the unbalanced source voltage, determined. Then, using the active and reactive power reference generator, the controller power reference was adjusted in such a way that the electromagnetic torque of the motor did not change. By this means, a number of features including speed, torque, and flux of the motor were improved in terms of the above-mentioned conditions. Simulations were analyzed using Matlab/Simulink software.