In this paper, an enhanced voltage sorting algorithm is proposed for balancing each sub module capacitor voltage in modular multilevel converter (MMC). However, the submodule (SM) voltage strategy becomes more complex when it increases to a number of levels, which eventually increases voltage ripples and circulating current. In order to overcome this problem, an enhanced voltage sorting algorithm is proposed, which implies an enhanced control logic function for capacitor voltage balancing of the converter. The switching pattern of each SM floating capacitor is determined based on the modulating signal methodology, which is then compared with the hybrid state condition using arm current direction in the MMC. Additionally, the generated hybrid signals are once again compared with the dynamic implicit number by conjoining the switching logic state variables. Due to this comparison of logic function, the capacitor voltage is balanced properly and reduces the capacitor ripple voltage to a permissible limit compared to the conventional method. Hence, compared to other balancing techniques, the circulating current is reduced by using conventional control techniques. The desired pulse signals for the power devices are obtained based on the multi carrier pulse width modulation techniques. The superior performance of the proposed algorithm is tested using MATLAB/Simulink software and real time laboratory hardware setup for different load conditions. Apart from this, the effective performance of the proposed method has been validated in the HVDC transmission system.