Overhung rotors are important for use in industrial turbo-machines. The effects of a lateral force can increase as a result of the rotor weight, misalignment, or the operating speed of the suspension system for which the rotor is carrying a transmission connection. In this paper, the reduction of vibration in supported lateral directions by varying control is discussed in a radial active magnetic bearing system (AMBS). An experimental test was conducted on the orbital response of an overhung rotor supported by an AMBS, to provide an alternative for improving precision. To simplify the system design, decoupling was achieved using a PID controller and harmonic disturbance compensator (HDC), which improved the rotating performance of an overhung active magnetic bearing (AMB) rotor system, using a frequency response function (FRF) approach and a description of the overhung rotor during normal operational conditions at unique frequencies. The experimental results show that the precision rotation, due to harmonic excitation of the shaft orbit, can be removed in real time using compensation signals using trigonometry. The compensation criteria for the changed run-up and coast-down consistently helped to maintain the rotational center in a central position. A reduction of up to 55% in vibration amplitude on average was achieved under appropriate conditions, and the significance of the overhung rotor symptoms faults were investigated.