Superconducting transmon qubits with fixed frequencies are widely used in many applications due to their advantages of better coherence and less control lines as compared to the frequency tunable qubits. However, any uncontrolled interactions with the qubits such as the two-level systems could lead to adverse impacts, degrading the qubit coherence and inducing crosstalk. To mitigate the detrimental effect from uncontrolled interactions between qubits and defect modes in fixedfrequency transmon qubits, we propose and demonstrate an active approach using an off-resonance microwave drive to dress the qubit and induce the ac-Stark shift on the qubit frequency. We show experimentally that the qubit frequency can be tuned well away from the defect mode so that the impact on qubit coherence is greatly reduced while maintaining the universal controls of the qubit initialization, readout, and single-qubit gate operations. Our approach provides an effective way for tuning the qubit frequency and suppressing the detrimental effect from the defect modes that happen to be located close to the qubit frequency.