Resonance frequencies of a mechanical system are the most important parameters that determine the dynamic characteristics of the system. When the frequencies of the harmonic forces under the influence of the system coincide with the resonance (natural) frequencies of the system, a resonance situation occurs and the mechanical system vibrates with high amplitudes. This can cause serious damage to the system by causing effects such as break, fatigue, impact, noise on the bearings, connections and elements in the system. It is possible to eliminate some resonance frequencies of mechanical systems in an examined frequency range using pole-zero cancellation methods by shifting a resonance frequency of the system to an anti-resonance frequency or an anti-resonance frequency to a resonance frequency. Thus, a solution can be obtained for some vibration problems. In this study, a new method using the Sherman-Morrison formula is presented to eliminate some of the resonance frequencies for some FRFs (Frequency Response Functions) of mechanical systems with pole-zero cancellation. In order to eliminate a selected resonance frequency of the system in any FRF determined using the presented method, a mass (kg) and a grounded spring (N/m) modification are made and the required modification values for this are calculated by solving the obtained non-linear equation set numerically. With the proposed method, it is aimed to contribute to the literature by presenting an alternative approach for the solution of vibration problems by cancelling some selected resonance frequencies. The main highlight of the method is there is no need of a matrix inversion for calculating the required modification values. This provides a very fast solution. In addition, the proposed method uses directly the FRFs of the active coordinates (response, excitation and modification) only, which makes the method very useful for practical engineering applications.