This communication addresses, theoretically and experimentally, the effect of the cross polarization modulation (XPolM) phenomenon on the performance of a wavelength converter based on a four wave mixing process (FWM) developed in a semiconductor optical amplifier (SOA). The main studied parameter for evaluating the performance of the wavelength converter is the conversion efficiency, whose degradation, provoked by the XPolM, is analyzed when the four wave mixing process is developed with input signals linearly co-polarized at 0, 45, 90, and -45 degrees. The analysis is based on the theoretical and experimental determinations of the polarization mismatch of the pump and probe signals. Finally, it is demonstrated that classic models lacking the inclusion of the XPolM effect can reflect a significant error in the estimation of the conversion efficiency of a wavelength converter based on FWM.