Optical filtering affects the fiber impairments in a fiber optic transmission system and its overall efficiency. The fiber impairments like attenuation, dispersion, and nonlinearities are challenges to overcome. The optical orthogonal frequency division multiplexing (O-OFDM) modulation is among the techniques to overcome these challenges. This paper uses O-OFDM with coherent transmission (denoted as CO-OFDM) and quadrature phase shift keying (QPSK) for modulating subcarriers. It is denoted as QPSK CO-OFDM modulation. This QPSK CO-OFDM (denoted here as QC) modulation system is further modified by introducing different optical filters in its fiber optic link. Different variants of QC are created by using optical filters in its fiber link. For example, the QC_BL is the variant having an optical Bessel filter in its fiber link. Performances of each variant have been evaluated and compared in terms of BER. This comparison allows us to select a suitable filter for a particular parameter, such as dispersion, fiber length, input power, etc. The QC variant having no filter is denoted as QC_WF and is taken as a reference for comparison.