The complementary integration of high spectrally efficient modulation formats, coherent receivers, and advanced electronic subsystems, has resulted in a revolution in the spectral efficiency of optical communications. This letter experimentally investigates the impact of varying the characteristics of high speed CMOS‐DAC on the performance of optical DP‐MQAM systems. We particularly consider two operating conditions, referred as over‐ and under‐sampling. In the former case, we relax the CMOS circuit; however limit the maximum transmitted baud rate (i.e., 32 GBd for 64 GSa/s). In the second regime, we stress the DAC in order to generate less than two samples for each symbol, hence increase the transmitted baud rate at the expense of higher signal bandwidth. Under the first regime, our experimental results show a measured EVM and BER of 4.7%rms and 5 × 10−2, respectively, for DP‐128QAM/32GBd. In the second regime, we obtain a variable baud rate ranging from 36 up to 56 GBd using a sampling ratio starting from 1.8 and decreasing down to 1.14, respectively, with maximum modulation order of QPSK at 56 GBd. © 2016 Wiley Periodicals, Inc. Microwave Opt Technol Lett 58:1128–1131, 2016