Transceiver Impairments Compensation via Deep Learning for High Baud-Rate Coherent Systems

Abstract

In this paper, we propose a transceiver impairments compensation method employing deep learning equalization for high baud-rate coherent optical systems. The method is based on a deep cascade-forward neural network. The performance evaluation of the nonlinear equalizer was carried out through numerical simulations based on back-to-back optical transmission considering a 1.2 Tb/s line rate single wavelength (DP-16QAM at 150 GBd). The results indicate that the proposed equalization achieves optical signal-to-noise ratio (OSNR) gains equal to 0.5 and 2 dB compared with the conventional deep feed-forward neural network and linear cases, respectively. The proposed equalizer also presents data rate gains, compared with the conventional deep neural network and linear, respectively, equal to 50 and 150 Gb/s, in the low OSNR regime, and 10 and 70 Gb/s, for the high OSNR regime. Moreover, the impact of equalizer architecture aspects is analyzed. The simulation results confirm that the proposed equalization technique is a good solution to mitigate linear and nonlinear transceiver distortions enabling the next generation of 1 Tb/s coherent modules.