Nonlinear impairments caused by devices and fiber transmission links in a coherent optical communication system can severely limit its transmission distance and achievable capacity. In this paper, we propose a low-complexity pruned-convolutional-neural-network-(CNN)-based nonlinear equalizer, to compensate nonlinear signal impairments for coherent optical communication systems. By increasing the size of the effective receptive field with an 11 × 11 large convolutional kernel, the performance of feature extraction for CNNs is enhanced and the structure of the CNN is simplified. And by performing the channel-level pruning algorithm, to prune the insignificant channels, the complexity of the CNN model is dramatically reduced. These operations could save the important component of the CNN model and reduce the model width and computation amount. The performance of the proposed CNN-based nonlinear equalizer was experimentally evaluated in a 120 Gbit/s 64-quadrature-amplitude-modulation (64-QAM) coherent optical communication system over 375 km of standard single-mode fiber (SSMF). The experimental results showed that, compared to a CNN-based nonlinear equalizer with a 6 × 6 normal convolutional kernel, the proposed CNN-based nonlinear equalizer with an 11 × 11 large convolutional kernel, after channel-level pruning, saved approximately 15.5% space complexity and 43.1% time complexity, without degrading the equalization performance. The proposed low-complexity pruned-CNN-based nonlinear equalizer has great potential for application in realistic devices and holds promising prospects for coherent optical communication systems.

The strong stochastic nonlinear impairment induced by random mode coupling appears to be a long-standing performance-limiting problem in the orbital angular momentum (OAM) mode division multiplexing (MDM) of intensity modulation direct detection (IM/DD) transmission systems. In this Letter, we propose a Bayesian neural network (BNN) nonlinear equalizer for an OAM-MDM IM/DD transmission with three modes. Unlike conventional Volterra and convolutional neural network (CNN) equalizers with fixed weight coefficients, the weights and biases of the BNN nonlinear equalizer are regarded as probability distributions, which can accurately match the stochastic nonlinear model of the OAM-MDM. The BNN nonlinear equalizer is capable of adaptively updating its weights and biases sample-by-sample, according to the probability distribution. An experiment was conducted on a 300-Gbit/s PAM8 signal with three modes over a 2.6-km OAM-MDM RCF transmission. The experimental results demonstrate that the proposed BNN nonlinear equalizer exhibits promising solutions to effectively mitigate nonlinear distortions, which outperforms conventional Volterra and CNN equalizers with receiver sensitivity improvements of 1.0 dBm and 2.5 dBm, respectively, under hard-decision forward error correction (HD-FEC) thresholds. Moreover, compared with the Volterra and CNN equalizers, the complexity of the OAM-MDM is significantly improved through the BNN nonlinear equalizer. The proposed BNN nonlinear equalizer is a promising candidate for the high capacity inter-data center interconnects.

No abstract