Low-Complexity Pruned Convolutional Neural Network Based Nonlinear Equalizer in Coherent Optical Communication Systems

Abstract

Nonlinear impairments caused by devices and fiber transmission links in a coherent optical communication system can severely limit its transmission distance and achievable capacity. In this paper, we propose a low-complexity pruned-convolutional-neural-network-(CNN)-based nonlinear equalizer, to compensate nonlinear signal impairments for coherent optical communication systems. By increasing the size of the effective receptive field with an 11 × 11 large convolutional kernel, the performance of feature extraction for CNNs is enhanced and the structure of the CNN is simplified. And by performing the channel-level pruning algorithm, to prune the insignificant channels, the complexity of the CNN model is dramatically reduced. These operations could save the important component of the CNN model and reduce the model width and computation amount. The performance of the proposed CNN-based nonlinear equalizer was experimentally evaluated in a 120 Gbit/s 64-quadrature-amplitude-modulation (64-QAM) coherent optical communication system over 375 km of standard single-mode fiber (SSMF). The experimental results showed that, compared to a CNN-based nonlinear equalizer with a 6 × 6 normal convolutional kernel, the proposed CNN-based nonlinear equalizer with an 11 × 11 large convolutional kernel, after channel-level pruning, saved approximately 15.5% space complexity and 43.1% time complexity, without degrading the equalization performance. The proposed low-complexity pruned-CNN-based nonlinear equalizer has great potential for application in realistic devices and holds promising prospects for coherent optical communication systems.

Noisy samples-robust Neural Network-based Equalizer for Coherent Optical Transmission Nonlinearity Compensation

“…Results in Fig. 6(e-h) show that signals equalized by the VNLE exhibit a diffuse shape in the middle and elongation at the edges, a phenomenon observed in [14]. This occurs because points at the edges receive more amplification than the middle in the calculation of higher order terms, dominating the weight in the optimization process.…”

“…This technique demonstrates a notable 3.8 dB improvement in the presence of high nonlinearity during 32Gbaud 64-QAM transmission. And in terms of fiber nonlinearity compensation, perturbation term-assisted machine learning (ML)-based equalizers currently dominate the mainstream [13][14][15]. Furthermore, [16] puts forward a numerical demonstration emphasizing that the non-Gaussian noise distribution resulting from NNLEs has a detrimental impact on the bit error rate (BER) performance.…”

Wide and Deep Learning-Aided Nonlinear Equalizer for Coherent Optical Communication Systems

“…The deep component is a convolutional neural network (CNN), as shown in the bottom right corner of Figure 1. Our previous work [23] described the structure of the CNN in detail. The dimensions of x D are mainly related to the receiving sequence.…”

“…The deep component is a bidirectional gated recurrent unit (BiGRU) neural network, as shown in the bottom right corner of Figure 2. Our previous work [23] described the structure of the BiGRU neural network in detail. The 2k + 1 input feature sequence x D is composed of the data on the I and Q components of the current M-QAM signal and its k preceding and k succeeding symbols.…”

“…Building on our previous work [23], the operation of the wide component and the full connection of the deep component-BiGRU output to the linear layer are added.…”