Noisy samples-robust Neural Network-based Equalizer for Coherent Optical Transmission Nonlinearity Compensation

Abstract Nonlinear impairments introduced by optical/electrical components in coherent transceivers (CO-TRx) and optical fiber are the primary bottleneck for enhancing optical transmission capacity. As most nonlinearity equalizers (NLEs) are training data-driven, they can provide significant gain under high-quality training signals but only a weak gain under poor ones. This limitation becomes noteworthy for high-speed transmission systems characterized by high-baud rate and high-order modulation formats resulting in signals with relatively low signal-to-noise ratio, which are susceptible to penalties from various impairments. In response to this issue, we propose a novel approach known as noisy samples-robust neural network-based nonlinearity equalizer (NS-NNLE). With the combination of a pre-trained model, a filter network for noise feature filtration, and a main network for nonlinear mapping, the proposed NS-NNLE can be trained efficiently using noisy training sequences and achieve more stable generalizability than common NLEs. We experimentally demonstrate its performance in a dual-polarization (DP) 64 GBaud 16QAM signals back-to-back (B2B) coherent optical transmission and a DP-40GBaud 16QAM signals 110 km fiber transmission. Experimental results illustrate that the proposed NS-NNLE can provide a stable gain at an acceptable complexity cost under various test conditions where conventional NLEs struggle to achieve adequate training, both for CO-TRx and fiber-induced nonlinearity