Joint High-Dimensional Soft Bit Estimation and Quantization using Deep Learning

Abstract Forward error correction using soft probability estimates is a central component in modern digital communication receivers and impacts end-to-end system performance. In this work, we introduce EQ-Net: a deep learning approach for joint soft bit estimation and quantization in high-dimensional multiple-input multiple-output (MIMO) systems. We propose a two-stage algorithm that uses soft bit quantization as pre-training for estimation, and is motivated by a theoretical analysis of soft bit compression bounds in MIMO channels. Our experiments demonstrate that a single deep learning model achieves competitive results on both tasks when compared to previous methods, with gains in quantization efficiency as high as 20% and reduced estimation latency by at least 21% compared to other deep learning approaches that achieve the same end-to-end performance. We investigate the robustness of the proposed approach, and demonstrate that the model is robust to distributional shifts when used for quantization, and is competitive with state-of-the-art deep learning approaches when faced with channel estimation errors in the task of soft bit estimation.