A novel MTP and MCMR approach for predicting early lung cancer using Longitudinal CT imaging

The prediction of pulmonary nodules being canceration is of vital importance to the diagnosis and treatment of early lung adenocar-cinoma. However, most traditional research paradigms only focus on image data at a single point in time, which makes it easy to ignore the correlation of lung computer tomography (CT) images at multiple points in time. Based on the the correlation between longitudinal images and medical characteristics of patients with pulmonary nodules, a novel multimodal feature analysis framework is proposed in this paper. Firstly, we collect 925 patients with pulmonary nodules from cooperative hospitals, utilize a Convolutional Long Short Term Memory (ConvLSTM) to extract the deep features of pulmonary nod-ules at dierent time points, and radiomics tool kits to extract their radiomics features. After that, the deep features, radiomics features, and risk factor are fused by Multimodal Trilinear Pooling (MTP), and the optimal feature is obtained using the Maximum Correlation and Minimum Redundancy (MCMR) selection. In the end, a total of 260-dimensional features were selected. The validity of the extracted features was veried by the leave-one-out cross-validation method on the data set. The experiment shows that the accuracy of nodule cancera-tion prediction reached 89.15%, which proved the eectiveness of the method proposed in this study. The multimodal nodule feature analysis framework can provide reference values for nodule canceration prediction.