Scalable parametric encoding of multiple modalities

A bstract A flexible model is introduced which shares ideas with the Autoencoder, Canonical Correlation Analysis, Singular Value Decomposition, and Procrustes Analysis. It is proposed to find relevant maps to transform multiple datasets of various types from one modality to another. Here, the Generative Encoder is used to transform spatial gene expression from breast tissue, to the images of histology tissue measured with Spatial Transcriptomics. The model is directly interpretable given all parameters are linked to the data space. It is scalable on Big Data, training reasonably on several thousand RGB images of 100 by 100 pixels in under an hour, which equates to 30,000 pixel features per sample image.