GenVarLoader: An accelerated dataloader for applying deep learning to personalized genomics

Abstract Deep learning sequence models trained on personalized genomics can improve variant effect prediction, however, applications of these models are limited by computational requirements for storing and reading large datasets. We address this with GenVarLoader, which stores personalized genomic data in new memory-mapped formats with optimal data locality to achieve ∼1,000x faster throughput and ∼2,000x better compression compared to existing alternatives. Competing Interest Statement The authors have declared no competing interest. Copyright The copyright holder for this preprint is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is made available under a CC-BY 4.0 International license.