Pre-processing of paleogenomes: Mitigating reference bias and postmortem damage in ancient genome data

ABSTRACT Ancient DNA analysis is subject to various technical challenges, including bias towards the reference allele (“reference bias”), postmortem damage (PMD) that confounds real variants, and limited coverage. Here, we conduct a systematic comparison of alternative approaches against reference bias and against PMD. To reduce reference bias, we either (a) mask variable sites before alignment or (b) align the data to a graph genome representing all variable sites. Compared to alignment to the linear reference genome, both masking and graph alignment effectively remove allelic bias when using simulated or real ancient human genome data, but only if sequencing data is available in FASTQ or unfiltered BAM format. Reference bias remains indelible in quality-filtered BAM files and in 1240K-capture data. We next study three approaches to overcome postmortem damage: (a) trimming, (b) rescaling base qualities, and (c) a new algorithm we present here, bamRefine , which masks only PMD-vulnerable polymorphic sites. We find that bamRefine is optimal in increasing the number of genotyped loci up to 20% compared to trimming and in improving accuracy compared to rescaling. We propose graph alignment coupled with bamRefine to minimise data loss and bias. We also urge the paleogenomics community to publish FASTQ files.