Rarefaction is better than robust Aitchison PCA and other compositional data analysis methods at controlling for uneven sequencing effort

Amplicon sequencing typically results in a wide distribution in the number of sequences obtained from each sample. How best to account for this variation has been a persistent problem in the microbial ecology literature. Historically, rarefaction was been used in ecology and this practice was adopted by microbial ecologists. But rarefaction has been strongly criticized leading to the development of compositional data analysis and normalization methods. Therefore, I reassessed the benchmarking data that was generated by the developers of one such method, robust Aitchison PCA. I found numerous problems in the Python code that led to the support of robust Aitchison PCA. These problems extended to the creation of simulated datasets, implementation of machine learning methods, and choice of analysis parameters. Furthermore, the analysis of the simulated and case study datasets was done in a manner that was foreign to standard microbiome analyses. I corrected the problems in the original code, added datasets with smaller effect sizes, and expanded the collection of methods that purport to correct for uneven sequencing effort. Contrary to the claims of the original analysis, robust Aitchison PCA was not insensitive to uneven sequencing effort and did not perform as well as rarefaction. In fact, even using the benchmarking framework from the original analysis, rarefaction outperformed robust Aitchison PCA, other compositional data analysis methods, and other normalization methods. Rarefaction remains the preferred method of controlling for uneven sampling effort in amplicon sequence studies. Importance Efforts to connect the the structure of microbial communities with environmental processes and host health have captured widespread interest among scientists and the general public. The methods used to generate the sequencing data that are fundamental to these efforts result in wide variation in the number of sequences per sample. This variation and how to account for it can have detrimental effects on the ability to draw valid conclusions. Compositional data analysis methods including robust Aitchison PCA have grown in popularity for mitigating these effects and have been proposed as alternatives to rarefaction. In this study, I reviewed and fixed the code that was used to benchmark robust Aitchison PCA and expanded the analysis. With this improved and expanded analysis, I found that rarefaction was still superior to robust Aitchison PCA and other methods of controlling for uneven sequencing effort.