A rigorous benchmarking of alignment-based HLA typing algorithms for RNA-seq data

Abstract

Accurate identification of human leukocyte antigen (HLA) alleles is essential for various clinical and research applications, such as transplant matching and drug sensitivities. Recent advances in RNA-seq technology have made it possible to impute HLA types from high throughput sequencing data, spurring the development of a large number of computational HLA typing tools. However, the relative performance of these tools is unknown, limiting the ability for clinical and biomedical research to make informed choices regarding which tools to use. Here, we rigorously compare the performance of 9 HLA callers on 652 RNA-seq samples across 6 datasets with molecularly defined gold standard. We find that OptiType has the highest accuracy at both low and high resolution with an accuracy above 99%, followed by arcasHLA and seq2HLA with accuracies above 96%. Despite OptiType’s high accuracy, it is only capable of Class I predictions, thereby limiting its application to clinical procedures like transplantation requiring Class II predictions. Furthermore, our findings reveal significant variation in accuracy across each HLA locus, with HLA-A exhibiting the highest accuracy and HLA-DRB1 exhibiting the lowest accuracy. We also find that class II genes are generally more challenging to impute than class I genes, with most typing algorithms capable of making Class I predictions to >97% accuracy whereas the best Class II tool predicts with 94.2% accuracy. Moreover, we identify notable differences in the computational resources necessary to run each tool. We find that the most computationally expensive tools are OptiType and HLA-HD which require 105and 102times greater RAM and CPU, respectively, than the least computationally expensive tools, seq2HLA and RNA2HLA. Furthermore, all tools have decreased accuracy for African samples with respect to European samples at four digit resolution. We conclude that RNA-Seq HLA callers are capable of returning high-quality results, but the tools that offer a good balance between accuracy, consistency, and computational expensiveness are yet to be developed.

Locityper: targeted genotyping of complex polymorphic genes

“…Multiple specialized tools have been developed for genotyping the MHC locus 19,20,33 ; the newest of them being T1K 21 , a state-of-the-art 34 genotyper for HLA and KIR genes that is capable of processing whole genome and whole exome short read sequencing data. To compare T1K and Locityper accuracy, we genotyped 40 Illumina HPRC WGS datasets at 23 genes and 13 pseudo genes from the MHC locus as well as 6 genes and 3 pseudogenes from the KIR locus, all combined into 25 target loci with sum length slightly over 1 Mb.…”

Full resolution HLA and KIR genes annotation for human genome assemblies

“…We in parBcular focused on the results from T1K (Song et al 2023). We did not evaluate other genotypers because most of them only work for classical HLA genes, could not genotype KIR genes, and have been evaluated in other benchmark studies (Klasberg et al 2019;Liu et al 2021;Thuesen et al 2022;Yu et al 2023). We carefully evaluated T1K for each of HLA and KIR genes (Figure 6), and also summarized in different categories (Supplemental Table S5).…”