“…The implementation detail of α-PACE is displayed in Appendix A.2, and we compare α-PACE with two categories of competitive baselines. The first category is the previous state-of-the-art (SOTA) baselines, such as BERT (Devlin et al, 2019), RoBERTa (Liu et al, 2019), DeBERTa (He et al, 2021), McQueen (Mitra et al, 2019), MHKA (Paul et al, 2020), ege-RoBERTa-large (Du et al, 2021), L2R 2 (Zhu et al, 2020), IMSL (Li et al, 2021a), UNIMO (Li et al, 2021b), and UNICORN (T5) (Lourie et al, 2021). Another category of baselines is T5-based models, and we include the fine-tuned T5 model to illustrate the performance gain of our model.…”