“…Many protein embeddings have been proposed, including ProtVec [4], SeqVec [10], PRoBERTa [22], MSA-transformer [27], ESM-1b and ESM2 [28], and the models in the ProtTrans project: ProtTXL, ProtBert, ProtXLNet, ProtAlbert, ProtElectra, and ProtT5 [8]. These protein embeddings have many important applications to various areas, such as: structure prediction [29, 33, 15], function prediction [16, 9, 17], interaction site prediction [13, 35, 12], etc. In many applications they help achieving state-of-the-art performance.…”