Universal Sentence Encoder for English

Abstract

We present models for encoding sentences into embedding vectors that specifically target transfer learning to other NLP tasks. The models are efficient and result in accurate performance on diverse transfer tasks. Two variants of the encoding models allow for trade-offs between accuracy and compute resources. For both variants, we investigate and report the relationship between model complexity, resource consumption, the availability of transfer task training data, and task performance. Comparisons are made with baselines that use word level transfer learning via pretrained word embeddings as well as baselines do not use any transfer learning. We find that transfer learning using sentence embeddings tends to outperform word level transfer. With transfer learning via sentence embeddings, we observe surprisingly good performance with minimal amounts of supervised training data for a transfer task. We obtain encouraging results on Word Embedding Association Tests (WEAT) targeted at detecting model bias. Our pre-trained sentence encoding models are made freely available for download and on TF Hub.

NRF2 drug repurposing using a question-answer artificial intelligence system

“…Biological question-answer systems employing AI could reduce the resources needed for the compound identification phase. AI has revolutionized text mining in the context of question-answer systems [6]. Manual extraction of data is biased and time-consuming [7].…”

“…Employing semantic similarity techniques have the potential to solve this problem. Nevertheless, current models suffer from low accuracy To tackle this problem, an AI approach was recently proposed [6]. Using a universal sentence encoder, researchers were capable of encoding text into high dimensional vectors (HDV) using a syntactic composition function [11].…”

“…Nevertheless, current models suffer from low accuracy To tackle this problem, an AI approach was recently proposed [6]. Using a universal sentence encoder, researchers were capable of encoding text into high dimensional vectors (HDV) using a syntactic composition function [11]. HDV can be used for text classification, semantic similarity, clustering and other natural language tasks with considerably higher accuracy [6].…”

“…Using a universal sentence encoder, researchers were capable of encoding text into high dimensional vectors (HDV) using a syntactic composition function [11]. HDV can be used for text classification, semantic similarity, clustering and other natural language tasks with considerably higher accuracy [6]. However, the full potential of applying AI in QA systems for drug discovery has not been achieved yet.…”

“…Google universal encoder uses a deep averaging network (DAN) [28]as its composition function. The primary advantage of the DAN encoder is that compute time is linear to the length of the input sequence [6], [19]. By this approach, input embedding for words and bi-grams are first averaged together and then passed through a feedforward deep neural network to calculate sentence embedding [18].…”