On2Vec : Embedding-based Relation Prediction for Ontology Population

Lecture Notes in Computer Science

Mao

et al. 2020

Self Cite

Analyzing the readability of articles has been an important sociolinguistic task. Addressing this task is necessary to the automatic recommendation of appropriate articles to readers with different comprehension abilities, and it further benefits education systems, web information systems, and digital libraries. Current methods for assessing readability employ empirical measures or statistical learning techniques that are limited by their ability to characterize complex patterns such as article structures and semantic meanings of sentences. In this paper, we propose a new and comprehensive framework which uses a hierarchical self-attention model to analyze document readability. In this model, measurements of sentence-level difficulty are captured along with the semantic meanings of each sentence. Additionally, the sentence-level features are incorporated to characterize the overall readability of an article with consideration of article structures. We evaluate our proposed approach on three widely-used benchmark datasets against several strong baseline approaches. Experimental results show that our proposed method achieves the state-of-the-art performance on estimating the readability for various web articles and literature.C. Meng-This work was done during the summer internships of CM and MC at Google, Mountain View. We thank the anonymous reviewers for their insightful comments.

Section: Discussionmentioning

confidence: 99%

ReadNet: A Hierarchical Transformer Framework for Web Article Readability Analysis

Meng

Lecture Notes in Computer Science

Mao

et al. 2020

Self Cite

“…With regard to the two domains of relational knowledge (proteins and gene ontology) G p and G o , we denote the learning objective losses as L Gp K and L Go K . Hierarchy-aware Encoding Regularization As mentioned in Section 2.1, it is observed that some ontological knowledge can form hierarchies [8], which is typically constituted by a relation with the implicit hierarchical property, such as "subclass of", as substructures. In gene ontology, more than 50% of the triples have such relations.…”

Section: Knowledge Modelmentioning

confidence: 99%

Bio-JOIE: Joint Representation Learning of Biological Knowledge Bases

Hao

et al. 2020

Preprint

Self Cite

The widespread of Coronavirus has led to a worldwide pandemic with a high mortality rate. Currently, the knowledge accumulated from different studies about this virus is very limited. Leveraging a wide-range of biological knowledge, such as gene ontology and protein-protein interaction (PPI) networks from other closely related species presents a vital approach to infer the molecular impact of a new species. In this paper, we propose the transferred multi-relational embedding model Bio-JOIE to capture the knowledge of gene ontology and PPI networks, which demonstrates superb capability in modeling the SARS-CoV-2-human protein interactions. Bio-JOIE jointly trains two model components. The knowledge model encodes the relational facts from the protein and GO domains into separated embedding spaces, using a hierarchy-aware encoding technique employed for the GO terms. On top of that, the transfer model learns a nonlinear transformation to transfer the knowledge of PPIs and gene ontology annotations across their embedding spaces. By leveraging only structured knowledge, Bio-JOIE significantly outperforms existing state-of-the-art methods in PPI type prediction on multiple species. Furthermore, we also demonstrate the potential of leveraging the learned representations on clustering proteins with enzymatic function into enzyme commission families. Finally, we show that Bio-JOIE can accurately identify PPIs between the SARS-CoV-2 proteins and human proteins, providing valuable insights for advancing research on this new disease.

“…They have provided an efficient and systematic solution to various knowledge-driven machine learning tasks, such as relation extraction [40], question answering [3], dialogues agents [14], knowledge alignment [6] and visual semantic labeling [10]. Existing embedding models, however, are limited to only one single view, either on the instance-view graph [2,25,42] or on the ontology-view graph [5,13].…”

Section: Introductionmentioning

confidence: 99%

Universal Representation Learning of Knowledge Bases by Jointly Embedding Instances and Ontological Concepts

Hao

Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery &Amp; Data Mining

et al. 2019

Self Cite

143

Many large-scale knowledge bases simultaneously represent two views of knowledge graphs (KGs): an ontology view for abstract and commonsense concepts, and an instance view for specific entities that are instantiated from ontological concepts. Existing KG embedding models, however, merely focus on representing one of the two views alone. In this paper, we propose a novel two-view KG embedding model, JOIE, with the goal to produce better knowledge embedding and enable new applications that rely on multi-view knowledge. JOIE employs both cross-view and intra-view modeling that learn on multiple facets of the knowledge base. The cross-view association model is learned to bridge the embeddings of ontological concepts and their corresponding instance-view entities. The intra-view models are trained to capture the structured knowledge of instance and ontology views in separate embedding spaces, with a hierarchy-aware encoding technique enabled for ontologies with hierarchies. We explore multiple representation techniques for the two model components and investigate with nine variants of JOIE. Our model is trained on large-scale knowledge bases that consist of massive instances and their corresponding ontological concepts connected via a (small) set of cross-view links. Experimental results on public datasets show that the best variant of JOIE significantly outperforms previous models on instance-view triple prediction task as well as ontology population on ontologyview KG. In addition, our model successfully extends the use of KG embeddings to entity typing with promising performance. CCS CONCEPTS• Computing methodologies → Knowledge representation and reasoning; Semantic networks; Ontology engineering.