Unsupervised Event Graph Representation and Similarity Learning on Biomedical Literature

Abstract

The automatic extraction of biomedical events from the scientific literature has drawn keen interest in the last several years, recognizing complex and semantically rich graphical interactions otherwise buried in texts. However, very few works revolve around learning embeddings or similarity metrics for event graphs. This gap leaves biological relations unlinked and prevents the application of machine learning techniques to promote discoveries. Taking advantage of recent deep graph kernel solutions and pre-trained language models, we propose Deep Divergence Event Graph Kernels (DDEGK), an unsupervised inductive method to map events into low-dimensional vectors, preserving their structural and semantic similarities. Unlike most other systems, DDEGK operates at a graph level and does not require task-specific labels, feature engineering, or known correspondences between nodes. To this end, our solution compares events against a small set of anchor ones, trains cross-graph attention networks for drawing pairwise alignments (bolstering interpretability), and employs transformer-based models to encode continuous attributes. Extensive experiments have been done on nine biomedical datasets. We show that our learned event representations can be effectively employed in tasks such as graph classification, clustering, and visualization, also facilitating downstream semantic textual similarity. Empirical results demonstrate that DDEGK significantly outperforms other state-of-the-art methods.

Comprehensive Analysis of Knowledge Graph Embedding Techniques Benchmarked on Link Prediction

“…For instance, having dense formal representations is becoming more and more indispensable to avoid hallucinations in conversational agents and produce more factual and semantically coherent summaries. Notably, authors have devised unsupervised and inductive methods for mapping small semantic parsing graphs into low-dimensional vectors (whole-graph granularity), reflecting their structural and semantic similarities [116]. For these reasons, we find the direction of evaluating the LP on semantic parsing graphs compelling and still never-explored.…”

Efficient Memory-Enhanced Transformer for Long-Document Summarization in Low-Resource Regimes

“…Future works should explore memory writing/reading operations with structured information extracted from text, comparing unsupervised techniques for document metadata acquisition (e.g., classes [ 49 , 50 ] and entity relationships [ 51 , 52 ]) with advanced semantic parsing solutions such as event extraction [ 53 , 54 ] and abstract meaning representation, which was recently used for knowledge injection into deep neural networks [ 55 , 56 ]. The community should envisage novel graph representation learning methods [ 57 , 58 , 59 , 60 ] to densely represent multi-relational structured data following a Linked Open Data vision centered on the integration of several source knowledge graphs or relational databases via automatic entity matching [ 61 ]. Taking inspiration from biology [ 62 , 63 ] and communication networks [ 64 , 65 , 66 , 67 ], we underline the importance of managing dynamic scenarios, tracking knowledge refinements among sentences, and propagating information, which is pivotal when processing lengthy inputs.…”

JKRL: Joint Knowledge Representation Learning of Text Description and Knowledge Graph

“…The loss function for all relation description statement sets is defined as follows: (11) where 𝛾 is an interval parameter greater than 0, and T represents the set of text description statements.…”

“…Similar to the idea of unsupervised semantic-aware-based graph representation learning [11], text information can provide rich semantic resources for graph representation and play an important auxiliary role in the representation learning optimization of a graph representation model. Therefore, recent research on KG representation learning research focuses on how to use the internal information of a knowledge graph to optimize the representation of knowledge.…”