We consider learning representations of entities and relations in KBs using the neural-embedding approach. We show that most existing models, including NTN (Socher et al., 2013) and TransE (Bordes et al., 2013b), can be generalized under a unified learning framework, where entities are low-dimensional vectors learned from a neural network and relations are bilinear and/or linear mapping functions. Under this framework, we compare a variety of embedding models on the link prediction task. We show that a simple bilinear formulation achieves new state-of-the-art results for the task (achieving a top-10 accuracy of 73.2% vs. 54.7% by TransE on Freebase). Furthermore, we introduce a novel approach that utilizes the learned relation embeddings to mine logical rules such as BornInCitypa, bq ^CityInCountrypb, cq ùñ N ationalitypa, cq. We find that embeddings learned from the bilinear objective are particularly good at capturing relational semantics, and that the composition of relations is characterized by matrix multiplication. More interestingly, we demonstrate that our embedding-based rule extraction approach successfully outperforms a state-ofthe-art confidence-based rule mining approach in mining Horn rules that involve compositional reasoning. INTRODUCTIONRecent years have witnessed a rapid growth of knowledge bases (KBs) such as Freebase 1 , DBPedia (Auer et al., 2007), and YAGO (Suchanek et al., 2007). These KBs store facts about real-world entities (e.g. people, places, and things) in the form of RDF triples 2 (i.e. (subject, predicate, object)). Today's KBs are large in size. For instance, Freebase contains millions of entities and billions of facts (triples) involving a large variety of predicates (relation types). Such large-scale multirelational data provide an excellent potential for improving a wide range of tasks, from information retrieval, question answering to biological data mining.Recently, much effort has been invested in relational learning methods that can scale to large knowledge bases. Tensor factorization (e.g. (Nickel et al., 2011;) and neural-embedding-based models (e.g. (Bordes et al., 2013a;Socher et al., 2013)) are two popular kinds of approaches that learn to encode relational information using low-dimensional representations of entities and relations. These representation learning methods have shown good scalability and reasoning ability in terms of validating unseen facts given the existing KB.In this work, we focus on the study of neural-embedding models, where the representations are learned using neural networks with energy-based objectives. Recent embedding models TransE (Bordes et al., 2013b) and NTN (Socher et al., 2013) have shown state-of-the-art prediction performance compared to tensor factorization methods such as RESCAL (Nickel et al., 2012). They are similar in model forms with slight differences on the choices of entity and relation representations. Without careful comparison, it is not clear how different design choices affect the ˚Work conducted while interning at Mic...

Sézary Syndrome is a rare leukemic form of cutaneous T-cell lymphoma defined as erythroderma, adenopathy, and circulating atypical T-lymphocytes. It is rarely curable with poor prognosis. Here we present a multi-platform genomic analysis of 37 Sézary Syndrome patients that implicates dysregulation of the cell cycle checkpoint and T-cell signaling. Frequent somatic alterations were identified in TP53, CARD11, CCR4, PLCG1, CDKN2A, ARID1A, RPS6KA1, and ZEB1. Activating CCR4 and CARD11 mutations were detected in nearly a third of patients. ZEB1, a transcription repressor essential for T-cell differentiation, was deleted in over half of patients. IL32 and IL2RG were over-expressed in nearly all cases. Analysis of T-cell receptor Vβ and Vα expression revealed ongoing rearrangement of the receptors after the expansion of a malignant clone in one third of subjects. Our results demonstrate profound disruption of key signaling pathways in Sézary Syndrome and suggest potential targets for novel therapies.

Whereas people learn many different types of knowledge from diverse experiences over many years, and become better learners over time, most current machine learning systems are much more narrow, learning just a single function or data model based on statistical analysis of a single data set. We suggest that people learn better than computers precisely because of this difference, and we suggest a key direction for machine learning research is to develop software architectures that enable intelligent agents to also learn many types of knowledge, continuously over many years, and to become better learners over time. In this paper we define more precisely this never-ending learning paradigm for machine learning, and we present one case study: the Never-Ending Language Learner (NELL), which achieves a number of the desired properties of a never-ending learner. NELL has been learning to read the Web 24hrs/ day since January 2010, and so far has acquired a knowledge base with 120mn diverse, confidence-weighted beliefs (e.g., servedWith(tea,biscuits)), while learning thousands of interrelated functions that continually improve its reading competence over time. NELL has also learned to reason over its knowledge base to infer new beliefs it has not yet read from those it has, and NELL is inventing new relational predicates to extend the ontology it uses to represent beliefs. We describe the design of NELL, experimental results illustrating its behavior, and discuss both its successes and shortcomings as a case study in never-ending learning. NELL can be tracked online at http://rtw.ml.cmu.edu, and followed on Twitter at @CMUNELL. 2. RELATED WORK Previous research has considered the problem of designing machine learning agents that persist over long periods research highlights

Events and entities are closely related; entities are often actors or participants in events and events without entities are uncommon. The interpretation of events and entities is highly contextually dependent. Existing work in information extraction typically models events separately from entities, and performs inference at the sentence level, ignoring the rest of the document. In this paper, we propose a novel approach that models the dependencies among variables of events, entities, and their relations, and performs joint inference of these variables across a document. The goal is to enable access to document-level contextual information and facilitate contextaware predictions. We demonstrate that our approach substantially outperforms the stateof-the-art methods for event extraction as well as a strong baseline for entity extraction.

This paper focuses on how to take advantage of external knowledge bases (KBs) to improve recurrent neural networks for machine reading. Traditional methods that exploit knowledge from KBs encode knowledge as discrete indicator features. Not only do these features generalize poorly, but they require task-specific feature engineering to achieve good performance. We propose KBLSTM, a novel neural model that leverages continuous representations of KBs to enhance the learning of recurrent neural networks for machine reading. To effectively integrate background knowledge with information from the currently processed text, our model employs an attention mechanism with a sentinel to adaptively decide whether to attend to background knowledge and which information from KBs is useful. Experimental results show that our model achieves accuracies that surpass the previous state-of-the-art results for both entity extraction and event extraction on the widely used ACE2005 dataset.