Discrete Stochastic Search and Its Application to Feature-Selection for Deep Relational Machines

Dash, Tirtharaj; Srinivasan, Ashwin; Joshi, Ramprasad; Baskar, A.

doi:10.1007/978-3-030-30484-3_3

Cited by 13 publications

(22 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…6 would seem to suggest that the answer is "yes" (Table 3). However, we caution against drawing such a conclusion for at least the following reasons: (a) Inclusion of propositions based on stochastic sampling of complex relational features in (Dash et al, 2019) can result in significantly better DRM models. It is possible also that BCP could be augmented with the same sampling methods to yield more informative propositions; (b) It is also possible that a BotGNN obtained with access to sampled relational features could improve its performance over what is shown here.…”

Section: Some Additional Resultsmentioning

confidence: 99%

See 1 more Smart Citation

Inclusion of domain-knowledge into GNNs using mode-directed inverse entailment

2021

Self Cite

View full text Add to dashboard Cite

We present a general technique for constructing Graph Neural Networks (GNNs) capable of using multi-relational domain knowledge. The technique is based on mode-directed inverse entailment (MDIE) developed in Inductive Logic Programming (ILP). Given a data instance e and background knowledge B, MDIE identifies a most-specific logical formula ⊥ B (e) that contains all the relational information in B that is related to e. We represent ⊥ B (e) by a "bottom-graph" that can be converted into a form suitable for GNN implementations. This transformation allows a principled way of incorporating generic background knowledge into GNNs: we use the term 'BotGNN' for this form of graph neural networks. For several GNN variants, using real-world datasets with substantial background knowledge, we show that BotGNNs perform significantly better than both GNNs without background knowledge and a recently proposed simplified technique for including domain knowledge into GNNs. We also provide experimental evidence comparing BotGNNs favourably to multi-layer perceptrons that use features representing a "propositionalised" form of the background knowledge; and BotGNNs to a standard ILP based on the use of most-specific clauses. Taken together, these results point to BotGNNs as capable of combining the computational efficacy of GNNs with the representational versatility of ILP.

show abstract

Section: Some Additional Resultsmentioning

confidence: 99%

“…These features can form the input feature-vectors for standard statistical models (as in (Saha et al, (2012)) or for multilayer perceptrons (MLPs). When used with MLPs, the resulting model is called as "deep relational machines" or DRMs, as introduced by Lodhi (2013) and studied extensively in (Dash et al, 2018) and (Dash et al, 2019). In Fig.…”

Section: Resultsmentioning

confidence: 99%

Inclusion of domain-knowledge into GNNs using mode-directed inverse entailment

2021

Self Cite

View full text Add to dashboard Cite

show abstract

“…It is a technique to transform a relational representation into a propositional single-table representation where each column in the table corresponds a feature that represents a relation constructed from data and domain-knowledge. Propositionalisation is the core technique in construction of deep relational machines [34,35,36]: these are multi-layered perceptrons constructed from propositionalised representation of relational data and domain-knowledge. Recent studies on domain-knowledge inclusion include construction of graph neural networks (GNNs) that can learn not only from relational (graph-structured) data but also symbolic domain-knowledge.…”

Section: Related Workmentioning

confidence: 99%

Using Domain-Knowledge to Assist Lead Discovery in Early-Stage Drug Design

Dash

Srinivasan

Vig³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

We are interested in generating new small molecules which could act as inhibitors of a biological target, when there is limited prior information on target-specific inhibitors. This form of drug-design is assuming increasing importance with the advent of new disease threats for which known chemicals only provide limited information about target inhibition. In this paper, we propose the combined use of deep neural networks and Inductive Logic Programming (ILP) that allows the use of symbolic domain-knowledge (B) to explore the large space of possible molecules. Assuming molecules and their activities to be instances of random variables X and Y, the problem is to draw instances from the conditional distribution of X, given Y, B (DX|Y,B). We decompose this into the constituent parts of obtaining the distributions DX|B and DY |X,B, and describe the design and implementation of models to approximate the distributions. The design consists of generators (to approximate DX|B and DX|Y,B) and a discriminator (to approximate DY |X,B). We investigate our approach using the well-studied problem of inhibitors for the Janus kinase (JAK) class of proteins. We assume first that if no data on inhibitors are available for a target protein (JAK2), but a small numbers of inhibitors are known for homologous proteins (JAK1, JAK3 and TYK2). We show that the inclusion of relational domainknowledge results in a potentially more effective generator of inhibitors than simple random sampling from the space of molecules or a generator without access to symbolic relations. The results suggest a way of combining symbolic domain-knowledge and deep generative models to constrain the exploration of the chemical space of molecules, when there is limited information on target-inhibitors. We also show how samples from the conditional generator can be used to identify potentially novel target inhibitors.

show abstract

“…Again, stochastic feature selection of relational features that are used as input to deep relational machines can improve the performance and interpretability (Dash et al. 2019 ).…”

Section: Related Workmentioning

confidence: 99%

“…Again, sparsity and size of the propositionalized representation is a problem for deep neural networks. Again, stochastic feature selection of relational features that are used as input to deep relational machines can improve the performance and interpretability (Dash et al 2019).…”

Section: Deep Relational Machinesmentioning

confidence: 99%

Propositionalization and embeddings: two sides of the same coin

2020

View full text Add to dashboard Cite

Data preprocessing is an important component of machine learning pipelines, which requires ample time and resources. An integral part of preprocessing is data transformation into the format required by a given learning algorithm. This paper outlines some of the modern data processing techniques used in relational learning that enable data fusion from different input data types and formats into a single table data representation, focusing on the propositionalization and embedding data transformation approaches. While both approaches aim at transforming data into tabular data format, they use different terminology and task definitions, are perceived to address different goals, and are used in different contexts. This paper contributes a unifying framework that allows for improved understanding of these two data transformation techniques by presenting their unified definitions, and by explaining the similarities and differences between the two approaches as variants of a unified complex data transformation task. In addition to the unifying framework, the novelty of this paper is a unifying methodology combining propositionalization and embeddings, which benefits from the advantages of both in solving complex data transformation and learning tasks. We present two efficient implementations of the unifying methodology: an instance-based PropDRM approach, and a feature-based PropStar approach to data transformation and learning, together with their empirical evaluation on several relational problems. The results show that the new algorithms can outperform existing relational learners and can solve much larger problems.

show abstract

Discrete Stochastic Search and Its Application to Feature-Selection for Deep Relational Machines

Cited by 13 publications

References 12 publications

Inclusion of domain-knowledge into GNNs using mode-directed inverse entailment

Inclusion of domain-knowledge into GNNs using mode-directed inverse entailment

Using Domain-Knowledge to Assist Lead Discovery in Early-Stage Drug Design

Propositionalization and embeddings: two sides of the same coin

Contact Info

Product

Resources

About